Code-speaking Manifesto: How Japan’s Public Data Stacks Up Technically

どうも〜おかむーです！ Today I want to look at how Japanese central and local government data and systems behave when you treat policy as code. エンジニア的に言うと、政策はデータとAPIで表現されるべきなんですよ。

• Governments publish lots of PDFs and policy goals, but machine-readable time series are scarce
• e-Gov and Tokyo provide APIs, but coverage and formats are inconsistent
• Technical fixes (CSV/JSON, schema, monitoring) could unlock reuse and auditability

Conclusion

Overall: there's clear momentum—Digital Agency, e-Gov APIs, Tokyo Open Data—but too much policy evidence remains buried in PDFs and ad-hoc reports. 要するに、公開はしてるけどエンジニアフレンドリーじゃないんです。API-first, schema-firstで整備すれば、KPIの追跡や市民監査が一気に楽になる。

Report: what's actually out there

What I checked

• Digital Agency pages and meeting materials about machine-readability rules (digital.go.jp, cas.go.jp)
• e-Gov API portal and catalog (e-gov.go.jp / api-catalog.e-gov.go.jp)
• Tokyo Open Data API portal (portal.data.metro.tokyo.lg.jp)
• Example local implementation: Yamaguchi prefecture PDF on "Digital Countryside" fund

これ見てくださいよ：中央はAPIカタログを公開しているけど、地方はPDFで実績を配るパターンが多い。PDFは人間には読めるけど、コードには扱いにくいんですよね。

Machine-readability: state of play

• Positive: national-level APIs exist and the government is publishing rules to ensure machine-readability for administrative data
• Negative: many KPI reports and grant performance summaries are PDFs (see Yamaguchi's project report), lacking CSV/JSON time series
• Coverage gaps: no single canonical schema for KPI events across ministries; formats vary (CSV, Excel, PDF tables)

Technical examples and quick wins

• API discovery: use e-Gov API catalog to find endpoints, then normalize responses with a small script

# normalize JSON/CSV responses into a Pandas dataframe
import requests, pandas as pd
r = requests.get('https://api-catalog.e-gov.go.jp/your-endpoint')
data = r.json()
df = pd.json_normalize(data['items'])

• PDF → structured CSV: use tabula-py or camelot to extract tables from PDFs when CSV absent

# example: extract tables from a PDF (requires Java)
tabula -o output.csv -p all input.pdf

• Define a minimal KPI JSON Schema and publish it alongside datasets so downstream apps can validate automatically. Example fields: period, region_code, measure_id, target, actual, source_url.

Policy vs. practice: KPI gaps

Guidelines for the Digital Countryside fund require setting KPIs and annual improvement (chisou.go.jp guidance), but local reports often show aggregate amounts in PDFs without machine-readable time series. That makes tracking progress programmaticly hard—you can't easily compute year-on-year achievement rates or feed dashboards.

Concrete technical proposals

Mandate an "open-kpi" JSON schema for all grant-funded programs; expose time-series via e-Gov API endpoints. 2. Convert legacy PDFs into verified CSV/JSON using an extract-validate pipeline (tabula → schema validation → publish). 3. Provide an API health dashboard (uptime, schema version, last-updated) per dataset. 4. Offer reference client libraries (Python/JS) and example notebooks for civil society.

Summary of benefits

• Faster civic analysis and independent audits
• Reproducible dashboards for policymakers and citizens
• Reduced manual rework for local governments (once pipelines exist)

まとめ

中央の仕組みはできつつあるけど、現場のデータがPDFに埋もれているのが現状。API-first、スキーマ公開、PDF→CSV自動化があれば、政策の実効性はぐっと検証しやすくなる。エンジニアの視点で言うと、これは全部"engineering problem"なんですよね。

おかむーから一言

I’ve built products that take messy government data and make useful services—so信じてほしいんです、技術で改善できる部分は山ほどあります！Let's ship APIs, not PDFs.