どうも〜おかむーです！今日はちょっとエンジニアっぽい話をしますよ〜

• Government data is increasingly available on go.jp domains, but formats and APIs vary a lot.
• Standardization efforts (Digital Agency / Soumu) are moving things forward, yet practical gaps remain in machine-readability and traceability.
• With a few engineering changes (OpenAPI, DCAT, consistent CSV/JSON schemas) Japan’s open data could be far more actionable.

結論

Public datasets on go.jp (and related domains) are trending toward better openness — e.g., Japan Dashboard and e-Stat — but the real bottleneck is engineering hygiene: inconsistent formats (PDF vs CSV), missing APIs, unclear schemas and versioning. 要するに、政策を“コードで語る”には、データの機械可読性とAPI設計の質を上げる必要があるということです。

Report: what I looked at and what it means

Sources examined (代表例)

• Digital Agency: Japan Dashboard initiative (digital.go.jp) — a central visualization push.
• Ministry of Internal Affairs and Communications (Soumu):自治体情報システムの標準化・共通化 (soumu.go.jp) — standardization PMO and reporting tools.
• e-Stat:統計ダッシュボード (dashboard.e-stat.go.jp) — official stats + API ecosystem.
• Direct CSV endpoints observed (examples): notice.go.jp/docs/status_notice.csv, various ministry CSVs under env.go.jp / mhlw.go.jp / soumu.go.jp.

これ見てくださいよ：いくつかのデータはちゃんとCSVで公開されている（good）、でも別の重要文書はPDFに埋め込まれたまま（bad）というケースがまだ多いんです！

Technical diagnosis

• Format heterogeneity: some datasets provide CSV files (notice.go.jp/...csv), others only PDFs or HTML tables. PDF → CSV is brittle and loses schema metadata. 要するに、PDFは人間向けで、コードには向かないということです。
• API coverage: e-Stat offers APIs and structured endpoints — this is the gold standard inside gov. But many ministry datasets lack RESTful JSON endpoints or OpenAPI specs, making integration ad-hoc.
• Schema and identifiers: local government systems reference different codes (JIS codes, local IDs) but mapping and canonicalization are inconsistent. That complicates joins across datasets (e.g., budget vs procurement vs service-level stats).
• Versioning & provenance: few datasets expose clear versioning or change logs. For reproducible policy analysis you want stable dataset URIs + changelogs.

Concrete code example

This is the kind of snippet engineers would drop into a notebook to standardize CSV ingestion (Python/pandas):

import pandas as pd
from urllib.request import urlopen

url = 'https://notice.go.jp/docs/status_notice.csv'
df = pd.read_csv(url, encoding='utf-8')
normalize column names
df.columns = [c.strip().lower().replace(' ', '_') for c in df.columns]
coerce codes to strings for stable joins
df['jis_code'] = df['jis_code'].astype(str).str.zfill(5)

print(df.head())

要するに、データ取り込みでやるべきことはカラム正規化、IDの標準化、型安全の確保なんですよね。

Policy gaps quantified

• Standardization policy exists: Soumu’s 「自治体情報システムの標準化・共通化」 describes PMO tools and migration paths. But the execution often leaves datasets siloed per municipality.
• Dashboard vs raw data: Japan Dashboard (Digital Agency) is great for visibility, but dashboards without raw, documented APIs limit reusability by civic tech teams.
• KPI vs results: many policies announce numeric targets (e.g., migration to standard-conformant systems) but public progress is reported in heterogeneous PMO tools and occasional CSV exports — hard to automate compliance checks across 1,700+ municipalities.

Improvement proposals (engineering-first)

Machine-first publication: prefer JSON/CSV with accompanying JSON Schema and sample records over PDFs.

Publish OpenAPI specs for every REST endpoint (even simple CSV endpoints can be wrapped). This enables client generation and easier audits.

Adopt DCAT/Data Catalog standards across ministries with a central catalog (Digital Agency) exposing dataset metadata, update frequency, provenance and license.

Enforce canonical identifiers (JIS codes, municipality IDs) and publish mapping tables for legacy IDs.

Dataset versioning + changelog endpoint (e.g., /datasets/{id}/versions) to allow reproducible analyses.

Provide example ingestion notebooks (collected on GitHub) and a sandbox API key for civic developers.

まとめ

• 日本の政府データは量・種類ともに揃ってきたんですけど、エンジニア視点ではまだ“接着剤”が必要な状態です！
• PDFをやめて、OpenAPI + JSON Schema + DCATで整備すれば、政策の検証が爆速でできるようになります。要するに、コードで語れる政策にするにはデータとAPIの品質向上が肝心なんです。

おかむーから一言

テクノロジーで社会をアップデートするの、マジでできると思ってます！まずはOpenAPI一本通して、次にデータカタログ作りましょう。エンジニアの力で政治をデバッグしていくぞ〜！