どうも〜おかむーです！ Today let's look at how government data and systems actually behave when engineers try to use them — a small manifesto: "code speaks louder than policy".

• Governments publish policies promising machine-readable data, but practice is mixed
• Some ministries already expose CSVs/APIs (good!), many still bury numbers in PDFs (bad)
• With pragmatic engineering fixes we can turn policy into usable data for real services

結論

The technical gap between policy and reality is real but fixable. Japan's recent decision on machine-readability rules (see digital.go.jp meeting materials, 2026-03-31) lays a good foundation — but on-the-ground datasets still vary: some endpoints are CSV/HTTP-friendly (notice.go.jp, env.go.jp, soumu.go.jp), while others remain PDF-bound. 要するに、ルールはできたけど実装と運用が鍵なんですよね。

Report: what I checked and what it means

Sources I looked at (examples)

• digital.go.jp meeting materials on machine-readability rules (PDF) — policy-level commitment
• notice.go.jp/docs/status_notice.csv — direct CSV endpoint, great example
• env.go.jp/content/900398071.csv — ministry CSV available
• soumu.go.jp/main_content/000323625.csv — national dataset CSV
• GovTech Tokyo case pages — shows dashboards and reuse patterns

これ見てくださいよ: some agencies already serve CSVs over HTTP, which is ideal for engineers. But others still publish only PDFs or embed tables in HTML. That makes automated reuse costly.

Technical issues observed

• PDF-first publications: high friction for parsing, brittle OCR or table extraction needed
• No consistent metadata: CSVs live in different schemas, different encodings (UTF-8 vs Shift_JIS), inconsistent column names
• Lack of formal API specs (OpenAPI) and no versioning strategy
• Sparse provenance and license metadata (hard to know reuse rules)

Why machine-readability rules matter (engineer perspective)

エンジニア的に言うと、machine-readable means: stable HTTP endpoints, structured formats (CSV/JSON/GeoJSON), charset consistency, schema definition, and an API contract. Without those, each consumer rebuilds a fragile ETL every time.

Concrete technical checklist (what govs should provide)

• Canonical dataset registry with machine-readable catalog (DCAT / Data Package)
• Per-dataset: schema.json (JSON Schema), license, last-updated, provenance
• REST API + bulk download link; OpenAPI spec hosted alongside
• UTF-8 encoded CSV/JSON; if legacy encodings necessary, clearly documented
• CI-backed data quality checks (schema validation, row-count sanity tests)

Code examples (practical)

Python: fetch CSV and validate schema with pandas/jsonschema

import pandas as pd
import requests
from jsonschema import validate

url = 'https://notice.go.jp/docs/status_notice.csv'
df = pd.read_csv(url)
print(df.head())

simple sanity check
assert 'notice_id' in df.columns
assert df.shape[0] > 0

Node.js: expose a small proxy API that stabilizes a CSV source

const express = require('express')
const fetch = require('node-fetch')
const csv = require('csv-parse/lib/sync')

const app = express()
app.get('/api/notices', async (req,res)=>{
const r = await fetch('https://notice.go.jp/docs/status_notice.csv')
const text = await r.text()
const records = csv(text, {columns:true})
res.json(records)
})
app.listen(3000)

These are simple building blocks — and yes, production needs caching, rate limits, and schema enforcement.

Policy vs Reality: numbers and gaps

Digital policy documents (digital.go.jp) set machine-readability as a rule. But operationally:

• Some ministries already serve CSVs and are ready for APIs (prove-by-link: env.go.jp, soumu.go.jp)
• Others still publish policy results in PDFs, making it hard to measure outcomes automatically

The measurable consequence: policy targets (like improving data reuse and dashboard coverage) can be tracked programmatically only where data is machine-readable. Where it's not, you get blind spots.

Practical roadmap for governments

Inventory: crawl gov domains (go.jp) for CSV/JSON endpoints vs PDFs

Prioritize high-impact datasets for API wrapping and schema publishing

Provide a central catalog (DCAT), with OpenAPI links and example clients

Add data-CI: automated schema checks, freshness alerts, and tests in CI pipelines

Encourage reuse by publishing SDK examples, rate-limited API keys, and datasets in both bulk and API forms

まとめ

• Policy commitments (machine-readability rules) are necessary but not sufficient
• Many agencies already provide CSVs — leverage those as early wins
• Engineering fixes are straightforward: standardize encodings, publish schemas, add lightweight APIs and CI
• Do this and you turn policy into real, testable, reusable data that powers services and accountability

おかむーから一言

テクノロジーで社会をアップデートするのは現場の積み重ねです！データ公開をコードで語る文化、ぜひ一緒に作りましょう！