For Digital Agency

The Basic Act on the Promotion of Public and Private Data Utilization obligates the national government and local public bodies in Japan to advance open data initiatives. The Digital Agency is responsible for coordinating open data policies across the government.

Taking this opportunity, we have begun designing and deploying our own CKAN environment, while also advancing exploratory collaborations with local governments such as Yamaguchi Prefecture for future integration and utilization.

Kumagaya City built an FIWARE Orion-based open data catalog supported by its DX HQ, but the biggest issue is data collection from departments. Staff lack motivation due to unclear benefits, while the Digital Promotion Division manually processes CSV/Excel and handles anonymization. Utilization is still low, APIs are hard for citizens to use, and the initiative remains at an early “publish whatever can be published” stage.

We have built a catalog-based CKAN data platform that integrates data sources such as MLIT and PLATEAU’s 3D urban models. It includes custom harvesting plugins, 3D web map visualization, Playwright-based automated testing, and an OpenAI-powered intelligent assistant, and can be deployed in a ready-to-use form.

Demo: https://opendata.uixai.org/

We have built a catalog-based CKAN data platform that integrates data sources such as MLIT and PLATEAU’s 3D urban models. It includes custom harvesting plugins, 3D web map visualization, Playwright-based automated testing, and an OpenAI-powered intelligent assistant, and can be deployed in a ready-to-use form.

In CKAN, FileStore stores original files and provides downloads, DataStore loads structured data into PostgreSQL for API-based search, and Datapusher automatically ingests CSV/XLS files. In practice, a recommended setup is to store small files (<50 MB) in the DataStore for querying, and large files (>50 MB) in S3 for download-only.

CKAN mainly provides the Action API (CRUD and user management), the Harvest API (automatic acquisition and synchronization of external catalogs), and the DataStore API (querying data ingested into the DataStore). By combining these, migration, synchronization, and search can be achieved. Additional interfaces such as the Resource API and ckanext-dcat are also available.

Our platform supports multiple visualization modes: Map View displays CSV data with latitude/longitude as markers on a map; the 3D Cesium Viewer renders CityGML/3D Tiles by specifying a tileset.json; Data Graph visualizes tabular data with configurable X/Y axes and grouping; and the Timeline view generates time-series line charts without the need for external tools.

Point cloud data refers to highly accurate 3D spatial datasets captured by LiDAR or drones, consisting of X/Y/Z coordinates with attributes such as RGB and reflectance intensity. In Japan’s disaster prevention domain, surface points are extracted from LAS point clouds to generate DEMs and perform hydrological analysis for legally mandated inundation hazard and evacuation route maps, while derived datasets such as DTMs, DSMs, and slope maps are also used for landslide risk assessment.

We provide a cloud-native geospatial platform based on the STAC standard, featuring a lightweight STAC API on AWS Lambda (FastAPI), COPC/COG storage on S3 with Range-Request on-demand access, EPSG:6676/6677 support, and automatic Parquet indexing from S3 updates. Online visualization (Potree / STAC Browser) and interoperability with pystac-client / stackstac are supported, with proven use cases in precision surveying, flood simulation, and seismic risk assessment.

STAC (SpatioTemporal Asset Catalog) is a geospatial metadata standard that represents satellite imagery, LiDAR point clouds, and other spatiotemporal data in a unified JSON format for efficient search and interoperability. Using a “core + extensions” model for Items, Collections, and Catalogs, it is commonly deployed with STAC API and STAC Browser on AWS to publish COPC point clouds and COG imagery, and has been used in municipal GIS projects such as Nagoya and Yamanashi.

Demo: https://stac.uixai.org/

COPC (Cloud Optimized Point Cloud) is a cloud-friendly point-cloud format based on LAZ, featuring an internal octree index that allows partial reads via HTTP Range Requests without downloading entire files. This enables point clouds stored on S3 or CloudFront to be streamed into web viewers such as Potree for fast 3D visualization in the browser. COPC integrates well with STAC for efficient discovery, management, and sharing, and is widely used in surveying, construction, and infrastructure applications.

We ran a flood simulation for the Kasugai area using Python (rasterio/numpy). A 0.5 m DEM (EPSG:6676) around Kasugai Station (500×500 m) was processed via STAC API on AWS under five water-level scenarios (+1/+2/+3/+5/+10 m). The +3 m case yielded 13.98 ha of inundation (≈2.84 Mm³) and the +10 m case 16.62 ha (≈3.93 Mm³). Results were saved as a dual-panel image (“kasugai_flood_simulation.png”) for flood-risk and planning use.

We also provide a RESTful API on AWS Lambda for managing STAC catalogs of COPC point clouds, currently hosting three collections with 25 point-cloud tiles and DEM layers. It supports standard STAC API / OGC API Features endpoints and can be accessed directly from STAC Browser, QGIS STAC Plugin, and PDAL. Native JGD2011 (Zone 8/9) metric bbox queries are supported, and S3 STAC JSON updates automatically trigger Parquet re-indexing with a 60-second TTL cache for fast responses.