https://github.com/Qwant/qwantmaps
Vector Tiles #
All the data are imported in a PostgreSQL database using imposm. All the world's vector tiles are then generated using kartotherian's tilerator and stored in Cassandra.
- Imposm : An importer for OpenStreetMap data. It reads PBF files and imports the data into PosgreSQL/PostGIS database. It is developed by Omniscale and is released as open source.
- Omniscale, Germany : "We provide commercial OpenStreetMap services since 2009."
- PBF Format (Protocolbuffer Binary Format), an alternative to the XML format. It is about half of the size of a gzipped OSM planet file and about 30% smaller than a bzipped planet. It is also 5x faster to write and 6x faster to read than a gzipped planet.
- Kartotherian is a map tile service originally built for the Wikimedia projects. Its primary components include Kartotherian and Tilerator. Tilerator is a multi-processor, cluster-enabled tile generator that allows both pre-generation and dirty tile re-generation.
- Apache Cassandra : Open source NoSQL distributed database.
Steps (my guess) :
- Download ALL OpenStreetMap data in PBF format. The actual OSM data is stored on XML format. PBF (protocolbuffer) is XML alternative with better compression and faster read/write operation.
- Import PBF OpenStreetMap data into PostgreSQL by using imposm.
- Generate vector tile from PostgreSQL by using Wikimedia Kartotherian's tilerator.
- Vector tile could be represented as JSON file, i guess. So, it could also be stored in NoSQL distributed database, like Cassandra.
Map Updates #
"after the initial import, a dedicated task is used to read the OSM differential updates and apply the changes in the PostgreSQL database on a daily basis."
"We update the remaining geocoder data (administrative regions, streets and addresses) on a monthly basis."