Search for HOT projects, news, people and jobs.
News — 03 October, 2017

Export Tool | 02 Data File Formats

This blog is the second in a series of ‘Learn Export Tool’ posts and focuses on the file formats available for OSM data to be converted to. Following the launch of the revamped tool on the 18th September 2017, the first Learn blog focused on ‘Selecting Area of Interest’. The other two posts to follow in the series will cover how to ‘Customise Map Features’ and examples of ‘Applying Exported Data’.

The Humanitarian OpenStreetMap Team (HOT) Export Tool allows users to create custom OpenStreetMap (OSM) extracts anywhere in the world, by selecting an area of interest, map features and file formats. Within minutes, up-to-date OSM data is exported, filtered and converted.


The tool allows OSM data to be extracted through the Overpass API in its native Protocol Buffer Binary (PBF) file format, before filtering the data for the map features and associated tags specified by the user. Once the data has been filtered it is then converted into the file formats chosen by the user.  


This blog is the second in a series of ‘Learn Export Tool’ posts following the launch of the revamped tool on the  18th September 2017. The first blog of the series was on ‘Selecting Area of Interest’ (September 25th), with another two to follow, covering how to ‘Customise Map Features’ (October 9th) and examples of ‘Applying Exported Data’ (October 23rd).


Currently the tool can convert OSM data into Shapefile .shp, GeoPackage .gpkg, Garmin .img, Google Earth .kml, OSM .xml, OSM .pbf, MAPS.ME .mwm, OsmAnd .obf, and MBTiles .mbtiles. At least one of the file formats must be selected to create the export, but there is no restriction as to how many can be chosen, which can be done by ticking multiple boxes.



Shapefile .shp

Shapefiles are a tabular format developed by Esri. They are the most popular file format for GIS data. A shapefile is actually 3-4 individual files, commonly bundled together as a ZIP archive. Shapefiles doe have certain limitations, such as the file size, capping at 2 gigabytes (GB) and a column name length limit of 10 characters. Read more about Shapefiles .shp in the Learn section. 

Figure 1. Shapefile Files



Geopackage .gpkg

OGC Geopackages store geospatial data in a single SQLite database. Geopackages are very similar to Spatialite-enabled SQLite databases. They should be usable in most major GIS applications. Geopackages support practically unlimited file sizes and numbers of columns in tables, and have full support for Unicode. They're especially ideal if you need to run SQL queries over the data. Read more about Geopackages.gpkg in the Learn section.

Figure 2. SQLite Database



Garmin .img

A .IMG file contains all information needed to render a map onto a Garmin GPS mobile device. Please note, that the cartographic .img style and feature choices of the map are not dependent on the feature selection submitted to the Export Tool - instead, a default style based on all OSM data is used. Read more about Garmin .img in the Learn section.


Figure 3. Garmin GPS Device



Google Earth .kml

The Keyhole Markup Language (KML) is an XML-based format for modeling points, lines, polygons and associated attributes geographically. Google Earth is the most widely known earther browsing platforms using KML, which is also the reason why it was developed. Read more about Google Earth .kml in the Learn section.


Figure 4. Google Earth .kml



OSM .pbf

The canonical data format of OSM is an XML document of nodes, ways and relations. The Protocol Buffer Binary Format (PBF) is an optimised representation of OSM XML, which is smaller on disk and faster to read. This format is only compatible with OSM specific tools, such as OSM editing software. Each .PBF provided by the export tool should be referentially complete - that is, any node, way or relation referenced by a way or relation will appear in the PBF. Learn more about OSM .pbf in the Learn section.

OSM .xml

An uncompressed, human-readable version of the OSM .PBF, as described above.  

    Figure 5. XML Code Example



MAPS.ME .mwm is a GPS Navigation and map application for Android and iOS smartphones and tablets, notably supporting offline mapping and navigation. Read more about MAPS.ME .mwm in the Learn section. To use a custom export with on Android, please follow these steps:

  1. Open and navigate to your region of interest

  2. Accept's prompt and download the proffered region

  3. Force close

  4. Create an MWM export

  5. Download the export, unzip it, and copy the .mwm file to your device

  6. Using the Android File Manager, navigate to the location containing the .mwm file

  7. Long-press to select it and touch the "copy" or "cut" button

  8. Navigate to "MapsWithMe" and open the highest numbered folder (e.g. 170917)

  9. Copy/move your .mwm file into this directory by tapping the "paste" button

  10. Delete the existing .mwm file for your region of interest, taking note of its filename

  11. Rename your .mwm file to match the region that was downloaded by (which you just deleted) by long-pressing (to select) and tapping the "more" button (3 vertical dots)

  12. Open

Figure 6. MAPS.ME

MBTiles .mbtiles

MBTiles is a file format for storing map tiles in a single file. The Export Tool allows users to create MBTiles containing tiles from OSM, which can be used as sources of offline context within applications that support them. Please note that MBTiles will extract all the OSM features in the selected area of interest through the Export Tool, with the ‘3 Data’ tab automatically switching to a dropdown source option and zoom range, in place of the standard tree tag and YAML feature selection option. Read more about MBTiles .mbtiles in the Learn section.

Figure 7. MBTiles


Additional file formats are continuously being suggested and added to the Export Tool. If there is a file format that you would like to see added in the future, please make a comment on the GitHub repository. For further detailed information regarding each file format type, please visit the ‘File Formats’ page in the ‘Learn’ section on the tool website.