How to translate OpenStreetMaps to other languages?

OpenStreetMap.org  is a wonderful community driven maps portal. We can create maps in our own language too.

Here is an Image of Tamil Map.

 

Demo

https://api.mapbox.com/v4/srikanthlogic.714e671e/page.html?access_token=pk.eyJ1Ijoic3Jpa2FudGhsb2dpYyIsImEiOiJuQ1RYS3pjIn0.7YUMcAQAc4A7T703-yAu2g#4/13.03/80.07

I am much inspired by the Tamil maps provided by OSM. Only thing we have to do is to translate all the strings to tamil.

I had a discussion on this with my friend Srikanth Logic long time back. Here is the chat notes from that discussion.

Srikanth Logic

Srikanth Logic

How to translate the strings on OSM to Tamil?

  1. We have to select a region in OSM
  2. Query for the nodes/paths that dont have tamil translation
  3. Get them in a google sheet
  4. Translate the strings to Tamil using Google Translate
  5. Translate the strings using wikidata (not implemented)
  6. Verify the translations manually and fix any errors
  7. Upload the translations to OSM.

 

1. Open google sheets
Tools -> Script Editor

//var langCode ='ta'; — TODO Make it language independent.
function doGet() {
return HtmlService.createTemplateFromFile('Index.html')
.evaluate();
}
function doSomething() {
Logger.log('I was called!');
}
function OverpassToGoogleSheets() {
Logger.clear();
//Show web popup of map?
//Suggest using Wikidata | Xlate memory?.
//Allow verification.
//Allow download of josm / other upload file.
//Explore using OSM API to directly upload.
//OverpassQueryBuilder();
//var query = GetOverpassQueryFromUser()
//var query = '[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node["place"]["name"~"."]["name:ta"!~"."];);out meta;';
var query = '[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node["amenity"="hospital"]["name"~"."]["name:ta"!~"."];);out meta;'
//var query = '[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node["highway"]["name"~"."]["name:ta"!~"."];);out meta;'
//var query = 'https://overpass-api.de/api/convert?data=%5Bout%3Ajson%5D%5Btimeout%3A360%5D%3B%2F%2F%20gather%20results(%20%20area%5Bname%3D%22Tamil%20Nadu%22%5D%3B%20%20%2F%2F%20query%20for%20%22place%22%20%22without%20name%3Ata%22%20tags%20%20node%5B%22place%22%5D(area)%5B%22name%3Ata%22!~%22.%22%5D(area)%3B)%3B%2F%2F%20print%20resultsout%20meta%3B%3E%3Bout%20meta%20qt%3B&target=compact'
//var query = '[out:json][timeout:3600];(node["railway"="station"](7.8586434,76.2256046,13.5375733,80.682728)["name:ta"!~"."](7.8586434,76.2256046,13.5375733,80.682728););out meta;>;out meta qt;';
//var query = '[out:json][timeout:3600];(node["railway"="station"](7.8586434,76.2256046,13.5375733,80.682728)["name:ta"!~"."](7.8586434,76.2256046,13.5375733,80.682728););out meta;>;out meta qt;';
//Chennai Highway nodes – tertiary'[out:json][timeout:3600];(way["highway"="tertiary"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;';
//Chennai Highway nodes – secondary[out:json][timeout:3600];(way["highway"="secondary"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;
//Chennai Highway nodes'[out:json][timeout:3600];(node["highway"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;';
//Tamil Nadu Railway stations'[out:json][timeout:3600];(node["railway"="station"](7.8586434,76.2256046,13.5375733,80.682728)["name:ta"!~"."](7.8586434,76.2256046,13.5375733,80.682728););out meta;>;out meta qt;'
//Chennai Railway stations'[out:json][timeout:3600];(node["railway"="station"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;'
//- Chennai Suburbs '[out:json][timeout:3600];(node["place"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;';
//'[out:json][timeout:3600];(node["place"]({{geocodeBbox:Chennai}})["name:ta"!~"."]({{geocodeBbox:Chennai}}););out meta;>;out meta qt;'
//var query = '[out:json][timeout:3600];(area[name="Andhra Pradesh"];node["place"](area)["name:te"!~"."](area););out meta;>;out meta qt;'
//var query = '[out:json][timeout:3600];(node["place"]["name:ta"!~"."](5.916667,79.516667,9.833333,81.866667););out meta;>;out meta qt;';
//var query = '[out:json][timeout:3600];(node["place"~"town"]["name:ta"!~"."]({{geocodeBbox:India}}););out meta;>;out meta qt;';
//area[name="Troisdorf"];way(area)[highway][name];out;
//var query = '[out:json][timeout:3600];([highway]["name:ta"!~"."]({{geocodeBbox:Chennai}}););out meta;>;out meta qt;';
//var query = '[out:json][timeout:3600];({{geocodeArea:India}}->.searchArea;node["place"~"city"]["name:te"!~"."](area.searchArea););out meta;'
var jsonresponse = QueryOverpass(query);
ParseReponsePopulateSheets(jsonresponse);
}
function ParseReponsePopulateSheets(jsonresponse) {
var doc = SpreadsheetApp.getActiveSpreadsheet();
if (doc.getSheetByName('OverpassOutput') != null)
doc.deleteSheet(doc.getSheetByName('OverpassOutput'));
var sheet = doc.insertSheet('OverpassOutput');
var OSMdataAll = JSON.parse(jsonresponse.getContentText());
var data = OSMdataAll.elements;
var OSMdata = [];
var count = 0;
var OSMNodedata = [];
sheet.getRange(2,1,1,7).setValues([[["Node ID"],["Name"],["Lat"],["Lon"],["Type"],["Google Translate"],["Wikidata Translate"]]]);
sheet.getRange(2,1,1,7).setFontFamily("Arial Bold");
for(element in OSMdataAll.elements) {
OSMdata = [];
OSMNodedata['id'] = "=HYPERLINK(CONCATENATE(\"www.openstreetmap.org/node/\"," + data[element]['id'] + ")," + data[element]['id'] + ")"
OSMNodedata['name'] = data[element]['tags']['name'];
OSMNodedata['lat'] = data[element]['lat'];
OSMNodedata['lon'] = data[element]['lon'];
OSMNodedata['place'] = data[element]['tags']['place'];
OSMNodedata['googx'] = "=GOOGLETRANSLATE(\"" + OSMNodedata['name'] + "\",\"en\",\"ta\")";
//OSMNodedata['wikix'] = QueryWikidata(OSMNodedata['name']);
OSMNodedata['wikix'] = "=INDEX(WIKITRANSLATE(CONCATENATE(\"en:" + OSMNodedata['name'] + "\"),\"ta\",true),1,1)"
//TODO Add xliterate option
OSMdata.push([OSMNodedata['id'],OSMNodedata['name'],OSMNodedata['lat'],OSMNodedata['lon'],OSMNodedata['place'],OSMNodedata['googx'],OSMNodedata['wikix']]);
count++;
sheet.getRange(2+count,1,1,7).setValues(OSMdata);
}
}
function OverpassQueryBuilder() {
//Generate Overpass Queries using common query template and defined inputs?
/*
* City / Area
* Target Language
* Untranslated / Review / All
* Node / Relation / Ways / POIs
* Filter
*/
}
function GetOverpassQueryFromUser() {
var ui = SpreadsheetApp.getUi();
var response = ui.prompt('Overpass Query', 'Input OverpassQuery with json output', ui.ButtonSet.OK_CANCEL);
if (response.getSelectedButton() == ui.Button.OK) {
query = response.getResponseText();
Logger.log(query);
}
return query;
}
function QueryOverpass(query) {
//Logger.clear();
var url = 'http://overpass.osm.rambler.ru/cgi//interpreter';
var options = {
'method': 'post',
'payload': query
//'muteHttpExceptions': false
};
var jsonresponse = UrlFetchApp.fetch(url, options);
Logger.log(jsonresponse);
Logger.log('End of QueryOverpass');
return jsonresponse;
}
function PopulateWikidataColumn() {
var doc = SpreadsheetApp.getActiveSpreadsheet();
var places = doc.getActiveSheet().getRange(1693,2,600,1);
Logger.clear();
var wikidataplaces = [];
for (place in places.getValues()) {
wikidataplaces.push([QueryWikidata(places.getValues()[place])]);
}
doc.getActiveSheet().getRange(1693,7,wikidataplaces.length,1).setValues(wikidataplaces);
}
/*function QueryWikidata(query) {
var HEADERS = { headers: {'X-User-Agent': 'OSM Translation using Google Spreadsheets'} };
var url = 'https://www.wikidata.org/w/api.php?action=wbgetentities&sites=enwiki&titles=' + query + '&languages=ta&props=labels&format=json';
var jsonresponse = UrlFetchApp.fetch(url, HEADERS);
var data = JSON.parse(jsonresponse.getContentText());
for (entity in data.entities) {
if(data.entities[entity].hasOwnProperty('labels')) {
if (data.entities[entity].labels.hasOwnProperty('ta')) {
return data.entities[entity].labels.ta.value;
}
}
}
return "";
}*/

2. copy paste in to script editor.

In the script editor, we need to chose OverpassToGoogleSheets on “Select function” dropdown
This is entry point for the script

var query = '[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node["amenity"="hospital"]["name"~"."]["name:ta"!~"."];);out meta;'

This is the overpass query.
basically, we can keep changing the overpass query.
run the same script, it will dump output into sheets that we opened.

There are lot of queries which I have executed in past in comments
These may not be of help (as they may have already been translated, but just for reference.

if everything works, you just need to change query as per data needs.
QueryOverpass –> This function actually posts the overpass query as a parameter to Overpass server, gets response from OSM database.

In case, you dont get data, check if you get response from server http://overpass.osm.rambler.ru/

You can change server in case its down. http://wiki.openstreetmap.org/wiki/Overpass_API#Introduction all other servers are shown here.

ParseReponsePopulateSheets –> After getting a successful response from OSM, next this function will actually parse data and populate sheets

It reads element by element in json, prepares an array for 1 row; (OSM Node ID, Name, Lat, Lon, Place, Google Translate suggestion, Wikidata Translate suggestion
=GOOGLETRANSLATE is used for populating google translate
QueryWikidata function is used for populating wikidata suggestion

In case you dont see data on sheets; View -> Log (or Control Enter keyboard shortcut) to show logs of script execution Logger.log statements will log messages into log

Now, lets see query writing.

=GOOGLETRANSLATE is Google sheets native function

QueryWikidata –> At bottom of script. I have commented now, wrote PopulateWikidataColumn function.

But there one limitation of google sheets.
Every HTTP call you do from this is counted as part of quota
=GOOGLETRANSLATE consumes quota, running this script (which makes HTTP calls to overpass server consumes quota)

Google has some strange limit (~2000). But we are dealing with say 500 strings in output. 500 Google Translate will eat quota for single query itself.

This is across sheets (not just for this), so I keep exhausting my quota even as I open some of these sheets. So, use these for smaller queries, dont open / close browser tab (these sheets) too often.

we should use separate gmail account for this.

Quota gets reset in a day.

Okay, Now query building.
http://overpass-turbo.eu/ is the place to play with; Its a query browser

var query = '[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node["amenity"="hospital"]["name"~"."]["name:ta"!~"."];);out meta;'

Lets take this query as example. Its a simple query. We can write even more complex queries.

The first two tags tell output must be in json and 3600 secs is timeout, which you are willing to wait. In case of large response, bbox specifies geo-coords within which you want to restrict the search.

that box is roughly main chennai

https://wiki.openstreetmap.org/wiki/Bbox
You can change bbox as needed.

(node[“amenity”=”hospital”][“name”~”.”][“name:ta”!~”.”];);

Here we say select all nodes, which has a key “amenity” and corresponding value as “hospital”

[] [], default behavior is AND. So, all nodes with amentiy value as hospital AND “name” as anything (. is for any value, like *) and “name:ta” is not equal to . (any value); Not equal to any is usually empty / null

[out:json][timeout:3600][bbox:12.9201721,80.1238331,13.2401721,80.4438331];(node[“railway”][“name”~”.”][“name:ta”!~”.”];);out meta;

This is all railway related nodes having some name in english but no tamil name

Open http://overpass-turbo.eu/

paste the above query
run

Will get the result in the right side map

So, steps are,

  1. translating our target data set in mind to queries, using this query browser.
  2. then run the query in overpass turbo to see what it looks like, how big / how useful the result is.
  3. then paste that query on google script, get the data populated in sheets

http://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL

Check above page for complete documentation of query language.

Then,

hospitals, pharmacies, schools, resturants, etc

think of data. run query, see data, get data in sheets, translate

roads –> Small difference is
its a way, not node

//Chennai Highway nodes - tertiary'[out:json][timeout:3600];(way["highway"="tertiary"](12.9201721,80.1238331,13.2401721,80.4438331)["name:ta"!~"."](12.9201721,80.1238331,13.2401721,80.4438331););out meta;>;out meta qt;';

But nothing changes from query / sheets perspective.
Instead of node id , you will get way id.
only while uploading back, upload script needs to take care.

Uploading to OSM after Tamil Translation

I will also give brief overview on upload script.

Please try the upload script with a new ID for a small data set; (2 translated nodes csv)
1. Create a new OSM ID
https://github.com/srikanthlogic/osm-ta/blob/master/osm-csv-names-loader.py is the script

its simple, you will get it in 2 minutes, read once..
Example csv is at https://github.com/srikanthlogic/osm-ta/blob/master/csv-example.csv

Using this script, we can upload the translations to OSM.

 

======

 

The above are not a detailed tutorial. This needs some hands on experience on OSM, OverPass, Google Sheets, Python, git, Linux, Commandline etc.

Explore these and share your thoughts.

If you are interested in creating OSM maps in Tamil or your language, mail me on tshrinivasan@gmail.com or comment here.

Let us collaborate to create maps in our own languages.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s