Hi, I wrote a small test script to give you more help. The following results are reproducible:
'<way id="39990889">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=8.509576000000006,49.34158799999985,8.559576000000007,49.39158799999985][@meta]' '<way id="92738103">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=8.509576000000006,49.34158799999985,8.559576000000007,49.39158799999985][@meta]' '<way id="24685758">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=8.559576000000007,49.34158799999985,8.609576000000008,49.39158799999985][@meta]' '<way id="99176827">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=8.75957600000001,47.44158799999996,8.80957600000001,47.49158799999996][@meta]' '<way id="11261866">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=8.959576000000013,48.74158799999989,9.009576000000013,48.791587999999884][@meta]' '<way id="22781417">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.009576000000013,48.74158799999989,9.059576000000014,48.791587999999884][@meta]' '<way id="10597166">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.059576000000014,48.4915879999999,9.109576000000015,48.5415879999999][@meta]' '<way id="29363076">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.059576000000014,48.4915879999999,9.109576000000015,48.5415879999999][@meta]' '<way id="8092073">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.109576000000015,48.84158799999988,9.159576000000015,48.89158799999988][@meta]' '<way id="24059413">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.159576000000015,48.4415879999999,9.209576000000016,48.4915879999999][@meta]' '<way id="62473930">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.159576000000015,48.4415879999999,9.209576000000016,48.4915879999999][@meta]' '<way id="65385528">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.159576000000015,48.4415879999999,9.209576000000016,48.4915879999999][@meta]' '<way id="32895245">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.159576000000015,48.4915879999999,9.209576000000016,48.5415879999999][@meta]' '<way id="52035311">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=9.209576000000016,48.4415879999999,9.259576000000017,48.4915879999999][@meta]' '<way id="134372194">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=10.809576000000039,48.391587999999906,10.85957600000004,48.4415879999999][@meta]' '<way id="38245700">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=10.85957600000004,49.841587999999824,10.90957600000004,49.89158799999982][@meta]' '<way id="38245704">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=10.85957600000004,49.841587999999824,10.90957600000004,49.89158799999982][@meta]' '<way id="27287573">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=11.009576000000042,49.54158799999984,11.059576000000042,49.59158799999984][@meta]' '<way id="52640474">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=11.359576000000047,48.4415879999999,11.409576000000047,48.4915879999999][@meta]' '<way id="28996953">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=11.55957600000005,49.89158799999982,11.60957600000005,49.94158799999982][@meta]' '<way id="5024389">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=13.059576000000071,50.64158799999978,13.109576000000072,50.691587999999776][@meta]' '<way id="25762694">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=13.059576000000071,50.64158799999978,13.109576000000072,50.691587999999776][@meta]' '<way id="30844055">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=13.509576000000077,48.29158799999991,13.559576000000078,48.34158799999991][@meta]' '<way id="84378596">' looks wrong at query 'http://www.overpass-api.de/api/xapi?*[bbox=14.309576000000089,51.34158799999974,14.35957600000009,51.391587999999736][@meta]' Yours Rüdiger -----Ursprüngliche Nachricht----- Von: Gubler, Ruediger [mailto:[email protected]] Gesendet: Donnerstag, 27. Oktober 2011 08:20 An: 'Roland Olbricht' Cc: [email protected] Betreff: Re: [OSM-dev] Overpass API delivers randomly invalid data Hi, we tried to download a huge area which is done with a groovy script. The download starts at 25.10.11 15:09:37 GMT+2 and the error was detected at 26.10.11 01:50:28 GMT+2 I can't say which paricular query is failed but i have a node ID wich was wrong: 679111231 The error is reproducable but it will cause at changing elements (nodes, ways, relations) and different ID's. There are only a few elements broken. The surrounding elements are OK. Here is the download part of our sript: MIN_X_Property = 8.059576 MIN_Y_Property = 46.741588 MAX_X_Property = 14.670413 MAX_Y_Property = 51.430882 CHUNK_SIZE = 0.05 PrintWriter pw = new PrintWriter(raw_unsortedFile, "UTF-8") pw.println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>") pw.println("<osm version=\"0.6\">") processErrorStatus = createJobsAndDownloadData(MIN_X_Property, MAX_X_Property, MIN_Y_Property, MAX_Y_Property, CHUNK_SIZE, pw, Master_Logfile) pw.println("</osm>") pw.close() def int createJobsAndDownloadData(BigDecimal min_x, BigDecimal max_x, BigDecimal min_y, BigDecimal max_y, BigDecimal chunk_size, PrintWriter destinationPrintWriter, File masterLogFile) { // Define a closure to log to the console as well as in our logfile def logALine = { String logMessage -> println(logMessage) masterLogFile.append(logMessage + "\r\n") } def int processErrorStatus = 0; // let's start double job_min_x = min_x double job_max_x = min_x + chunk_size boolean break_x = false //long count_x = 0L while(job_max_x <= max_x + chunk_size && !break_x) { if(job_max_x > max_x) { job_max_x = max_x break_x = true } double job_min_y = min_y double job_max_y = min_y + chunk_size boolean break_y = false //long count_y = 0L while(job_max_y <= max_y + chunk_size && !break_y) { if(job_max_y > max_y) { job_max_y = max_y break_y = true } URL api_url = null try { api_url = new URL("http://www.overpass-api.de/api/xapi?*[bbox="; + job_min_x + "," + job_min_y + "," + job_max_x + "," + job_max_y + "][@meta]"); logALine("Calling: " + api_url.toString()) URLConnection yc = api_url.openConnection() BufferedReader br = new BufferedReader(new InputStreamReader(yc.getInputStream(), "UTF-8")) String inputLine; while ((inputLine = br.readLine()) != null) if(!inputLine.contains("<osm")) if(!inputLine.contains("</osm")) if(!inputLine.contains("<?xml")) if(!inputLine.contains("<bound")) if(!inputLine.contains("<note")) if(!inputLine.contains("<meta")) destinationPrintWriter.println(inputLine) br.close() // wait to jolly OSM server along Thread.sleep(1000) } catch(e) { processErrorStatus = -1 logALine ("Error at URL: " + api_url.toString()) } if(processErrorStatus != 0) { return processErrorStatus } //count_y++ job_min_y = job_max_y job_max_y = job_max_y + chunk_size } //count_x++ job_min_x = job_max_x job_max_x = job_max_x + chunk_size } return processErrorStatus } Yours Rüdiger -----Ursprüngliche Nachricht----- Von: Roland Olbricht [mailto:[email protected]] Gesendet: Mittwoch, 26. Oktober 2011 19:20 An: [email protected] Cc: Gubler, Ruediger Betreff: Re: [OSM-dev] Overpass API delivers randomly invalid data Hi, > We are querying data for a bounding box and randomly entries are delivered > as > <way id="30134048"> > or > <node id="31881194" lat="47.5137295" lon="11.3250178"/> > > The entries are cut before the version attribute. If we query the entries > with the id all attributes are delivered. I've tried to reproduce this. But all changesets I have receieved have complete meta data. So could you please give more detailed information, in particular - At what time as exactly as possible have you tried to download? There have been an outage on Sunday including a backlog for meta data until Monday evening, and I would need to figure out whether it is related to this or not. - Which query exactly failed? - Does is fail sometimes or reproducably? - Are in a damaged response no meta data at all or is it partly missing? Best regards, Roland _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
