Re: [OSM-talk] API a lot slower?
On 2018-07-19 09:04, Roland Olbricht wrote: Hi Maarten, Nothing special really, I was just downloading the data from a lot of relations sequentially in JOSM. I downloaded relation 1360154 with its members which are bus master_relations that then also load the bus relations but without their contents, and then I selected all the bus relations and downloaded the members (in JOSM: select all relations and do "download members"). By the way: Please consider to use "Download data > from Overpass" with the query ( rel(1360154); >>; ); out meta; The runtime is expected to be 3 to 10 seconds, data is usually less than two minutes off. As a side effect, this takes load off the Main API. Thanks, that is a good tip. I am in a habit of doing that when downloading nodes or ways with a certain k/v combination in an area, but haven't yet done it with relations in that way. Regards, Maarten ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
Hi Maarten, > Nothing special really, I was just downloading the data from a lot of > relations sequentially in JOSM. I downloaded relation 1360154 with its > members which are bus master_relations that then also load the bus > relations but without their contents, and then I selected all the bus > relations and downloaded the members (in JOSM: select all relations and > do "download members"). By the way: Please consider to use "Download data > from Overpass" with the query ( rel(1360154); >>; ); out meta; The runtime is expected to be 3 to 10 seconds, data is usually less than two minutes off. As a side effect, this takes load off the Main API. Regards, Roland ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
On 2018-07-18 11:04 AM, Andrew Hain wrote: Will there be a local team or does whatever can’t be done remotely need a visit? For the install there will be two people traveling with the equipment, and I've reached out to some locals that were recommended. We should have enough people to get it done in the two days scheduled, but more hands are welcome. If you're a local with the technical background to help install servers or experience moving equipment and interested and available on Wed July 25th or Thurs July 26, please let me know off-list. The data centre is Equinix AM6 in De Omval, Oost, Amsterdam: https://www.openstreetmap.org/way/460515888 Long-term, the remote hands are capable of performing most tasks, including installing new servers. Local help would still be useful, so if you're interested, also let me know off-list. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
Will there be a local team or does whatever can’t be done remotely need a visit? -- Andrew From: Grant Slater Sent: 17 July 2018 13:55:05 To: Daniel Koć Cc: Talk Openstreetmap Subject: Re: [OSM-talk] API a lot slower? On 17 July 2018 at 13:42, Daniel Koć wrote: > W dniu 17.07.2018 o 12:22, Grant Slater pisze: >> Our primary hardware is moving to a new data centre next week, and >> will take some time to get up and running. > > What data center do you mean? Is it the one which OSMF was looking for > at the beginning of the year? > > https://blog.openstreetmap.org/2018/02/19/osmf-request-for-proposals-data-centre-2018/ > > Could you give some more details about it? > Yes. We are moving some core servers to an Equinix data centre in Amsterdam, NL. OpenStreetMap's Brexit ;-) Kind regards, Grant Part of the OSM Operations team. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
On 18/07/18 18:15, Maarten Deen wrote: Nothing special really, I was just downloading the data from a lot of relations sequentially in JOSM. I downloaded relation 1360154 with its members which are bus master_relations that then also load the bus relations but without their contents, and then I selected all the bus relations and downloaded the members (in JOSM: select all relations and do "download members"). From the Java console: 2018-07-18 19:13:55.553 INFO: GET https://api.openstreetmap.org/api/0.6/relation/7449166/full -> 200 and that for about 320 relations. Thanks. That's a call handled by cgimap so this is likely related to the fact that there is currently increased latency between the application server and the database for those calls. There is a pending pull request for cgimap that may help. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
On 2018-07-18 15:50, Tom Hughes wrote: On 17/07/18 10:42, Maarten Deen wrote: Is it just me or just today or is the API a lot slower after the move on sunday? I'm downloading a number of busrelations and it takes a lot longer than before the weekend. I know the cause of this has been explained already but I'd be interested to know exactly which API call you are referring to here as it may be that (long term at least) there is something here which should be fixed but it rather depends if it is a call being handled by rails or one being handled by cgimap. Nothing special really, I was just downloading the data from a lot of relations sequentially in JOSM. I downloaded relation 1360154 with its members which are bus master_relations that then also load the bus relations but without their contents, and then I selected all the bus relations and downloaded the members (in JOSM: select all relations and do "download members"). From the Java console: 2018-07-18 19:13:55.553 INFO: GET https://api.openstreetmap.org/api/0.6/relation/7449166/full -> 200 and that for about 320 relations. Regards, Maarten ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
W dniu 18.07.2018 o 14:40, Frederik Ramm pisze: > Yes, the situation would be more relaxed if work were distributed on > more shoulders but at this precise point in time I'd like our operations > group to focus on getting the move done with minimum fuss, and not on > recruiting new people! First of all - I appreciate the work behind the hardware and software support in OSMF. That is a huge bonus for me that in general I can focus on developing default style without special care for the infrastructure it will run on. Kudos for you all! I also absolutely don't mean this precise moment. Being system administrator myself, I can imagine how stressful and complicated that could be. I just thought that there could be some message after the solution has been selected and before the move has started. I try to warn my users beforehand when doing anything that could directly affect their work or if there's some risk involved. At least that's a Best Practice TM. :-) What I wanted to express was just a friendly moniker that OWG/EWG work is still not visible from my perspective (even if I'm an active member of OSM community) and that it could help to gain some people this way. Especially as this aim was explicitly stated by technical team (precisely I mean https://blog.gravitystorm.co.uk/2016/07/28/getting-involved-in-owg/ - BTW the same person that has successfully started and supported OSM Carto team building!). It's always hard to set the right priorities how much would one spend on communicating and finding new people versus doing actual work, but my experience is that it pays off to risk some evangelizing - now we have a team of active people in OSM Carto again, but it required putting some effort into making the style more available without clear outcome what exactly might work. And we still keep trying - see this fresh announcement: https://www.openstreetmap.org/user/Tomasz_W/diary/44420 -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
On 17/07/18 10:42, Maarten Deen wrote: Is it just me or just today or is the API a lot slower after the move on sunday? I'm downloading a number of busrelations and it takes a lot longer than before the weekend. I know the cause of this has been explained already but I'd be interested to know exactly which API call you are referring to here as it may be that (long term at least) there is something here which should be fixed but it rather depends if it is a call being handled by rails or one being handled by cgimap. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
Daniel, On 18.07.2018 14:18, Daniel Koć wrote: > Thanks for the message! However given how big this change is and that it > directly affects comfort of using OSM services, I would be glad to hear > much more details what are the plans and why there? The server move is currently eating up all the extra resources that OWG have (and more). We'll certainly be able to document it after the fact, but there is no free time available to provide any running commentary for the public while the move is being planned and executed. The decision for Amsterdam was made after carefully reviewing the bids/offers that were received in response to the request for proposals put out in February this year. The request also explained why the move was necessary. > As I understand, OSMF technical department (I mean Operations Working > Group and Engineering Working Group) wants to engage more people > (https://blog.openstreetmap.org/category/organisation/osmf-working-groups/) > and giving some general informations in public is a basic tool to > encourage them. Yes, the situation would be more relaxed if work were distributed on more shoulders but at this precise point in time I'd like our operations group to focus on getting the move done with minimum fuss, and not on recruiting new people! Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
W dniu 17.07.2018 o 14:55, Grant Slater pisze: > Yes. We are moving some core servers to an Equinix data centre in > Amsterdam, NL. Thanks for the message! However given how big this change is and that it directly affects comfort of using OSM services, I would be glad to hear much more details what are the plans and why there? As I understand, OSMF technical department (I mean Operations Working Group and Engineering Working Group) wants to engage more people (https://blog.openstreetmap.org/category/organisation/osmf-working-groups/) and giving some general informations in public is a basic tool to encourage them. -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
On 17 July 2018 at 13:42, Daniel Koć wrote: > W dniu 17.07.2018 o 12:22, Grant Slater pisze: >> Our primary hardware is moving to a new data centre next week, and >> will take some time to get up and running. > > What data center do you mean? Is it the one which OSMF was looking for > at the beginning of the year? > > https://blog.openstreetmap.org/2018/02/19/osmf-request-for-proposals-data-centre-2018/ > > Could you give some more details about it? > Yes. We are moving some core servers to an Equinix data centre in Amsterdam, NL. OpenStreetMap's Brexit ;-) Kind regards, Grant Part of the OSM Operations team. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
W dniu 17.07.2018 o 12:22, Grant Slater pisze: > Our primary hardware is moving to a new data centre next week, and > will take some time to get up and running. What data center do you mean? Is it the one which OSMF was looking for at the beginning of the year? https://blog.openstreetmap.org/2018/02/19/osmf-request-for-proposals-data-centre-2018/ Could you give some more details about it? -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
Hi All, Yes, the API will be slower. We are running on our backup hardware for the next few weeks. Our primary hardware is moving to a new data centre next week, and will take some time to get up and running. It is a major move for the server operations group. Will announce more details in the next few days. Kind regards, Grant On 17 July 2018 at 10:50, James wrote: > https://twitter.com/OSM_Tech/status/1018447091423744000?s=20 > > On Tue, Jul 17, 2018, 5:46 AM Maarten Deen, wrote: >> >> Is it just me or just today or is the API a lot slower after the move on >> sunday? I'm downloading a number of busrelations and it takes a lot >> longer than before the weekend. >> >> Regards, >> Maarten >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] API a lot slower?
https://twitter.com/OSM_Tech/status/1018447091423744000?s=20 On Tue, Jul 17, 2018, 5:46 AM Maarten Deen, wrote: > Is it just me or just today or is the API a lot slower after the move on > sunday? I'm downloading a number of busrelations and it takes a lot > longer than before the weekend. > > Regards, > Maarten > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] API a lot slower?
Is it just me or just today or is the API a lot slower after the move on sunday? I'm downloading a number of busrelations and it takes a lot longer than before the weekend. Regards, Maarten ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
