On Fri, Feb 6, 2009 at 7:32 PM, sly (sylvain letuffe) wrote:
> Hi there,
>
> ( that will probably bring some more fear to the different API/server
> admins ;-) )
>
> In france, next to the cadastre's agreement of using their data for OSM, we
> might soon be able to import something like 60% of Fra
On Fri, Feb 6, 2009 at 10:44 PM, Blars Blarson
wrote:
> Trapi takes about 16 gigabytes with a custom filesystem and the tag
> filtering on. I'm working on changes to make it smaller, that may
> save a few gigabytes.
>
> While it is currently optimized for ti...@home, other uses are
> envisioned.
On Fri, Feb 6, 2009 at 11:29 PM, Frederik Ramm wrote:
> Hi,
>
> sly (sylvain letuffe) wrote:
>> In france, next to the cadastre's agreement of using their data for OSM, we
>> might soon be able to import something like 60% of France's builings.
>
> Assuming that most houses will have 4 nodes, you'
My previous patch probably conflicts with
http://trac.openstreetmap.org/ticket/534 and
http://trac.openstreetmap.org/changeset/6562
which allows compressed (eg .gpx.gz) data usually served by
http://www.openstreetmap.org/trace/137465/data
to be served as uncompressed gpx xml if requested as
http:/
Frederik Ramm wrote:
> I would imagine some complaints from those users who are not interested
> in buildings, and for whom 90% of the data they download is useless
> after the import. You might have to provide filtered extracts for them.
>
>
I agree with Frederik, although I see a practical p
On Sat, Feb 7, 2009 at 4:56 AM, Stefan de Konink wrote:
> Matt Amos wrote:
>> indeed, with this method either the node and way IDs are cached (i.e:
>> stored in memory) or the query is repeated (wasting time).
>
> Some databases cache their results or even complete queries when noting is
> modifie
Matt Amos wrote:
> well, i find join-style syntax easier, but essentially yes.
Finally we found a mutual understanding ;) I am happy we didn't got as
low as mathematics ;)
>> But for performance reason Matthias would like to split this process in two
>> distinct parts, so reuse of the resultset
On Sat, Feb 7, 2009 at 2:51 AM, Stefan de Konink wrote:
> Ok do I understand you right in best readable case want to do the following?
>
> SELECT * FROM nodes WHERE BBOX(...) OR id IN (
> SELECT node FROM way_nds WHERE way IN (
> SELECT way FROM way_nds WHERE node IN (
> SELECT * FROM nodes WH
Matthias Julius wrote:
> Why would storing of the data be such a big overhead? Especially
> comparing to storing everything in (limited) memory? I think compared
> to all the other processing the API is doing writing the data to disk
> and then reading it again would not be so significant.
Sadly
Stefan de Konink writes:
> Matt Amos wrote:
>> On Fri, Feb 6, 2009 at 11:47 PM, Stefan de Konink wrote:
>>> Matt Amos wrote:
On Fri, Feb 6, 2009 at 8:32 PM, Stefan de Konink wrote:
> Even then; the ordering can be done in SQL.
not in the two-step scheme that Matthias was suggestin
Matt Amos wrote:
> On Fri, Feb 6, 2009 at 11:47 PM, Stefan de Konink wrote:
>> Matt Amos wrote:
>>> On Fri, Feb 6, 2009 at 8:32 PM, Stefan de Konink wrote:
Even then; the ordering can be done in SQL.
>>> not in the two-step scheme that Matthias was suggesting. it might be
>>> possible using
On Fri, Feb 6, 2009 at 11:47 PM, Stefan de Konink wrote:
> Matt Amos wrote:
>> On Fri, Feb 6, 2009 at 8:32 PM, Stefan de Konink wrote:
>>> Even then; the ordering can be done in SQL.
>>
>> not in the two-step scheme that Matthias was suggesting. it might be
>> possible using temporary tables, but
Matt Amos wrote:
> On Fri, Feb 6, 2009 at 8:32 PM, Stefan de Konink wrote:
>> Matthias Julius wrote:
>>> Is anyone relying on the ascending ID order?
>> Even then; the ordering can be done in SQL.
>
> not in the two-step scheme that Matthias was suggesting. it might be
> possible using temporary
On Fri, Feb 6, 2009 at 8:32 PM, Stefan de Konink wrote:
> Matthias Julius wrote:
>> Is anyone relying on the ascending ID order?
>
> Even then; the ordering can be done in SQL.
not in the two-step scheme that Matthias was suggesting. it might be
possible using temporary tables, but care would be
Hi,
sly (sylvain letuffe) wrote:
> In france, next to the cadastre's agreement of using their data for OSM, we
> might soon be able to import something like 60% of France's builings.
Assuming that most houses will have 4 nodes, you're talking about
importing roughly 10 million ways and 40 milli
Trapi can return node, way, relation, bbox, and tile. The ways are
always returned with nodes ("full") and relations members are not
included. Bbox and tile requests are refused when more than 1000
(configurable) stored tiles are included. (Stored tiles are z11-z14
depending on node density.) T
Hi!
It is a nice practice to indicate the file type in the url of the
download link to let users know what they are getting before actually
pushing it down their throat.
Filename on the trace detail page often says "somecity.gpx" (as at the
time of upload), download link is simply "data" and the
Matthias Julius wrote:
> Is anyone relying on the ascending ID order?
Even then; the ordering can be done in SQL.
> To be able to send proper error codes to the client when something
> goes wrong halfway down the path the data could be cached on disk and
> sent out once it is complete.
This is
Frederik Ramm writes:
> Hi,
>
> Marcus Wolschon wrote:
>> So...why is it that you hold the result-set of the nodes-query in memory
>> again?
>
> While not mandated by the XML structure, the XML document we return is
> usually sorted - nodes, ways, relations, each in ascending ID order.
> Becaus
Frederik Ramm wrote:
> Hi,
>
> Stefan de Konink wrote:
>> Is this also the case for non-full bbox requests?
>
> We do not make a distinction between full/non-full bbox requests, there
> is only one type and that gives you all nodes that belong to ways of
> which at least one node lies in the bb
Hi,
Stefan de Konink wrote:
> Is this also the case for non-full bbox requests?
We do not make a distinction between full/non-full bbox requests, there
is only one type and that gives you all nodes that belong to ways of
which at least one node lies in the bbox.
> Or is it actually calculated
Hi there,
( that will probably bring some more fear to the different API/server
admins ;-) )
In france, next to the cadastre's agreement of using their data for OSM, we
might soon be able to import something like 60% of France's builings.
The remaining question is : should we do it ?
- qualit
On Fri, Feb 6, 2009 at 5:20 PM, Marcus Wolschon wrote:
> On Fri, Feb 6, 2009 at 4:35 PM, Tom Hughes wrote:
>> Erik Johansson wrote:
>>
>>> How much of the DB load comes from the read only part of the API, and
>>> what if you remove the area limit on the map call?
>>
>> If I remove the area limit
Hi Frederik,
Frederik Ramm wrote:
> Marcus Wolschon wrote:
>> So...why is it that you hold the result-set of the nodes-query in memory
>> again?
>
> While not mandated by the XML structure, the XML document we return is
> usually sorted - nodes, ways, relations, each in ascending ID order.
> Be
Marcus Wolschon wrote:
> The most expensive call would probably be /map with a bounding-box.
> You query all the node, stream them out and keep the nodeIDs
> (but not the whole resultset with lat, lon, version, ...) in memory.
> Query the ways that use these nodes, keeping their IDs in memory.
> Th
Hi,
Marcus Wolschon wrote:
> So...why is it that you hold the result-set of the nodes-query in memory
> again?
While not mandated by the XML structure, the XML document we return is
usually sorted - nodes, ways, relations, each in ascending ID order.
Because a way or relation may require additi
On Fri, Feb 6, 2009 at 4:35 PM, Tom Hughes wrote:
> Erik Johansson wrote:
>
>> How much of the DB load comes from the read only part of the API, and
>> what if you remove the area limit on the map call?
>
> If I remove the area limit then somebody will do a massive query that
> will suck up all t
Tom Hughes wrote:
>>> The alternative is do much more complicated database queries which
>>> are likely to be slower (impossibly so on mysql, but possibly
>>> manageable with a different database).
>>
>> I still do not see how your database queries can be your main pain.
>> That is just a one ti
Stefan de Konink wrote:
> Tom Hughes wrote:
>> This is only partly about rails. Even if you take rails out of the
>> equation you still need to keep substantial amount of data in memory
>> in order to know which objects to fetch.
>
> The substantial amount of memory you are talking about is the
Tom Hughes wrote:
> This is only partly about rails. Even if you take rails out of the
> equation you still need to keep substantial amount of data in memory in
> order to know which objects to fetch.
The substantial amount of memory you are talking about is the resultset?
Or the partially gen
Stefan de Konink wrote:
> Tom Hughes wrote:
>> Yeah, it really is that simple, which is why us simple minded idiots
>> that run the server are still doing it all in memory.
>
> There are at least 3 known alternative implementations that run
> semi-realtime :) So yeah keeping it in rails while th
Tom Hughes wrote:
> Yeah, it really is that simple, which is why us simple minded idiots
> that run the server are still doing it all in memory.
There are at least 3 known alternative implementations that run
semi-realtime :) So yeah keeping it in rails while there are
alternatives make you stu
Stefan de Konink wrote:
> Tom Hughes wrote:
>
>> That limit is not (primarily) about the cost of gathering the required
>> data, though obviously that might become an issue as well, it's about
>> the fact that we are holding the whole result set in memory (several
>> times over in fact) on the r
Tom Hughes wrote:
> Erik Johansson wrote:
>
>> How much of the DB load comes from the read only part of the API, and
>> what if you remove the area limit on the map call?
>
> If I remove the area limit then somebody will do a massive query that
> will suck up all the memory on the machine and e
Erik Johansson wrote:
> How much of the DB load comes from the read only part of the API, and
> what if you remove the area limit on the map call?
If I remove the area limit then somebody will do a massive query that
will suck up all the memory on the machine and everything will die.
That limi
On Fri, Feb 6, 2009 at 10:20 AM, Tom Hughes wrote:
> Russ Nelson wrote:
>> It seems to work for them. I think it's a good idea for us.
> while Marcus was suggesting clustering third party servers which is not
> something we will be doing as far as I'm concerned.
How much of the DB load comes fr
On Fri, 06 Feb 2009 15:31:10 +0100, Stefan de Konink
wrote:
> marcus.wolsc...@googlemail.com wrote:
>> On Fri, 06 Feb 2009 15:16:28 +0100, Stefan de Konink
>> wrote:
>>> Florian Lohoff wrote:
API 0.7 should contain a "referral" as LDAP does - So a client could
connect to a cluster of re
marcus.wolsc...@googlemail.com wrote:
> On Fri, 06 Feb 2009 15:16:28 +0100, Stefan de Konink
> wrote:
>> Florian Lohoff wrote:
>>> API 0.7 should contain a "referral" as LDAP does - So a client could
>>> connect to a cluster of read-only copies and once you write to it you
>>> get a referral to th
On Fri, 06 Feb 2009 15:16:28 +0100, Stefan de Konink
wrote:
> Florian Lohoff wrote:
>> API 0.7 should contain a "referral" as LDAP does - So a client could
>> connect to a cluster of read-only copies and once you write to it you
>> get a referral to the master database. Synchronization is a big is
Florian Lohoff wrote:
> API 0.7 should contain a "referral" as LDAP does - So a client could
> connect to a cluster of read-only copies and once you write to it you
> get a referral to the master database. Synchronization is a big issue
> here but it should be a good point for scalability ...
I wo
+-le 06.02.2009 08:38:24 -0500, Milenko a dit :
| That link actually includes the TRAPI servers, which do not return all
| data. Anything not used by t...@h is stripped out of the TRAPI databases.
| Since my ROMA server is down while it rebuilds the database on a new disc
| array (SSDs!!), pretty
On Fri, Feb 06, 2009 at 12:29:44PM +0100, marcus.wolsc...@googlemail.com wrote:
> It does not?
> http://wiki.openstreetmap.org/wiki/ROMA
> does not mention that it does not implement the full API.
> btw, is it 0.6 -capable already?
It does not - ROMA does only support the "map" call as thats the o
> -Original Message-
> From: dev-boun...@openstreetmap.org [mailto:dev-
> boun...@openstreetmap.org] On Behalf Of Mathieu Arnold
> Sent: Friday, February 06, 2009 6:41 AM
> To: marcus.wolsc...@googlemail.com
> Cc: Dev; t...@openstreetmap.org
> Subject: Re: [OSM-dev] [OSM-talk] donating read
Hello everyone,
Slightly off topic but it relates to an idea I have for a 3D navigation
tool for walkers using OSM and SRTM data, for both desktop and mobile
devices. I like the sound of Qt for series 60 phones (e.g. the N95) but
unfortunately the preview release is for Windows only, which I ha
+--On 6 février 2009 12:29:44 +0100 marcus.wolsc...@googlemail.com wrote:
| On Fri, 06 Feb 2009 11:25:45 +0100, Mathieu Arnold wrote:
|> I'm not certain we're talking of the same thing, I see no relation
| between
|> the minute diff and the exactness of the API, for now, ROMA only serves
|> /api/0
On Fri, 06 Feb 2009 11:25:45 +0100, Mathieu Arnold wrote:
> +--On 6 février 2009 11:12:29 +0100 Stefan de Konink
> wrote:
> | Mathieu Arnold wrote:
> |> +--On 6 février 2009 10:23:22 +0100 "Jonas Krückel (John07)"
> |> wrote:
> |> | I think you know about ROMA and TRAPI. But i think you are spea
On Fri, Feb 6, 2009 at 11:18 AM, wrote:
> On Fri, 6 Feb 2009 10:23:22 +0100, "Jonas Krückel (John07)"
> wrote:
> > Am 06.02.2009 um 08:28 schrieb :
> >
> >>
> >> Hello,
> >>
> >> what about packaging everything one needs to set up a
> >> read-only api-server that applies the minutely diffs
> >>
On Fri, 6 Feb 2009 10:23:22 +0100, "Jonas Krückel (John07)"
wrote:
> Am 06.02.2009 um 08:28 schrieb :
>
>>
>> Hello,
>>
>> what about packaging everything one needs to set up a
>> read-only api-server that applies the minutely diffs
>> and re-importes the planet lets say once a month?
>>
>> Many
Mathieu Arnold wrote:
> +--On 6 février 2009 11:33:25 +0100 Stefan de Konink
> wrote:
> | Mathieu Arnold wrote:
> |> I'm not certain we're talking of the same thing, I see no relation
> |> between the minute diff and the exactness of the API, for now, ROMA only
> |> serves /api/0.5/map?bbox=foo,ba
+--On 6 février 2009 11:33:25 +0100 Stefan de Konink
wrote:
| Mathieu Arnold wrote:
|> I'm not certain we're talking of the same thing, I see no relation
|> between the minute diff and the exactness of the API, for now, ROMA only
|> serves /api/0.5/map?bbox=foo,bar if it is to replicate the API, i
Mathieu Arnold wrote:
> I'm not certain we're talking of the same thing, I see no relation between
> the minute diff and the exactness of the API, for now, ROMA only serves
> /api/0.5/map?bbox=foo,bar if it is to replicate the API, it needs to serve
> all the /api/0.5/node way relation...
Aah!
+--On 6 février 2009 11:12:29 +0100 Stefan de Konink
wrote:
| Mathieu Arnold wrote:
|> +--On 6 février 2009 10:23:22 +0100 "Jonas Krückel (John07)"
|> wrote:
|> | I think you know about ROMA and TRAPI. But i think you are speaking
|> | about a exact readonly-copy of the api.
|>
|> Well, ROMA
Mathieu Arnold wrote:
> +--On 6 février 2009 10:23:22 +0100 "Jonas Krückel (John07)"
> wrote:
> | I think you know about ROMA and TRAPI. But i think you are speaking
> | about a exact readonly-copy of the api.
>
> Well, ROMA could be extended to an exact read only copy, once all t...@h load
> i
+--On 6 février 2009 10:23:22 +0100 "Jonas Krückel (John07)"
wrote:
| I think you know about ROMA and TRAPI. But i think you are speaking
| about a exact readonly-copy of the api.
Well, ROMA could be extended to an exact read only copy, once all t...@h load
is all sent to the TRAPI cluster (whi
Am 06.02.2009 um 08:28 schrieb :
>
> Hello,
>
> what about packaging everything one needs to set up a
> read-only api-server that applies the minutely diffs
> and re-importes the planet lets say once a month?
>
> Many do not need data that is accurate up to an hour
> but as there are no other se
Russ Nelson wrote:
> It seems to work for them. I think it's a good idea for us.
I'm sure at some point we'll need to do it, but we haven't really
reached that point yet and we certainly don't have the resources to do
it yet.
There are also a number of significant but important differences be
On Thu, 5 Feb 2009, Raphael Mack wrote:
> That said, a few words about mine. I know a bit of Java and Software
> Engineering and even have (may be had) an svn account for the josm repo.
> The feature I ever missed in JOSM was to be able to select gpx points
> and delete some of them from a track,
Soyour point being?
Are you giving Pro, Contra, an offer to help
or a helpful suggestion?
Marcus
On 6 Feb 2009 09:35:21 +0100, "Russ Nelson" wrote:
>
> This is the Wikipedia model. If you're not logged-in and you're not
> editing, you NEVER touch the main server. You're always accessing
58 matches
Mail list logo
