RE: [OSGeo-Discuss] Anyone interested in geocoding and routing?
Hi Everyone, The mailing list is up (and also posted on the wiki pages): http://lists.osgeo.org/mailman/listinfo/routergeocoder We decided to go with a single list given the synergies between the two systems and overlap in those that are interested. We can always split it later. Thanks again, Andrew http://ingres.com http://osbootcamp.org _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Ross Sent: November 11, 2008 1:41 PM To: 'OSGeo Discussions' Subject: RE: [OSGeo-Discuss] Anyone interested in geocoding and routing? Hi Everyone, Thank you for the great posts and discussion! I'm thinking given the interest, it may be time to create a separate mailing list for each of the OpenGeocoder and OpenRouter initiatives. If no one disagrees, we'll do this and I'll share the lists here so people can opt-in to the conversation. Andrew _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Ross Sent: November 10, 2008 2:17 PM To: 'OSGeo Discussions' Subject: RE: [OSGeo-Discuss] Anyone interested in geocoding and routing? Thank you everyone for the great response to this post. I'd like to poll everyone's thoughts regarding the appropriate next steps. It seems to me we have good support for this initiative. It isn't as clear to me yet what the best way forward is. My instincts are leaning towards spending time to capture requirements, identify key use cases, and crucial standards. The thinking is that with this information captured, we're in a better position to review the implementations available to identify which candidates are best positioned to satisfy those needs. I would like to echo Andrew Turner's comments that we're focusing on API's that can be used offline rather than a web service. (though it's good to see great web service projects out there!) Input, thoughts, feedback are greatly appreciated. Thank you in advance, Andrew _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Ross Sent: November 5, 2008 7:59 PM To: OSGEO Discussion list Subject: [OSGeo-Discuss] Anyone interested in geocoding and routing? Hi Everyone, A few of us have been talking and thought the timing might be right to try and start projects to work on Geocoding and Routing. We're still gathering information and checking to learn who's interested. If you are interested, please check out the following wiki's and add your name in the section you are interested in. http://wiki.osgeo.org/wiki/OpenGeocoder http://wiki.osgeo.org/wiki/OpenRouter There are experts out there that are far more knowledgeable about Geocoding and Routing than I am. To these good people: please feel empowered to make modifications, provide feedback in whatever way you feel most comfortable. Thanks, Andrew ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Language Specific Lists Summary
On Wed, Nov 12, 2008 at 10:51:44PM +0100, Markus Neteler wrote: > On Tue, Nov 11, 2008 at 8:07 PM, Christopher Schmidt > <[EMAIL PROTECTED]> wrote: > ... > > * Germany > > ... is here: > https://lists.fossgis.de/mailman/listinfo/GAV-talk > > Italy: > http://www.faunalia.com/cgi-bin/mailman/listinfo/gfoss That's the same as the one in my original list, right? Regards, -- Christopher Schmidt Web Developer ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Language Specific Lists Summary
On Tue, Nov 11, 2008 at 8:07 PM, Christopher Schmidt <[EMAIL PROTECTED]> wrote: ... > * Germany ... is here: https://lists.fossgis.de/mailman/listinfo/GAV-talk Italy: http://www.faunalia.com/cgi-bin/mailman/listinfo/gfoss Markus ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Discussion on Routing
I agree with the needed improvements described by Stephen. I think the memory issue is big issue too and not well addresses right now. On the other hand, I don't think that a routing solution should really strongly depend on a Spatial databases. Actually, the optimization is more a network optimization type of problem. The geographic data is in fact important only for the presentation of the path in one specific case of routing. So I'm really looking to have some datasource independence. There is also more and more needs for multi-modal routing, not only by cars, but point to point that can combine walk, bicycle, train, metro, car, ... I think that Open Source routing could play a role on that field. For a project sponsored by the french ministery in collaboration with a partner, we did some investigations combining algorithms for multimodal routing optimization. I think it is important to include these needs when we will be thinking on the global architecture, even if the those complex algorithms won't be the first one to be integrated. We are ready to put some effort on this. Claude > I have started playing with pgRouting and find it to be an impressive > start. It was easy to rewrite some of the plpgsql wrappers to better > suit my needs. I was able to write a driving directions module the > explicates the direction as text and can be configured for multiple > languages. > > There are a few limitation that I see: > > 1) poor support for turn restrictions. It supports turn restrictions but > if you want to enter multiple restrictions at a given intersection it is > difficult to do and not intuitive. I have updated a couple of the wiki > pages to add more information. This is really needed to be able to > support commercial datasets like Navteq. > > 2) There is no way to optimize fetching of data that is needed for a > potential solution other than via the postGIS spatial index which is > based on bounding boxes. This is very bad for route that follows > basically a 45 degree diagonal path. Another option would be to organize > data into squares of a fixed spatial size, these could be loaded and > cached as needed. > > 3) The data for the US and Canada or all of Europe has in the ballpark > about 27-32 million segments. I would like to see optimization like the > above, and/or support for multilevel routing as an option. > > 3a) There is also some very new research that has been done gives 2+ > orders of magnitude faster routing by doing some preprocessing of the > network. It might be nice to see something like this integrated. > http://algo2.iti.uni-karlsruhe.de/english/routeplanning.php > > 4) I would like to see a decoupling of the routing engine from the > backing stores to support it. PostGIS is nice and I'm happy to work with > it, but I can see a lot of situations where not using postgresql could > be desirable. It would be nice to have an API that would allow plugable > or code-able back end data stores. > > 5) It would be nice to have the ability to set some standard costs like > it cost more to make a turn across traffic than with traffic. It costs > more to make a turn than to stay on a street, so the cost of the turn > must add value to the overall route. This helps to avoid the stair-step > routes in a gridded city region. The USPS has save millions of dollars > in fuel costs by avoiding left hand turns across traffic unless it is > absolutely required. begin:vcard fn:Claude Philipona n:Philipona;Claude org:Camptocamp SA adr:Parc scientifique EPFL;;PSE-A;Lausanne;;CH-1015;Switzerland email;internet:[EMAIL PROTECTED] tel;work:+41 21 619 10 11 tel;fax:+41 21 619 10 00 tel;cell:+41 78 648 32 84 x-mozilla-html:FALSE version:2.1 end:vcard ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Anyone interested in geocoding and routing?
I think too that it is possible to break down the problem as described by Frank to get a widely usable geocoder. But I do think to that is not so trivial as the way to described and use addresses varies considerably even in Europe. There are quite a few possibilities of strategies, from very rigid scheme search to quasi full text. We worked on a geocoder for a client a couple of years ago, implementing some of these ideas trying to use a more or less universal way of geocoding, and a way of describing how the data and addresses are organized. I will try to take look back to it to see if there is ideas or code that could be interesting. Claude Frank Warmerdam wrote: > (Orkney)Toru Mori wrote: >> Apart from technical design, geocoder is "useless without data" :) >> >> Address systems are varied and messy, at least here in Japan and in other >> Asia region. For example, Japan has more than 3 systems. Additionally it >> is very tough to get good "enough" data. There is no separation in address >> text. >> >> We developed geocoder.ja already for our region specifically, but >> unfortunately it won't work in even other countries in Asia. >> http://www.postlbs.org/ja/geocoder >> >> >> A sigle universal, global geocoder may sound perfect. However, there is >> very limited space in terms of standardization as follows. >> >> - >> API (can be standardized) >> - >> thin parser (might be standardized) >> - >> geocoding logic (cannot be standardized) >> - >> local dataset (varied and messy) >> - >> >> So what OSGeo should lead would be just APIs. If OSGeo wants to >> standardize lower levels, then the project won't finish probably. > > Toru Mori, > > I do think any final solution needs to support plugging in > distinct address parsers and geocoder matching logic for > differ locales and underlying datasets. My understanding is > that some of the commercial data providers have fairly > standard schemes for how to break down address data into a > standard tabular layout - at least for quite a bit of the world. > We might want to learn something from the breakdown approach > they used. > > Generally I agree that we won't be able to write one > universal geocoder but it seems to me a good architecture > with the ability for people to contribute local parsers > and geocoding matchers might be an ideal sort of open source > project. > > Of course, I'm speaking hypothetically (without experience) > and you are speaking from experience! > > Best regards, -- Claude Philipona Camptocamp SA PSE A CH-1015 Lausanne Switzerland +41 21 619 10 11 (direct) +41 21 619 10 10 (centrale) +41 21 619 10 00 (fax) +41 78 648 32 84 (mobile) http://www.camptocamp.com http://www.cartoweb.org begin:vcard fn:Claude Philipona n:Philipona;Claude org:Camptocamp SA adr:Parc scientifique EPFL;;PSE-A;Lausanne;;CH-1015;Switzerland email;internet:[EMAIL PROTECTED] tel;work:+41 21 619 10 11 tel;fax:+41 21 619 10 00 tel;cell:+41 78 648 32 84 x-mozilla-html:FALSE version:2.1 end:vcard ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Anyone interested in geocoding and routing?
Toru Mori, Your experience with geocoders seems to be very broad and your input on constructing a new tool will be very valuable. I have never and probably will never work with Asian address systems because I can not read the languages. I have written and maintained 3-4 other geocoders mostly oriented toward US, Canada, Europe. I also have a lot of experience working with the Oracle Spatial geocoder which handled all of US, Canada, and England and most of Western Europe very well using a single engine. By the way, the geocoders do not all have to fit into the same data structures or tables (assuming SQL). I would also not assume that addresses have to be in street ranges, in fact some places have all houses identified by a point. TeleAtlas data has a combination of address ranges, and exceptions to the ranges. The framework should provide structure for the various efforts, and maybe provide standard services that many of the plugins can use but it should not get in the way. For example if you do not want your addresses standardized then you would install the do_not_mess_with_my_data standardizer. Someone mentioned in another post (that I can't find at the moment), that a lot of the work will be pre-processing the raw data to load it, this is true. The pre-processing is to standardize the data before it is loaded, so when you standardize an input request, you have a better chance of matching it. But you obviously should use the same standardizer in both cases, so data loading design can not be done separately from the overall design of an individual engine and the country standardization tools. Once an engine has been designed, it is possible to write data loaders for various different datasets that can be serviced by a given engine. I look forward to your input, experience and lessons learned. -Steve (Orkney)Toru Mori wrote: > Steve, > > I totally agree to your opinion. We can have a standard framework and we > may call it "OSgeo international geocoding framework", not "geocoder". > > Everybody would love concept of GDAL/OGR, however, geocoder will have > deferent story. > > In my experience of working for a U.S. based GIS company which has global > operations, an international geocoder was always requested by customers. > The company developed geocoders of U.S., Canada, England, Western part of > Europe and Australia. However, they are not really unified. Even they have > similar address systems, the U.S. engine couldn't handle other regions > instantly. > > Regarding Japan, Korea and China, they have unique address systems each, > so even normalization of inbound address data process did not work with > one algorithm. Geocoder always works with data. Address data and system > are made based on each local culture and history. > > I won't discourage that we try to have an open geocoder at all. > Just want to avoid possible misunderstanding on this list that "i18n" > geocoder can be a snap. > > Mori > > Stephen Woodbridge <[EMAIL PROTECTED]> wrote： >> Yes, you are correct, but this is not a reason for not trying. For >> example, US, Canada, and most (all?) of Europe could be represented by a >> single engine. It is likely, that large chunks of the rest of the world >> can be represented by a similar engine, and then there will be a bunch >> of exceptions, like those you reference in Asia. >> >> It seems to me that if you scan the in coming request you should be able >> to guess the country or identify it as part of the request and load the >> engine/data to support it, if it is supported, and go from there. >> >> Data is always a problem. That is one of the reasons that I think a SQL >> backing store is required. Each data set would need to be mapped and >> standardized to a standard schema that is supported by one of the >> geocoder engines. Data would need to be converted from its raw format >> and standardized into the standard schema. >> >> So beyond the API, we would have some representative geocoder engines >> and specific data loaders targeted at specific data sets, and data >> standardizers targeted at specific countries. >> >> If we take a modular approach of using plugins, then we can design an >> API, design the plugin API, design some of the data standards, and make >> reference implementations. This will allow the project to work with many >> people that have specific data/country needs that can build into the >> framework. >> >> I do not expect to have a one size fits all solution, I just think we >> can have a standard framework that any solution can fit into and from >> that we can have lots of collaborators working on it. I think of this >> like OpenStreetMap when it first started. I thought to myself that this >> is crazy it will never work, there are too many roads, etc. Today they >> have an impressive collection of spatial data. This is a huge task, but >> it can be tackled one step at a time. >> >> -Steve >> > ___ > D
Re: [OSGeo-Discuss] Discussion on Routing
Steve, We, pgRouting developers are willing to contribute our code and know how to "database independent" routing engine development. Current pgRouting connects with PostgreSQL very much. But this does not mean we have to write codes from scratch again. Additionally, we have lessons to learn from pgRouting developers' and users' communities such as Anton pointed out. I believe they should be useful in designing new routing engine. Mori Stephen Woodbridge <[EMAIL PROTECTED]> wrote： > 4) I would like to see a decoupling of the routing engine from the > backing stores to support it. PostGIS is nice and I'm happy to work with > it, but I can see a lot of situations where not using postgresql could > be desirable. It would be nice to have an API that would allow plugable > or code-able back end data stores. ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
Re: [OSGeo-Discuss] Anyone interested in geocoding and routing?
Steve, I totally agree to your opinion. We can have a standard framework and we may call it "OSgeo international geocoding framework", not "geocoder". Everybody would love concept of GDAL/OGR, however, geocoder will have deferent story. In my experience of working for a U.S. based GIS company which has global operations, an international geocoder was always requested by customers. The company developed geocoders of U.S., Canada, England, Western part of Europe and Australia. However, they are not really unified. Even they have similar address systems, the U.S. engine couldn't handle other regions instantly. Regarding Japan, Korea and China, they have unique address systems each, so even normalization of inbound address data process did not work with one algorithm. Geocoder always works with data. Address data and system are made based on each local culture and history. I won't discourage that we try to have an open geocoder at all. Just want to avoid possible misunderstanding on this list that "i18n" geocoder can be a snap. Mori Stephen Woodbridge <[EMAIL PROTECTED]> wrote： > Yes, you are correct, but this is not a reason for not trying. For > example, US, Canada, and most (all?) of Europe could be represented by a > single engine. It is likely, that large chunks of the rest of the world > can be represented by a similar engine, and then there will be a bunch > of exceptions, like those you reference in Asia. > > It seems to me that if you scan the in coming request you should be able > to guess the country or identify it as part of the request and load the > engine/data to support it, if it is supported, and go from there. > > Data is always a problem. That is one of the reasons that I think a SQL > backing store is required. Each data set would need to be mapped and > standardized to a standard schema that is supported by one of the > geocoder engines. Data would need to be converted from its raw format > and standardized into the standard schema. > > So beyond the API, we would have some representative geocoder engines > and specific data loaders targeted at specific data sets, and data > standardizers targeted at specific countries. > > If we take a modular approach of using plugins, then we can design an > API, design the plugin API, design some of the data standards, and make > reference implementations. This will allow the project to work with many > people that have specific data/country needs that can build into the > framework. > > I do not expect to have a one size fits all solution, I just think we > can have a standard framework that any solution can fit into and from > that we can have lots of collaborators working on it. I think of this > like OpenStreetMap when it first started. I thought to myself that this > is crazy it will never work, there are too many roads, etc. Today they > have an impressive collection of spatial data. This is a huge task, but > it can be tackled one step at a time. > > -Steve > ___ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
