On Fri, Oct 9, 2009 at 4:06 AM, <[email protected]> wrote:
> On Thu, 8 Oct 2009 10:56:27 -0700, Sam Vekemans > <[email protected]> wrote: > > Hi, > > in the next week, i'll update the wiki 'importing government (bulk > > provider) data' and list the 3-step approach. > > I'm looking forward to it. > But please don't limit your self too much to shape-files. > > The concept is the same. 1. convert 2. share and 3. import manual (josm) / automatic (bulk_upload) / semi-automatic (road_matcher/automatch) > > If i get through on the conference call today, i'll try to explain it, > > and maybe someone else who is a better wikipedian can fix it. :-) > > Sorry. I'll not be on any of the conference-calls at all. > ( > http://wiki.openstreetmap.org/wiki/Foundation/Import_Support_Working_Group > ) > > > Step 1: Data conversion > > -using eithor 'shp-to-osm.jar' or 'shp2osm.py' or 'mp2osm' or 'other' > > we make a conversion script > > -1 person is in charge of editing this script (as users will submit > > errors) (just as for other programs) > > Yes, my parser for the common interchange-format of LCLs for TMC-services > exists for a while now. No known errors, well structured and commented > (code quality is very important to me). > > > > Step 2: data sharing > > -the person that is running the conversion script would make these > > .osm files available to the community so people can see it. > > Well. Guess what I did quite a few days ago. ;) > > > > > 3 -data import > > eithor a.1 use josm and copy the features over > > That's what we do for everything that cannot be automatically imported. > I had to add even more warnings as despite quite a lot of fat warnings > people DID upload some of these reference .osm -files directly. > > the nature of the beast :). ... which is why i want to be making the rivers (geobaseNHN data) available at the same time, and only for those areas that have roads already imported. (so to save people work) > > 3.2 use bulk_import.pl for direct api load (for large areas, > > if given permission to access the API at non-peak hours, and on que) > > "que"? > On the status chart i note the progress, and list what is next to load. With bulk import, you can set it so that it uploaded to the API only at non-peak hours, the sys admins, can set it so that changesets would get held back, to make room for the regular OSM traffic flow. > > > 3.3 use a postGIS database and use OpenJUMP Automatch to > > compare datasets. > > Never heard of that but the LCL-map if extremely simplified. I don't think > the complex matching needed can be done by a generic tool. > > It just helps by finding exact matches, and highlights where it couldn't match right based on a set of rules. .... thats why i prefer having the .osm files available, and letting local people copy what they like. (avoiding openJUMP all together) > > The facts that remain are; > > 1 -the available data is constantly changing > > Not much changes to LCLs expected here. But many other countries with > lots of TMC-providers may follow. That's why I took so much care my tools > are well written and easy to understand for a developer. > > In Canada it's speratic, and somethings would be 6 months for a partial update and 1 year for full update. and sometimes the source features tagging gets changed too. Because it's a federal holding house, they provide data for other people too, not just OSM :) so they want to keep their dataset uptodate also. > > utility tool, and having the 'tag matching list' be maintained by 1 > > person, will make it easier for everyone to work with the data. > > What do you mean by 'tag matching list'? > > with shp-to-osm the rules.txt lists the tag matches (source=osm) key/value comparisions. from the google docs chart. (or wiki chart if it's not that big) > > By making it available, we can let the users decide what they want to > > add, or omit. and put the emphasis on simply making all available data > > available to use when and if we want. > > Sounds less like hints for people doing an import then general advice > for people offering datasets to import. > > lost me there, it's learning for everyone IMO :) > Marcus > re: next message > >> The "Merge layers" option in JOSM is also not really usable because it >> can take really long (several days) for two layers with 100k nodes. >Well I don't think he was talking about single .osm -files with that many elements. . The .osm -files I generate (@topic)never have THAT kind of node-count. Ya actually it was refering to how lakes can be a higher node count, and the shp-to-osm script has a hard time with that. So the solution is to manually split up the big lake to different polygons. >Your script is VERY "without fancy logic". I wouldn't use that at all if there is any already existing data in the area you are doing an import in. Besides there can be elements with the same id in your case. You dont need to use it :) Its designed to only handle this data. (and im not an expert programmer eithor) ... i just ask that they make the .osm files available rather than direct upload so local area mappers can copy in the data. .. so where elements are existing, its up to the local area mapper to decide on what to add, or not to add, and how or if they want to merge it. That decision shouldn't be my own to make remotly. (not having being physically been in the area). And a last point, it's fine if the converted .osm files remain on a server for a year before getting added, as long as people know it's there. I don't see a point in rushing, 'just to get it copied in' . Cheers, Sam
_______________________________________________ Imports mailing list [email protected] http://lists.openstreetmap.org/listinfo/imports
