On Fri, Mar 19, 2010 at 2:46 AM, Adam Dunn <[email protected]> wrote:
> The reason why it is functionally useless at this point is because the > id/et_id is not being transferred through the process correctly. Perhaps an > explanation of the process is in order: > The SQL stage is basically for reprojection and selecting a bounding box. > If you already had both the government and the OSM roads converted into a > Shapefile format, and they both had the same projection, and you were > willing to work on the entire country at the same time, then you wouldn't > even need SQL. Here are here: http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp Here are the osm data (from cloudmade) http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp You generally don't want to do the whole country at a time (trust me, it's > not something you do in OpenJUMP), and you generally need to do some > reprojection/format conversion, so we need SQL. What gets spit out of the > SQL step is two files, OSM.shp and GOV.shp, and they each have geometry > (where the roads are) and id numbers (internal numbers used by each). > These two files get loaded into OpenJUMP Roadmatcher, where you find the > matches, find the standalones, then output the result. This result file has > a list of the id numbers for the roads that are standalone in the government > database. > Then (for Canada) geobase2osm.py will use the Roadmatcher result file as a > mask to see which roads to copy from the government gml file over to an osm > file. It does this by looking at the standalone id numbers, and then copying > only the roads in the gml that have those ids. > I kind of think of it like a transistor. A transistor takes some voltage > input and connects it to the output, depending on whether the gate voltage > is on or not. The input/output voltages can be large or small, but the gate > voltage only needs to be tiny. The gate voltage itself doesn't get passed > through to the output. Something like this: > gate > | > input -- transistor -- output > > Equivalently, geobase2osm.py connects the Government.gml geometry to the > Standalone.osm geometry, but only if the Standalone Result ID is on. The > Result ID itself doesn't get passed through, it's only used as a gate mask. > So we get something like: > Result ID# > | > GovGeometry.gml -- geobase2osm.py -- StandaloneGeometry.osm > > Since I haven't yet gotten the ID numbers to pass through, we can't use > them as a mask for geobase2osm.py, rendering it useless. We can't generate > osm files that have no duplicates with what's already on OSM. > I see, so the ids are the problem. I can assign them new ids. We can rename the column? > > Let's say you want to do things more manually. You could run the match > process in Roadmatcher, then visually look and see what roads need to be > copied over, then go over to JOSM and manually pick out those roads. This is > silly, as you could do this faster by skipping Roadmatcher, and just > matching by eye within JOSM. > I see, well that is what i hoped roadmatcher would help with. > > Even if I do get ID numbers to transfer through, I don't know how portable > geobase2osm.py will be to the Albanian data. What tools have other imports > used? Maybe there's something else that uses a slightly different method? > > OK, well we need to look into this in detail. I was working on buffering the roads to remove duplicate. http://osmopenlayers.blogspot.com/2010/03/comparison-of-maps-using-buffering.html http://osmopenlayers.blogspot.com/2010/03/maps-of-albania-comparison-of-roads.html Thanks mike > Adam > > > On Thu, Mar 18, 2010 at 1:38 PM, [email protected] < > [email protected]> wrote: > >> >> The roads shp files are all the same,, that is true. >> >> >> On Thu, Mar 18, 2010 at 9:19 PM, Adam Dunn <[email protected]> wrote: >> >>> So far, I've got the LL_Roads2 files imported in PostgreSQL, and I've >>> written some functions to export them (modified from the NRN process on >>> wiki), but it's not working properly. Using pgAdmin on Linux, I can see what >>> data values there are, and their types: >>> gid serial NOT NULL, >>> id numeric, >>> "type" character varying(50), >>> category smallint, >>> cat2 smallint, >>> shape_leng numeric, >>> "name" character varying(30), >>> shape_le_1 numeric, >>> et_id integer, >>> the_geom geometry, >>> >>> When I export this to shp and open in OpenJUMP, I can see that some of >>> the values aren't getting exported properly. gid doesn't seem to be exported >>> at all. id, type, shape_leng, shape_le_1, and the_geom all seem to be fine. >>> Unfortunately, category, cat2, and et_id are all coming up with the value 0, >>> even though they had other values in the original dataset. Name I can't >>> test, since it seems to be null in the original data. >>> Here's the function I'm using: >>> >>> >>> DROP TYPE albania_gov_data; >>> CREATE TYPE albania_gov_data AS ( >>> --Included all Albanian keys, since I don't know what they mean/which >>> are important >>> gid integer, >>> the_geom geometry, >>> id numeric, >>> way_type text, >>> category smallint, >>> cat2 smallint, >>> shape_leng numeric, >>> way_name text, >>> shape_le_1 numeric, >>> et_id integer >>> >>> ); >>> CREATE OR REPLACE FUNCTION select_albania_gov_roadtile(coordinates text) >>> RETURNS SETOF albania_gov_data >>> AS $$ >>> SELECT gid,ST_Transform(the_geom,4326),id,"type", --might need 4191 >>> rather than 4326 >>> category,cat2,shape_leng,"name",shape_le_1,et_id FROM >>> albania_roadseg >>> WHERE >>> ST_Intersects(ST_Transform(the_geom,4326), >>> ST_GeomFromEWKT($1)) >>> >>> >>> $$ >>> LANGUAGE SQL; >>> >>> CREATE TYPE albania_osm_data AS ( >>> way geometry, >>> osm_id integer, >>> name text >>> ); >>> >>> CREATE OR REPLACE FUNCTION select_albania_osm_roadtile(coordinates text) >>> RETURNS SETOF albania_osm_data >>> AS $$ >>> SELECT ST_Transform(way,4326),osm_id,substr(name,0,80) --might need >>> 4191 rather than 4326 >>> FROM planet_osm_line WHERE >>> ST_Intersects(way,ST_GeomFromEWKT($1)) >>> AND ((highway is not null) >>> OR (railway is not null)) --Albania appears to have >>> railways in gov_data >>> $$ >>> LANGUAGE SQL; >>> >>> >>> Anybody got any ideas? >>> >> But the shapes are working? >> >> >>> >>> On the data itself: >>> What is the difference between all the files on the ftp? There is >>> LL_Roads, LL_Roads2, LL_Roads_OSM, but a quick check in OpenJUMP, and they >>> all look the same. >>> >> >> no difference >> >> >>> For my test, I exported an area around Gjirokastra [ >>> http://www.openstreetmap.org/index.html?minlat=40&minlon=20&maxlat=40.2&maxlon=20.2&box=yes&layers=B000FTF]. >>> In this area, the id tag was only partially useful: In downtown Gjirokastra >>> (the densely roaded area that is not currently visible on OSM, but is approx >>> bounded by [http://www.openstreetmap.org/browse/way/30455602] and [ >>> http://www.openstreetmap.org/browse/way/36777258]) the id tag seemed to >>> be missing for roads. The id tag was only available in the outer (rural?) >>> areas. Shape_leng was mostly 0.0, except for about 20 roads at the south end >>> of Gjirokastra (out of 2479 roads), in which case it matched up with >>> shape_le_1. This data also includes Footpath as a type, along with the other >>> types I mentioned in my last email. A listing of all the different >>> ways/types/categories in this data would be nice to have. >>> >> >> I can tag them afterwards. >> I have a tag tool as well to do a report. >> the most important thing is to get rid of the duplicates >> >> >> >>> >>> The data is missing a gml file. This is necessary to run the >>> geobase2osm.py step. You could probably use ogr2ogr to do it, but you'd need >>> proper id or et_id, whichever would work better as a unique identifier >>> (probably not id, since it's not available in urban areas!). >>> >> >> hmmm... >> >>> >>> I did get both the OSM data and the (I assume) government data into >>> openjump roadmatcher, so it would be possible to roadmatch with what I have, >>> but not very useful. >>> >> >> why not? >> thank you for your efforts to you irc? come to irc.freenode.net #flossk >> or add me on skype : h4ck3rm1k3 >> >> thanks, >> mike >> >> >>> >>> Adam >>> >>> >>> On Wed, Mar 17, 2010 at 2:47 PM, Adam Dunn <[email protected]> wrote: >>> > >>> > Hi, I was kind of hoping someone more knowledgeable in GIS programming >>> would've responded first for the following reasons: geobase2osm.py is an >>> integral part of the whole roadmatching process for Canada, and it's a very >>> Canadian-specific script (there are chunks of code that deal with naming of >>> major trunks on a province-by-province basis using if-else programming). >>> > >>> > Also, I don't have much experience programming GIS stuff. For example, >>> the Canadian way of doing this uses EPSG 3348 for projection [1]. I did some >>> Google searching, and it looks like you would want to use EPSG 4191 for the >>> Albania area (see [2]), but 2462 might also the one you want to use ([3], >>> although it looks like you get weird projected bounds with it). I don't know >>> why this reprojection is really even necessary. When you look at EPSG 3348 >>> (the Canadian one), the projected bounds are really weird there as well, so >>> maybe it has to be done just to match up with NRN. If the Albania dataset is >>> already in Lat/Long and 4191 is in Lat/Long and OSM is in Lat/Long, maybe >>> you don't need to reproject at all? Someone with more GIS knowledge should >>> know. >>> > >>> > You'll also want a script to automatically convert various attributes >>> in the shp file to tags that are used by OSM. For example, I opened up the >>> Roads_OSM.shp file in OpenJUMP (just the way it is, no changes or >>> reprojections or anything), and I see that one road has the following >>> properties: >>> > ID: 175879.0 >>> > Type: Well-Kept Gravel Road >>> > Category: 2 >>> > Cat2: 20 >>> > Shape_Leng: 0.0 >>> > Name: >>> > Shape_Le_1: 774.851 >>> > ET_ID: 167693 >>> > >>> > Most of these you probably won't care about, but Type: Well-Kept Gravel >>> Road should be converted to >>> > surface=gravel; >>> > and highway=unclassified or residential or track=* depending on what >>> the Category: 2 and Cat2: 20 mean. >>> > >>> > Another road has Type: Dwelling Area Road, which would probably be >>> highway=residential; >>> > >>> > I've also seen Type: Village Road, Type: Railway, Type: Seasonal Road, >>> and Type: National Asphalted Road. >>> > >>> > You can see where geobase2osm.py makes similar mappings for Canada, by >>> looking at lines 63 to 77 in geobase2osm.py [4]. >>> > >>> > None of the roads I sampled had names associated with them (even the >>> national highway, the name was blank). You'll probably have to match up road >>> names to the roads using one of the other data sets? >>> > >>> > I would like to help out more, but I'm afraid I don't have the >>> experience. I could lend a hand in modifying the SQL tables and >>> geobase2osm.py, but I would be of limited help there, and I haven't a clue >>> how you would get road names imported. You'll want more expertise help in >>> getting the proper toolchain. >>> > >>> > [1] http://spatialreference.org/ref/epsg/3348/ >>> > [2] http://spatialreference.org/ref/epsg/4191/ >>> > [3] http://spatialreference.org/ref/epsg/2462/ >>> > [4] >>> http://svn.openstreetmap.org/applications/utils/import/geobase2osm/geobase2osm.py >>> > >>> > Adam >>> > >>> > On Wed, Mar 17, 2010 at 12:31 PM, Sam Vekemans < >>> [email protected]> wrote: >>> >> >>> >> Hi Adam, >>> >> James is looking to use RoadMatcher, i know its on the wiki, but that >>> >> page needs to be fixed. >>> >> Would you be able to explain it? >>> >> >>> >> Personally, i now favour tracing WMS, but for massive areas >>> >> RoadMatcher is the way to go :-) >>> >> >>> >> Also, Robert (Bob) Shand from PEI is currently Itching to also learn >>> >> how its done. >>> >> >>> >> Cheers, >>> >> Sam >>> >> >>> >> ---------- Forwarded message ---------- >>> >> From: Michael Barabanov <[email protected]> >>> >> Date: Wed, 17 Mar 2010 12:13:39 -0700 >>> >> Subject: Re: [Talk-ca] call for help, importing roads >>> >> To: "[email protected]" <[email protected]> >>> >> Cc: [email protected] >>> >> >>> >> Sounds like you just need to convert SHP to OSM. Then you can upload >>> from >>> >> JOSM. There's a script here (though I haven't tried it): >>> >> http://redmine.yellowbkpk.com/projects/list_files/geo >>> >> >>> >> On Mon, Mar 15, 2010 at 12:48 PM, [email protected] < >>> >> [email protected]> wrote: >>> >> >>> >> > Hi, >>> >> > I have been talking to Sam V. who mentioned that you guys are >>> experts in >>> >> > openjump for road importing. >>> >> > before I dig too deep into this myself, let me ask if anyone can >>> help me >>> >> > with my current problem: >>> >> > >>> >> > I have these files from a public domain source, projected into >>> Lat/Lon >>> >> > >>> >> > http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/ >>> >> > >>> >> > [image: [ ]] LL_Roads_OSM.dbf< >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.dbf >>> >05-Mar-2010 >>> >> > 07:48 53M [image: [ ]]LL_Roads_OSM.prj< >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.prj>05-Mar-2010 >>> 07:48 143 >>> >> > [image: [ ]]LL_Roads_OSM.shp< >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp>05-Mar-2010 >>> 07:48 >>> >> > 35M [image: [ ]]LL_Roads_OSM.shx< >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shx>05-Mar-2010 >>> 07:48 >>> >> > 1.5M >>> >> > They are from http://tpginc.net/gis/albania/albania.php >>> >> > >>> >> > we would like to find the new roads that are not in OSM and import >>> them. >>> >> > Can anyone help? can you tell me exactly what software to install, I >>> am a >>> >> > bit confused my the many pages and broken links I found for jump. >>> >> > >>> >> > thanks, >>> >> > mike >>> >> > >>> >> > _______________________________________________ >>> >> > Talk-ca mailing list >>> >> > [email protected] >>> >> > http://lists.openstreetmap.org/listinfo/talk-ca >>> >> > >>> >> > >>> >> >>> >> >>> >> >>> >> -- >>> >> Twitter: @Acrosscanada >>> >> Blog: http://Acrosscanadatrails.blogspot.com >>> >> Facebook: http://www.facebook.com/sam.vekemans >>> >> Skype: samvekemans >>> >> OpenStreetMap IRC: http://irc.openstreetmap.org >>> >> @Acrosscanadatrails >>> > >>> >>> >> >
_______________________________________________ Talk-ca mailing list [email protected] http://lists.openstreetmap.org/listinfo/talk-ca
