Re: [OSM-dev] multipolygons - dealing with topology errors
On Tuesday 21 July 2009 09:59:29 Jochen Topf wrote: > On Mon, Jul 20, 2009 at 10:36:21PM -0400, Ben Supnik wrote: > > Two things about multipolygons: > > > > 1. My software now detects topologically broken multipolygons, e.g. > > cases where I can't form rings out of the node list. Is this > > useful, or is this information already generated by things like > > pgsql? > > > > 2. While looking at one test set of data, I found a lake where one > > island (modeled as a hole in the lake multipolygon) had not been > > "sealed". > > > > Mapnik simply ignored the island, but osmarender had linked the > > start and end of the ways to form the island. > > > > I have learned from experience that it isn't useful for me to ask > > "what's the spec"...OSM is its own spec, etc. etc. > > > > So I'll just ask: any comments? Is one interpretation considered > > better or worse than the other? > > Depends on what you are doing. When you are generating a map that OSM > mappers will see, its probably better not to draw anything, to > encourage people to fix it. When using the data for maps not related > to OSM directly, it might be better to automagically fix things, > because people wont fix it anyway. With Temap, I found it is much better to draw such things in bright red, that way they get fixed much much faster than when you just drop them - out of the sight, out of the mind :) All the best, Tels -- Signed on Thu Jul 23 09:39:03 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "A witty saying proves nothing." -- Voltaire signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Fw: [Geowanking] [Fwd: [Ann] LinkedGeoData.org]
On Friday 17 July 2009 02:26:11 andrzej zaborowski wrote: > 2009/7/15 Mikel Maron : > > From discussion about linkedgeodata.org/ on geowanking... > > > > From: Sean Gillies > > To: geowank...@geowanking.org > > > >> I'm skeptical about RDF too, but the linked geodata folks are > >> adding some extra value (at least for a particular group of > >> users): hypertext, so that you or your software can follow your > >> nose through > >> linked nodes and ways without having to know an API specific to > >> OSM, and persistent URIs. I'm not knocking OSM's API, but > >> > >> http://api.openstreetmap.org/api/0.6/node/264695865 > >> > >> isn't meant to be a "cool URI" for the Cafe B'liebig, is it? > >> Requests for the version 0.5 URIs like > >> http://api.openstreetmap.org/api/0.5/node/1 return a 403, which > >> suggests to me that I shouldn't get too > >> attached to the 0.6 ones. Of course, URIs like > >> > >> http://linkedgeodata.org/triplify/node/264695865 > >> > >> have their own issue. Stamping the name of the semantic web > >> framework you happen to be using today on the URIs you want to > >> make future-proof isn't a cool thing to do. > > > > Sean has a very good point here about the permanence of URIs. > > Permalinks to OSM objects permits integration of these identifiers > > into other systems. At the moment, we don't have permalinks, > > because the API version is included in the URL. Perhaps the API > > could also support read-only, permalinks for objects, like > > http://api.openstreetmap.org/api/node/264695865 > > I don't believe linking to a XML snippet defining an object in osm is > extremely useful, but I can see other uses for objects in OSM being > universally "addressable" in the URL space (e.g. comparing whether > two objects are the same). In that case the word "api" probably > shouldn't appear in the URL either and also there's no point in it > being a http URL, so why not define something like > osm://openstreetmap.org/node/XXX as being the URL of a node in OSM. > This could then be easily translated by client into a http URL for > the xml snippet or the history web page for the object. It would still pose the problem that nobody/nothing prevents the node moved 100km south-west, all tags renamed/removed/rewritten, and a new node created at the old place with the same tags instead... All the best, Tels -- Signed on Sat Jul 18 17:46:52 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "... [the wireless lightbulb] rivals the invention of the cordless lightsaber. Luke Skywalker used to lose a LOT of battles until he ditched that awful extension cord. Kept getting his feet tangled up. And Count Doofus would laugh as he yanked on it and watched Luke fall on his ass. Also, sometimes opponents would pull the plug out of the wall socket and snicker at Luke's bewilderment. Yoda would just smack his head and say 'Duh-oh! The Force is not especially smart in this young one! Save up for Duracell adapter, he must.'" Walt Dismal (534799) on /. 2007-06-08 signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] A consistent format for the multipolygon relation
On Tue 16/06/09 10:45 , Andreas Kalsch andreaskal...@gmx.de sent: > I don't like this way of discussion. It leads to nowhere ... While you are right about the tone that seems to have developed, I think it is actually a very useful and important discussion. Sorry for intruding here, I wanted to reply to Wolfgang directly, but thanx to the crappy webmail interface I can't find his email so I am replying to this email instead :) > The point I wanted to demonstrate was to start an initiative to get > multipolygons into the right format because the quality of OSM data is > crucial for all projects which use it. To get a more consistent > definition of multipolygons is an important step to make it easier for > everyone (including the map renderers) to build on top of the data. I > don't want any silver plates, too, but bite-sized data. > > Pretty easy, huh? That is a very noble goal, and I fully support it. Unfortunately, it seems that OSm is heading into the opposite direction :( Instead of getting more consistent and easy-to-interpret-and-use data, all we get is more and more garbage, duplicated information (attribution="blah" on every node!), badly designed data structures (multipolygons, I look at you!) and inconsistencies (fuzzy logic like "true, false, 0, 1, -1, maybe, yes, not today") and so on with every passing week. I might be biased because I am a "user" of the OSM data (see http://bloodgate.com/wiki/map) and I am not happy about the current state of affairs when I see what is all in the DB. However, I am *also* a mapper and I am not happy about the current editing/tagging helps, either. You never really know how to tag something right (the map features wiki pages is highly incomplete and sometimes outright conflicts with the real world usage), it always takes time to find out sometimes even simple things, and everytime I map, I just cry when I see the mistakes of others, then wondering, should I correct them? Or better not? What if I accidentily destroy something? "Making it easy for mappers" is certainly not a reality on OSM, unless you are already an experienced mapper. People who are new to OSM struggle even more. Also, letting the mappers design the data structures only leads to the situation where "data entry" is easy, but "data extraction" is hard. Since "data entry" happens only once, and "data extraction" multiple times, this is exact the opposite on what you would design around! If we follow this train of thought to the end, we better create a "write-only" OSM editor, so mappers have it absolutely easy, and no-one needs to worry about making something out of the data later :) Sure, without mappers there is no data, but without any applications using that data there are no users, and without users, the entire project becomes a moot point, anyway. Anyway, in closing I think that the OSM data representation and structures as well as their documentation leave a lot to be desired and improving them would be benefit us all, mappers, coders and most importantly, users (as in "using the map to find places") alike. And now I am back to programming :) Tels ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] JSON/GeoJSON output format for 0.6 api
On Thursday 04 June 2009 19:48:30 you wrote: > 2009/5/31 Frederik Ramm : > > Hi, > > > > Tels wrote: > >> Well, one could fetch the data at z17, see it is below some > >> $ARBITRARY_THRESHOLD, zoom out to z16m, fetch again, and if still > >> below $THRESHOLD, repeat it until either there is too much data > >> (display message) or the user-requested zoomlevel was reached. > > > > I think the ti...@home folks already keep a database that says how > > complex each level-12 tile is. So if we were not so busy telling > > them how they're technologically backwards and how their whole > > project is rubbish, they might just give us that. > > Each t...@h user uploads a tileset at z12, which is indeed a pretty good > indicator of the complexity of the area, see this heatmap rendering > of the world at z12 from the t...@h data: > > http://u.nix.is/~avar/osm-heatmap-black.png > > I made if after requesting the z12 tileset sizes on the t...@h list: > > http://lists.openstreetmap.org/pipermail/tilesathome/2009-May/005859. >html > > The code to generate it is under applications/rendering/tah-heatmap > in SVN: > > http://trac.openstreetmap.org/browser/applications/rendering/tah-heat >map/README > > In particular the parse-filesize.pl script in that directory converts > ls -R format provided by spaetz to an easy-to-use CSV format, e.g.: > > ,1042 158211 > ,1043 203915 > ,1055 172469 > ,1056 80728 > > t...@h currently has around a million z12 tiles, or ~5.6% of potential > z12 tiles: > > $ wc -l tile-sizes.dat > 937254 tile-sizes.dat > > It would be easy to make a web service which kept these tile sizes in > a hash table and told e.g. Potlatch whether or not the z13 tile it's > requesting is part of a z12 area that's say 10 times bigger than the > median z12 tile. Which should cover the yellow/read areas on the > above heatmap. If such a webservice is made public, I could use it to determine wether an area of 0.1° or 1° should be requested. However, getting a server that can respond to API requests in real-time (e.g. saturate the download link) would be the preferable situation - after all, all the data needs to be downloaded for rendering, anyway. (Rendering only half the area the user wants to see is useless :) All the best, Tels -- Signed on Thu Jun 4 21:36:19 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "I live the way I type; fast, with a lot of mistakes." -- Unknown signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] JSON/GeoJSON output format for 0.6 api
On Wednesday 27 May 2009 15:20:26 Richard Fairhurst wrote: > Ed Loach wrote: > > Is there anywhere though that z13 Yahoo imagery is of any use? > > Sure. It's ok for lakes, woods, built-up areas, that sort of thing. > I'm just looking at somewhere near Rhayader and it's pretty useful. > > > I like Shaun's suggestion of defaulting to z17 and let users zoom > > out. > > So would I, if I too lived somewhere with more people than sheep. The > problem with that is when (say) you want to edit a lake somewhere in > mid-Wales, you navigate to it using View as per usual, then click > "Edit", and that zooms in to somewhere with absolutely no data. At > which point you send a message to talk@ saying "where has the data > gone I blame Potlatch pls to revert the changeset omgwtfbbq" - I'm > not joking - and we all get a headache. > > From a usability point of view the View and Edit tabs should be > equivalent - i.e. wherever you're viewing, clicking 'Edit' preserves > the bbox. > > We currently have a JS alert at z1-z10 saying "zoom in to edit map", > and we italicise the tab to show that it's not clickable. (It should > really be z1-z12, I'm not sure why it isn't.) > > This is good. If it were context-sensitive depending on the amount of > data in the area, it would be wonderful. Well, one could fetch the data at z17, see it is below some $ARBITRARY_THRESHOLD, zoom out to z16m, fetch again, and if still below $THRESHOLD, repeat it until either there is too much data (display message) or the user-requested zoomlevel was reached. Might fetch more data (double fetching) in a few cases, but might spare huge requests for dense areas. All the best, Tels -- Signed on Thu May 28 18:33:50 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "Memory is like an orgasm. It's a lot better if you don't have to fake it." -- Seymore Cray, on virtual memory signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] JSON/GeoJSON output format for 0.6 api
On Wednesday 27 May 2009 14:22:27 Richard Fairhurst wrote: > Tom Hughes wrote: > > Richard Fairhurst wrote: > >> Out of interest, do we have any stats on the proportion of server > >> load caused by each client? > >> (not sure I really want to ask that question...) > > > > I'm not sure you did either ;-) > > Heh. > > I suspect a lot of the Potlatch load is caused by people trying to > edit large areas at z13. > > We pretty much have to keep offering z13 because for most of the > world, it's the only level at which Yahoo imagery is available. What > would be good (we discussed this on IRC) would be if Potlatch - or > the Rails site, which is after all the code that actually tells > Potlatch "edit this at this zoom, this latitude and this longitude" - > could access some metric or other to find out how dense an area is, > and then zoom in accordingly if there'll be too much to edit. One for > the hack weekend maybe... I was thinking about this problem, too. My code currently fetches areas of 0.1° * 0.1° but this can be a pain when the area is dense, and wastes time when the area is very sparse. So, is there any way to access somehow how dense one area is? All the best, Tels -- Signed on Wed May 27 14:34:45 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "The Palmer proposal. What ever happened to that?" signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] API 0.6 returns 403 forbidden
On Thursday 14 May 2009 18:40:54 you wrote: > Tels - I'd like to reiterate my offer of a hosted machine with lots > of bandwidth and space for a planet dump for our applications. But i > also agree that another solution might be better long-term. > For the short term, are you interested in working with me to get a > dump running and serving... JSON? Of course! Sorry that I didn't reply the last days, I was extremely busy and had not a lot of time to work on OSM :( Getting a dump up and running is probably something we both need anyway. I'll send you an off-list email in 10 seconds :) All the best, Tels -- Signed on Thu May 14 18:52:28 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "Karate is a form of martial arts in which people who have had years and years of training can, using only their hands and feet, make some of the worst movies in the history of the world." -- Dave Barry signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] API 0.6 returns 403 forbidden
Moin, On Thursday 14 May 2009 18:35:32 Tels wrote: > Moin, > > On Thursday 14 May 2009 09:31:56 Tom Hughes wrote: > > Jeffrey Warren wrote: > > > Hi, I suddenly started getting "403 - Forbidden" from my any > > > machine. I also get a 403 when i simply go to > > > "www.openstreetmap.org > > > <http://www.openstreetmap.org>" in a browser (just discovered > > > this...). Have I been blacklisted or is this an outage? > > > > I suspect you're the person I blocked last night because you were > > apparently trying to scrape map data from the API with a large > > number of presumably automated map calls. > > > > If you are the person I blocked then in the last four days you've > > made a total of 8186 map calls, an average of about 1.5 a minute. > > > > If you want bulk data, please use the planet dump not the API to > > get it. > > I presume my own application (http://bloodgate.com/wiki/map) will > then be blocked, too. > > Using the "plant dump + diffs" sounds easy, but is in fact impossible > for me. My online server does not have the capacity to store the > entire world, and I do not have the computing power and time to > convert it up-front, either. (Not to mention do upload the converted > data to my server from my home DSL line :( > > (There is also the question what happens if importing an hourly diff > takes regulary longer than 60 minutes. So far the only data I have on > that is someone mentioning that it "takes between 40 and 70 minutes > with a fast harddisk"...) > > Also, to download the whole dump, update it with minute diffs and > pregenerate a huge database is vastly more complicated and uses a lot > more processing power than just to fetch (and cache for X days) the > few areas someone is actually looking at. I know it's bad style to reply to yourself, but :) * took out api.openstreetmap.org from osmapi at bloodgate.com for now * the API server is waay to slow, in any case, so I am looking at the planet dump method, but it will take quite a while to developt the converter and I am not sure it will actually work Still, it would be good to know what the plans for the future are and how one could help to improve the situation. How expensive would it be to run an XAPI server like xapi.informationfreeway.org? All the best, Tels -- Signed on Thu May 14 18:41:06 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "Un bon mot ne prouve rien." -- Voltaire signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] API 0.6 returns 403 forbidden
Moin, On Thursday 14 May 2009 09:31:56 Tom Hughes wrote: > Jeffrey Warren wrote: > > Hi, I suddenly started getting "403 - Forbidden" from my any > > machine. I also get a 403 when i simply go to > > "www.openstreetmap.org > > <http://www.openstreetmap.org>" in a browser (just discovered > > this...). Have I been blacklisted or is this an outage? > > I suspect you're the person I blocked last night because you were > apparently trying to scrape map data from the API with a large number > of presumably automated map calls. > > If you are the person I blocked then in the last four days you've > made a total of 8186 map calls, an average of about 1.5 a minute. > > If you want bulk data, please use the planet dump not the API to get > it. I presume my own application (http://bloodgate.com/wiki/map) will then be blocked, too. Using the "plant dump + diffs" sounds easy, but is in fact impossible for me. My online server does not have the capacity to store the entire world, and I do not have the computing power and time to convert it up-front, either. (Not to mention do upload the converted data to my server from my home DSL line :( (There is also the question what happens if importing an hourly diff takes regulary longer than 60 minutes. So far the only data I have on that is someone mentioning that it "takes between 40 and 70 minutes with a fast harddisk"...) Also, to download the whole dump, update it with minute diffs and pregenerate a huge database is vastly more complicated and uses a lot more processing power than just to fetch (and cache for X days) the few areas someone is actually looking at. So, how will OSM handle this situation in the future? Will there be a fast API server (plus a few "shadow", copy servers) than can deliver tile data in (semi) real-time, or will each and every person that wants to create a "view" of the data have to go to the hassle of dealing with the entire "download dump, update it and convert it" scheme? The latter seems like it will rule out quite a few applications just because it places a really high burden on each and every project to duplicate the same work. And that is without the small detail that next year the dump might be easily twice as large :) All the best, Tels -- Signed on Thu May 14 18:26:38 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. ". . . my work, which I've done for a long time, was not pursued in order to gain the praise I now enjoy, but chiefly from a craving after knowledge, which I notice resides in me more than in most other men. And therewithal, whenever I found out anything remarkable, I have thought it my duty to put down my discovery on paper, so that all ingenious people might be informed thereof." -- Antony van Leeuwenhoek. Letter of June 12, 1716 signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
Moin, On Thursday 07 May 2009 01:46:36 you wrote: > d) oh, and localStorage. I've partially implemented that but haven't > had much testing... other work... ugh. So caching on a few levels, > basically. Ah, I think I get it now. (yeah, took a long time :) localstorage could be used when you are offline, so you can load a map, store it, diconnect and then go into the wilderness and still view your map? That would be cool, because currently one would need to install my proxy locally to keep the proxy-side data and that is cumbersome. All the best, Tels -- Signed on Mon May 11 08:19:16 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. Neulich in Dresden gehört: "Gundach. Schbindadoni. Isvleisch dadidada?" -- Ex-Kahl-Libur signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Finding water vs. land
Moin, my renderer downloads data for tiles (currently in the 0.1° grid) and then renderes these. This works fine, except for coastlines, where I have two problems: * The coastlines are just open lines, but I need closed areas for rendering * When I download data in the middle of the ocean, no coastline crosses the tile and I do not know that the current tile must be filled completely with water That there is some coastline data for mapnik, but I don't quite understand how that process works. So how would I: * figure out that a certain area is water-covered? * get a complete, closed coast line (possible for the entire world?) and both best via some HTTP request :) All the best, Tels -- Signed on Sat May 9 14:43:52 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "I'm not a vegetarian, but I eat animals who are" -- Groucho Marx signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
Moin, On Friday 08 May 2009 22:09:17 you wrote: > Great, this is a good discussion. I've put up a wiki page with some > of the things we've covered, with pros/cons. I hope we can continue > to talk about our approaches and as we optimize for different > problems post some of it back up here: > http://code.google.com/p/cartagen/wiki/FeatureTradeoff > > I put in what I could gather about Temap, but feel free to update and > add more pros and cons... this is just my thought process so far. We > might also add a "status" column so we can annotate what we learn > from each approach. Ah, interesting. Since I don't have a google account, I reply here instead: * Server-side proxy and filter Yes ? Temap has a server side proxy and filer, called "osmapi" and it is running at http://bloodgate.com/cgi-bin/osmapi :) * loading data live & direct from an API server Makes editor possible: I think that editing would be even possible if you do not load live from the API server, because you could: A: instruct the proxy to bypass its cache and reload from the API server again, and then upload the changes back to the proxy, which in turn uploads to the API server. (Due to cross-domain security the JS cannot talk to the API server directly unless a bit of JS is served from there, too. (There is a possibility, but it only works in Firefox 3.5 IIRC) * Pruning datasets before handing to JavaScript Under cons "Looses important metadata" - I would say "non-important metadata". The only real con is that if you need that metadata (f.i. for editing), you need to download it for the features the user wants to touch and edit. Or in other words, you do not need to download every "FIXME" and "attribution" just to render the map, but when the user inspects features, you need the data. Like the live-loading above, I think basically it will boil down to hybrid approaches. E.g. you load the bulk in a pruned way from the proxy, but if the user wants to change a street, you download the data for that street directly and 100%, then upload the change. * Serve reduced polygons for lower zoom levels I would add "unclear how much it speeds things up". I might be that the entire coastline of Europe is less data than lets say the inner parts of Washington DC :) * use localStorage to persist a cache in the browser Again, I don't see what localStorage adds over the traditional browser cache - if I download 7.3,50.4,7.4,50.4.json.gz it will get cached and the browser will fetch it from the cache next time (when the server says the data is not too old). That automatically limits the storage (user settable!), validates the freshness of the data etc. The traditional cache is very good at solving these things, so implementing it manually in localstorage seems like re-inventing the wheel to me. (You can prove me wrong, tho :) Con: Doesn't work in all browsers (which ones do have it btw?) * Request plots filtered by tag Yes ? temap does this, it loads reduced datasets for zoom level 11, and for 10 and below. When you zoom in, it loads the full data set. (What it doesn't is reduce the dataset, once you have a certain level of data, zooming out will just reduce the data during rendering by skipping things. That's because I figured that pre-filtering the data client side would be as much as work as jus skipping parts. However, that might be able to be improved upon. In summary I would also like to add that all these various "pre-computation" and "caching" strategies are quite nice and helpful, but they are also all "premature" optimizations in the sense that I'd first get it to work at all, then toy around reducing the work. E.g. rendering labels on a canvas is problem that is not solved in all browsers, and no matter how much or little you cache, it won't change the fact that Opera 9.6 has no labels on the map :( Btw, jeffry: > > > > * There is a talk I proposed for State of the Map and I don't > > > > want to spoil everything before :) > > > > > > yes, me too! so if you want to discuss off-list that's fine. > > > > Heh, you have a talk scheduled, too? :) That sounds like fun :) Will you be at the conference? :) All the best, Tels -- Signed on Sat May 9 13:37:44 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life." -- Terry Pratchett signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
On Friday 08 May 2009 15:34:13 Tels wrote: > Moin, > > > But you're right, it's a challenge. I'm impressed that you rendered > > a whole city like Berlin - do you have some code online so I can > > see, or a screenshot? I bet it looks great... > > http://www.bloodgate.com/wiki/index.php?title=Temap_-_Screenshots :) > > Actually, I managed to render Bonn, Germany, but Berlin, Germany is > out because the amount of data exceeds Firefox stack limit. Oops. > > I'll upload a new screenshot soon. Done: http://www.bloodgate.com/wiki/index.php?title=Temap_-_Screenshots The page also contains a few detals to the data size and timings. The red areas were features where I had missing render rules, fixed a few of them now. The rest are wrongly or untagged ways. All the best, Tels -- Signed on Fri May 8 21:02:52 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "Wo die Schoschonen schön wohnen." signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
threw together yesterday, based on the tags of measured > width instead of on a width rule: > > http://map.cartagen.org?gss=http://unterbahn.com/cartagen/width.gss > > A more fully-rendered screenshot is here: > > http://www.flickr.com/photos/jeffreywarren/3510685883/ Yeah, that is what I have in mind, too. But so many things to do, so little time :) > Anyways, thanks for sharing; one thought I had was that besides > sharing ideas and solutions online, we should try *different* > approaches, so that we try all the possibilities. I think multiple > projects working on the same problem can sometimes be redundant, but > more often it's beneficial for all parties since there's a diversity > of approaches to a problem. Let's take advantage of that by > specifically attempting different solutions to the problems we face, > and discussing the results... if you're willing. If one of us tries a > technique and it doesn't work, we can all learn from the attempt. Sure, I am working on my ideas, anyway :) A few things you might find interesing: * no dashed lines on canvas, need to roll your own * rendering 6 lines/areas takes a long time (>1minute), which means you need a sort of "slippy tiles" setup like I have currently. That allows the user to pan the map in real-time and the renderer can only render tiles off-screen. All the best, Tels -- Signed on Fri May 8 20:47:12 2009 with key 0x93B84C15. Get one of my photo posters: http://bloodgate.com/posters PGP key on http://bloodgate.com/tels.asc or per email. "If Duke Nukem Forever is not out in 2001, something's very wrong." -- George Broussard, 2001 (http://tinyurl.com/6m8nh) signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
On Thursday 07 May 2009 02:51:34 you wrote: > Jeffrey Warren wrote: > Since an area will always be closed any segment can be taken to be > build upon [as start point]. Even without the routing data the object > can be fully connected, based on the start and endpoints. How does that scheme deal with multipolygons? (e.g. areas with holes in it, as these seem to be getting popular :) All the best, Tels -- Signed on Fri May 8 15:35:17 2009 with key 0x93B84C15. View my photo gallery: http://bloodgate.com/photos PGP key on http://bloodgate.com/tels.asc or per email. "Sacrificing minions: Is there any problem it CAN'T solve?" -- Lord Xykon signature.asc Description: This is a digitally signed message part. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
Moin, On Thursday 07 May 2009 01:46:36 you wrote: > Hi, Tels - > > > > It's not been optimized yet, so loading is a little slow, but I'm > > optimistic > > > that it will scale. > > > > Based on my experience, I can tell you right away it won't scale :) > > Not to discourage you, but: > > > > * the amount of data is really huge. Throwing a few dozend megabyte > > XML or even > > JSON at the browser will bring it to its knees. Not to mention the > > data you need > > to render even a small city. > > * re-rendering everything takes a long time. You want to avoid that > > :) > > I was actually talking about server-side load time. I'm running it > off the 0.6 API, so it packs up XML, sends it to my server, i unpack, > re-encode to JSON, send to the browser, render. Obviously that's > SUPER inefficient, so I'm looking forward to cutting a lot of that > out in the next week or so. Heh, that sounds like my setup :) My client requests data in (currently) 0.1° tiles, like 7.4,50.1 - 7.5, 50.2 from a proxy. The public proxy runs at http://bloodgate.com/cgi-bin/osmapi and serves these requests as gzipped JSON. It is also possible to run one locally so you don't need internet access. (I intend to document it, please give me a day or two). * The proxy receives XML from the api or xapi server. Currently it requests the full dataset. * Then it removes unnec. tags (like note, fixme, attribution and a whole bunch of others that are not needed for rendering). Some of them are very minor, but 1 nodes with "attribution=veryvery long string here" can make up like 40% of all the data, and just clog the line and browser :) * The data is then converted into a structure that is easier to work with (nodes as hash, ways as list, multipolygon ways attached to the outer polygon to avoid issues with the render order etc.). * The data is then pruned into (currently 3) levels and stored in a cache: * level 0 - full * level 1 - no POI, no paths, streams, tracks etc. used for zoom 11 * level 2 - no tertiary roads etc. used for zoom 10 and below * The client is served the level it currently requested as JSON.gz. That scheme can surely be improved but it works for now. The problem with that scheme is that: * There are three servers in the list (api.openstreetmap, xapi.informationfreeway and tagwatch) and a lot of them do not complete the request (internal error, not implemented etc. etc.). It can take a lot of retries to finally get the data. * Even when you get the data, it takes seconds (10..40 seconds is "normal") to minutes - upwards to 360 seconds just to serve one request. So currently all received data is stored in the cache for 7 days to avoid the very very long loading times. Ideas of fetching the full dataset and pre-computing the cache simple don't work because I don't have a big enough machine and no big enough online account to store the resulting JSON :( > Actually, rendering in the browser's been pretty good - for example > this page loaded with no noticeable slowdown, and I haven't even > begun optimizing: > > http://www.flickr.com/photos/jeffreywarren/3476532351/ > > But you're right, it's a challenge. I'm impressed that you rendered a > whole city like Berlin - do you have some code online so I can see, > or a screenshot? I bet it looks great... http://www.bloodgate.com/wiki/index.php?title=Temap_-_Screenshots :) Actually, I managed to render Bonn, Germany, but Berlin, Germany is out because the amount of data exceeds Firefox stack limit. Oops. I'll upload a new screenshot soon. > What I'm looking at now is: > > a) rendering only some tags per zoom-level, so no rendering footpaths > and buildings as you zoom out... but that's dependent on the xapi, > which I haven't been able to fetch from reliably (help anyone?) I already have that implemented, the render rule specifies for which zoom level that feature applies. Plus a few other hard-coded rules like "no borders from zoom X upwards", but these should be pushed into the ruleset, just like the default map background should be pushed there. (Nice idea :) > b) cutting the API out of the loop and running direct from a > planet.osm, but then you can't use it to view live edits, like you > can here: http://vimeo.com/4435969 Also, somehow processing 150 Gbyte XML into JSON will prove to be a challange :) > c) trying to serve partial polygons... I'd like to try plotting only > every 3rd or 10th node... do the polygons collapse? Can i cull nodes > in a more intelligent way? Someone on this list or geowanking pointed > to a company that can serve lower-res polys over an API. I'm sure > folks have worked on this in til
Re: [OSM-dev] Cartagen - client-side vector based map renderer, dynamic maps
Moin, 2009/4/25 Jeffrey Warren : > I'm working on a Javascript map renderer, non tile-based. It's really > early-stage alpha, and not publicly released yet, but I'd love some feedback > from folks as I'm continuing to develop it. Sorry for not replying directly or earlier, I wasn't subscribed to this list until 5 minutes ago :) Initially I didn't want to spread my project to far, as it was (and is) still quite beta. However, now that so many people start pursing the same idea... ;) Jeffrey, you wrote: > It's not been optimized yet, so loading is a little slow, but I'm optimistic > that it will scale. Based on my experience, I can tell you right away it won't scale :) Not to discourage you, but: * the amount of data is really huge. Throwing a few dozend megabyte XML or even JSON at the browser will bring it to its knees. Not to mention the data you need to render even a small city. * re-rendering everything takes a long time. You want to avoid that :) My app has already quite a few optimizations, and it still chokes at big cities like Berlin or London. However, I am confident that things can be improved :) (Browser limitations non-withstanding. Single-threaded dead-slow JS and incomplete Canvas spec without dashe I hate thee... :( Regarding the rule sets and CSS: I've already considered adding a different rule-set (just to show that it can be done). However, from a technical viewpoint, that is not that spectacular. As long as the renderer is flexible enough to handle the wanted features, it doesn't really matter in what format the rules are (CSS, GSS, JSON, XML, you name it) or where they come from (hard-coded, web, URI, user input), as long as you can load, parse and convert them, it can display them. In my eyes the much bigger impact is that you no longer need different sets of tiles or tile providers - just the data and the rules to display it and the map can morph in real-time from mapnik to cyclemap to whatever-you-want. And one more button click and the user can save it locally. That's at least my vision I work towards :) All the best, Tels ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
