My planet.osm download cancels every time. I do it via Firefox. My first
try canceled @ 237M, my second @ 1.1G
I tried it via curl: (18) transfer closed with 4171221735 bytes
remaining to read
Why do the downloads cancel every time although my disk has enough space
left?
Andi
coordinates you could index by virtual
tiles as it is done in OSM's main DB since a year ago with nice
performance boost:
http://wiki.openstreetmap.org/index.php/QuadTiles
good luck,
Stefan
On Thu, Sep 11, 2008 at 1:49 PM, Andreas Kalsch [EMAIL PROTECTED]
wrote:
Hey,
last week I made
On Thu, Sep 11, 2008 at 08:31:23PM +0200, Andreas Kalsch wrote:
All, thanks for your quick responses!
Quad tiles look like a smart way to create an index. So to lookup a
single point or a quad tile, this is fine. But for my application I
need another lookup - by bounding box with any
My current optimization includes:
- using mediumint for lat/lon - enough for ~2 meters resolution
- using a bounding box first for point+radius calculation and then
selecting the circle with a Pythagoras approximation, which is exact enough
For the first, I want to use MySQL.
I have set up a OSM database with Germany data. Now I want to update my
data daily. I have not found diff files for Germany so I want to use the
global diff files, which are not too big.
To re-import German data every day would be too expensive - it already
took some hours.
In Osmosis, I tried
Brett Henderson schrieb:
Andreas Kalsch wrote:
I have set up a OSM database with Germany data. Now I want to update
my data daily. I have not found diff files for Germany so I want to
use the global diff files, which are not too big.
To re-import German data every day would be too expensive
Brett Henderson schrieb:
On Fri, Sep 19, 2008 at 2:18 AM, Andreas Kalsch [EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] wrote:
Thank you for your help.
The fourth point means that in my planned application I am not
going to use OSM database but just a subset of it - so I need kind
... are nearly 4 times faster than multi.colum indices for lat/lon in
MySQL. Extracting 1 column / 17.000 rows out of 1 million takes .39 secs
vs. 1.39 - I think all Postgres guys know a similar value. I think this
is a pretty impressing result.
Marcus Wolschon schrieb:
Andreas Kalsch schrieb:
... are nearly 4 times faster than multi.colum indices for lat/lon
in MySQL. Extracting 1 column / 17.000 rows out of 1 million takes
.39 secs vs. 1.39 - I think all Postgres guys know a similar value.
I think this is a pretty impressing
Osmosis is very slow on my server.
There is this option --write-null (--wn) which can be useful just to
check integrity of data. So I think Osmosis does everytime check it and
this could be a bottleneck because it has to cache some data in memory.
Can this be the origin for being slow (next to
I decompress the data before putting them into Osmosis, but it's still slow.
So back to my question -- ;)
(The best would be raw dump files for MySQL's LOAD DATA INFILE - I can
imagine that it would be pretty quick)
Stefan de Konink schrieb:
Frederik Ramm wrote:
The Java implementations of
What I will try now:
1) combine(There is another approach. ALTER all InnoDb tables to
MyISAM, run
osmosis and then ALTER them back to InnoDb.
It seems that the INSERT approach scales badly to large InnoDb tables.
Sincerely,
Joachim
, It scales always bad; no exceptions. Some advise on the
OK,
1) does not work
- Osmosis needs the InnoDB tables:
Write to database ..
Nov 30, 2008 2:35:09 PM com.bretth.osmosis.core.Osmosis main
INFO: Osmosis Version 0.29
Nov 30, 2008 2:35:09 PM com.bretth.osmosis.core.Osmosis main
INFO: Preparing pipeline.
Nov 30, 2008 2:35:09 PM
OK, I will create the tables as InnoDB tables directly, this is surely
better ;)
Joachim Zobel schrieb:
There is another approach. ALTER all InnoDb tables to
MyISAM, run
osmosis and then ALTER them back to InnoDb.
It seems that the INSERT approach scales badly to large InnoDb tables.
Be
I have taken a look on your script. I think it could be useful, IF it is
quicker than using Osmosis.
Do you have some benchmarks?
Best,
Andi
Joachim Zobel schrieb:
Am Samstag, den 29.11.2008, 18:45 +0100 schrieb Andreas Kalsch:
I decompress the data before putting them into Osmosis
Hi,
I have created a ranked geonames search with OpenStreetMap tag
statistics - based on extracted OpenStreetMap points of interest. The
ranking is experimental and currently just working for Germany. Give it
a try and let me know what you think:
http://78.47.150.5/opw/dev/
I cannot
Hi,
I have updated my application to find places in Germany. You specify an
OSM tag and the location. I will update the database as soon as possible
to include more data inside Europe and the US. My focus is now to
support more important features like editing and featuring third party APIs.
I
Hi,
I have imported a dump into MySQL with Osmosis and current_nodes has
more rows than nodes and the count of the other tables isn't equal
either. I think the current tables must have the some row count like the
history tables on initial dump. Probably I am wrong and you know more.
Andi
Problem was solved quickly. InnoDB shows false row counts in PHPMyAdmin,
to count(*) is the solution.
Hi,
I have imported a dump into MySQL with Osmosis and current_nodes has
more rows than nodes and the count of the other tables isn't equal
either. I think the current tables must have the
It is OK that Osmosis supports just 1.6, because it has no UI (has it?).
So there are 2 reasons:
- You use it more on servers with Linux, so 1.6 is supported.
- The user has the competence to deal with this kind of things,
including setting up his own DB.
JOSM - in opposite - should be as easy
Hi,
suddenly Osmosis does not connect any more to my database. The first
time this happend was while I was updating a database with a changefile.
When I tried to repeat the update the problem was there from the beginning.
- Connection data is correct (PHP connects successfully)
- Both
I installed it from Sun. You can see it in the trace below my entry:
...at com.sun.org.apache...
Roberto Navoni schrieb:
One week ago,
I had the same problem . Check if you are using sun java virtual
machine ... don't use other kind of java machine because you can have
some problem with db
This is a problem of MySQL. I have connection errors with the C API,
too. Has anyone had these errors with MySQL 5.0.x on Debian, too?
The error is: MySQL server has gone away (2006)
Andreas Kalsch schrieb:
Hi,
suddenly Osmosis does not connect any more to my database. The first
time
marcus.wolsc...@googlemail.com schrieb:
On Wed, 28 Jan 2009 00:07:20 +, Simon Ward si...@bleah.co.uk wrote:
[Moved to dev; followups to dev]
On Tue, Jan 27, 2009 at 10:58:25PM +, Ævar Arnfjörð Bjarmason
wrote:
I think multiple keys with the same name should be allowed
Shaun McDonald schrieb:
On 30 Jan 2009, at 15:11, Andreas Kalsch wrote:
Question: I have found a PGSQL schema for v0.6 where key and value are
TEXT fields. Is the length of 255 still correct for v0.6?
Are you sure it is the rails schema, there are other gpsql database
schemas for osm
I'd like to add several tags with the same key. In the most cases this
no good practice, but with urls it makes sense, e.g. to add several
images to one feature.
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
A function inside a PHP class I coded some time ago to populate a
database with Google Maps API data. The code is currently not maintained
any more so I am not sure if it's still working correctly. The input
string is not a complete geocode request but the thoroughfare of an
address, made of
of
the application and I need all the easy MySQL stuff like memory tables
and query cache.
Andi
Iván Sánchez Ortega schrieb:
El Viernes, 12 de Junio de 2009, Andreas Kalsch escribió:
Is there a script to gisify OSM relations - a script which creates
OpenGIS multipolygons or geometrycollections
Ramm schrieb:
Hi,
Andreas Kalsch wrote:
Is there a script to gisify OSM relations - a script which creates
OpenGIS multipolygons or geometrycollections inside PostGIS or MySQL
as WKT ?
No but there's this:
http://wiki.openstreetmap.org/wiki/Boundaries.pl
It generates .poly files from
Frederik Ramm schrieb:
Hi,
Andreas Kalsch wrote:
1) complete:
2) correct:
3) performant:
It strikes me as odd that someone who wants to write something
complete, correct, and peformant should deal with OSM. Are you sure
you have the right data for your philosophy, or the right
Don't understand me wrong. I like OSM's data model how it is. I have
studied the OpenGIS features and it makes sense to build a system on top
of OpenGIS if you are interested to index the geometries of OSM objects.
All a question of data representation. So in the end it would be great,
if all
I don't like this way of discussion. It leads to nowhere ...
The point I wanted to demonstrate was to start an initiative to get
multipolygons into the right format because the quality of OSM data is
crucial for all projects which use it. To get a more consistent
definition of multipolygons is
r...@mesolt11:/ops# osmosis/bin/osmosis --read-xml file=bremen.osm
--wd dbType=mysql host=127.0.0.1 database=api06_test user=xx
password=xx
Jun 24, 2009 11:52:54 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.31
Jun 24, 2009 11:52:55 PM
This would indeed be possible with current tiles - take a look at
http://maps.cartifact.com/ - it is built with Flash, but you could
easily build it with JS, too.
Christoph Boehme schrieb:
I had a similar thought a couple of weeks ago when I tried to find my
way using the map on the small
Tobias Knerr schrieb:
Andreas Kalsch wrote:
What about installing the Mediawiki API for the OpenStreetMap wiki?
This one?
http://wiki.openstreetmap.org/api.php
Thanks ;) Thought you use the standard endpoint path/w/api.php
___
dev
Marcus Wolschon schrieb:
On Wed, Jul 22, 2009 at 4:25 PM, Andreas Kalschandreaskal...@gmx.de wrote:
I don't render maps with it, so I don't know how it scales. But Mapnik
will connect easier to Postgres/PostGIS. I will outsource rendering for
my project.
Note: I don`t use Mapnik. I
Marcus Wolschon schrieb:
On Thu, Jul 23, 2009 at 8:35 PM, Andreas Kalschandreaskal...@gmx.de wrote:
How much would that be without filtering? Since I have no clue what
are relevant tags and relations for you.
Relevant features are features which represent a GeoObject. Nodes which
When I call the binary, JDBC will not be found. When I call it like this ...
java --classpath
3rdparty/osmosis/osmosis.jar:3rdparty/osmosis/lib/compile/postgresql-8.3-603.jdbc4.jar:/usr/lib/jvm/java-1.5.0-gcj-4.3-1.5.0.0/jre/lib/rt.jar
org.openstreetmap.osmosis.core.Osmosis --rx
://wiki.openstreetmap.org/index.php/Osmosis
Brett
Andreas Kalsch wrote:
When I call the binary, JDBC will not be found. When I call it like
this ...
java --classpath
3rdparty/osmosis/osmosis.jar:3rdparty/osmosis/lib/compile/postgresql-8.3-603.jdbc4.jar:/usr/lib/jvm/java-1.5.0-gcj-4.3
Thank you, it's working. I didn't intend to install the GNU stuff ...
Andreas Kalsch schrieb:
It's the Java 1.5 version from
http://cloudmade-osmosis.s3.amazonaws.com/api0.6-java1.5 ... Debian etch
delivers packages for Java 1.5 by default.
Brett Henderson schrieb:
Hi Andreas,
How
When I retrieve Wiki pages via wget I get a confusing bunch of characters:
?????[?$??=??~r y?te?jwUwg??G?G?wF??v?lA?|
??(AJ(? 5???=|??'???{?_?3#3w????n77??l?|???}?ëO???5????O/?{em?l??W???o)t?i(??w-kk dz3??
'??G?
Yes, of course, but when the config will change next week, how will I
know this? I have to do an extra check.
I would like to download OSM wiki websites like I download every other
website.
Roland Olbricht schrieb:
Anyone who can solve this puzzle, so that we can download with simple
Explicitly omit accepted encodings:
1) wget --header='Accept-Encoding: '
'http://wiki.openstreetmap.org/wiki/Main_Page'
2) ?php
$streamContext = stream_context_create(array(
'http' = array(
'header' = 'Accept-Encoding: \r\n'
)
));
I have found this in the OpenStreetMap news, and I wondered why I have
given the first vote for it.
http://twitter.com/kalsch/status/4582749178
Please spread this!
Andi
___
dev mailing list
dev@openstreetmap.org
1,452 positive votes as I
write this
2009/10/3 Andreas Kalsch andreaskal...@gmx.de:
I have found this in the OpenStreetMap news, and I wondered why I have
given the first vote for it.
http://twitter.com/kalsch/status/4582749178
Please spread this!
Andi
When I change the position of a node, will the way be in the changeset, too?
When I change members of relations (without adding / removing members),
will the relation be in the changeset, too?
I have currently no data to test these cases, so does anybody know how
the results of these operations
Error occurs in Osmosis 0.33:
/backup/projects/3rdparty/osmosis/bin/osmosis --rx
file=/backup/projects/data/gos/andi/bremen.osm --wd database=b
validateSchemaVersion=no ... populateCurrentTables=no
java.io.FileNotFoundException:
/backup/projects/3rdparty/osmosis/config/plexus.conf (No such
Is there a way to make Osmosis populate and refresh just the current
tables and ignoring the history tables? It would be faster and it is
easier to work with tables which have exactly one version - the recent -
for every feature.
Andi
___
dev
Using populateCurrentTables=yes on refresh will make Osmosis stop. When
I use populateCurrentTables=no, then all is OK, but then I don't have
updated data in the current tables. What can I do about that?
Feb 14, 2010 4:15:28 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version
FOR EACH ROW EXECUTE PROCEDURE ways_insert();
CREATE TRIGGER relation_insert BEFORE INSERT ON relations
FOR EACH ROW EXECUTE PROCEDURE relations_insert();
Am 14.02.10 15:40, schrieb Andreas Kalsch:
Is there a way to make Osmosis populate and refresh just the current
tables and ignoring
/14
Subject: Re: [OSM-dev] Osmosis error
2010/2/14 Andreas Kalsch andreaskal...@gmx.de
mailto:andreaskal...@gmx.de:
Error occurs in Osmosis 0.33:
/backup/projects/3rdparty/osmosis/bin/osmosis --rx
file=/backup/projects/data/gos/andi
It seems the US are doing land reclamation ...
Am 24.02.10 00:50, schrieb Apollinaris Schoell:
any ideas what is going on here?
http://www.openstreetmap.org/?lat=35.2lon=-78.9zoom=5layers=B000FTFT
http://www.openstreetmap.org/?lat=10.28lon=-78.83zoom=6layers=B000FTF
I miss a license notice in every map tile ;)
Am 26.02.10 15:48, schrieb Peter Körner:
Frederik Ramm schrieb:
Brian,
Just at the moment I'm at a loss to know why I spend my free (unpaid)
time developing a tool when a small mistake gets me a load of abuse.
Sorry I didn't
I am still reading some old mailing list posts ...
What about a relation with type=data, which is a relation that can
include tags and other relations recusively?
This relation has no geometric reference but it is just there to save
data. So we could reuse relations for a purpose which is
be
machine readable.
Using prototypes for common features would complicate things a lot,
because mappers would need to use an additional definition next to the
feature list in the wiki.
Am 05.05.10 01:39, schrieb Scott Crosby:
On Tue, May 4, 2010 at 3:12 PM, Andreas Kalsch andreaskal
Am 06.05.10 16:36, schrieb Roland Olbricht:
Am Dienstag, 4. Mai 2010 22:12:02 schrieb Andreas Kalsch:
I am still reading some old mailing list posts ...
What about a relation with type=data, which is a relation that can
include tags and other relations recusively?
It is a really
Hi,
were there any successful attempts to read OSM data into CouchDB and
Geocouch? Does somebody know of a backend?
Andi
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
The web frontend is mainly for searching - it needs fast reads,
horizontal scaling across several servers.
In the backend it needs functions that check for geometric relations or
that compute new geometries. It must be able to handle huge, complex
geometries.
Fast geo indexes for both frontend
What about rewriting this stuff in C? I have written a MySQL importer in
C some time ago, so what about reusing its XML parsing part?
If you are interested, I'll put it on Github.
Andi
Am 03.07.10 22:43, schrieb Ian Dees:
On Sat, Jul 3, 2010 at 1:17 PM, Nolan Darilek no...@thewordnerd.info
To make it even faster: Can you scale Postgres indexes across serveral
servers?
Andi
Am 28.07.10 09:48, schrieb Juan Lucas Domínguez Rubio:
After exporting the OSM planet to Postgres, I get -among others- a 43 GB table
with lines, and this random spatial query only takes one second to
What about some metrics (performance, size)? Data is the same, whether
binary or not. So binary really has to pay off significantly.
Am 01.08.10 13:39, schrieb Brett Henderson:
On Sun, Aug 1, 2010 at 7:34 PM, Erik Johansson erjo...@gmail.com
mailto:erjo...@gmail.com wrote:
On Sun, Aug 1,
http://lists.openstreetmap.org/pipermail/dev/2009-July/016215.html
Am 14.08.10 11:52, schrieb M∡rtin Koppenhoefer:
2010/8/2 Ben Supnikbsup...@xsquawkbox.net:
- If the outer multipolygon only has one way, then __maybe_- that way might
contain all of the attributes I care about? This second
Am 20.08.10 05:45, schrieb Michael Daines:
- I can only add predefined tags - I think there should be a more common way to
add tags. The best way is to simplify adding relevant tags by using data mining
(Which tags are often used together with the tags already defined?) to propose
new tags
Please don't make end users change too much on the next update!
I think the current data model is pretty OK, it is more a data data model than a sematical one. And
I think we should keep it like that ;)
Andi
Am 12.10.10 21:45, schrieb Chris Browet:
I am wondering (I wonder a lot lately ;-))
+1
This is the point
Am 12.10.10 22:33, schrieb Alan Millar:
Frankly, one of the main problems with the classic GIS
shapefile-style data paradigm is that it does not give you good
topological connectivity information, and therefore is inadequate for
OSM's multi-use data model.
If you think
I agree with
whitespace - this can be very confusing
=
To add:
Make keys lowercase (or even remove diacritics), because keys are always simple
names.
Am 16.10.10 20:44, schrieb Jochen Topf:
Hi!
I am currently fighting some issues where tags with strange characters in them
need to be
Is there a way to use simple schema in Osmosis without hstore? And why was this changed? A separate
table for tags can more easily be indexed. I think it is not a good idea to use hstore because then
we can drop SQL, use NoSQL for storing data and use PostGIS/Postgres for Geometry only.
What
a cell ...
Am 18.11.10 14:00, schrieb Frank Broniewski:
Am 18.11.2010 10:18, schrieb Andreas Kalsch:
Is there a way to use simple schema in Osmosis without hstore? And why
was this changed? A separate table for tags can more easily be indexed.
I think it is not a good idea to use hstore because
, but this is now down to well under 10 minutes.
On the long run, this is an argument ;) I am critical, because I still haven't thought through all
dependant scripts that do something with tags. But there are many ...
Hope that helps,
Brett
On Thu, Nov 18, 2010 at 8:18 PM, Andreas Kalsch
One simple answer: The drivers do not work appropriately with complex SQL data types. In PHP or
node.js I will get a string that I have to parse, in MongoDB, I get a proper object or list. If I
used hstore in a consequent way (I like consequence and unification), I would have sets in sets, and
Am 19.11.10 10:06, schrieb Frederik Ramm:
Hi,
Andreas Kalsch wrote:
One simple answer: The drivers do not work appropriately with complex SQL data types. In PHP or
node.js I will get a string that I have to parse, in MongoDB, I get a proper object or list. If I
used hstore in a consequent way
I agree - my approach is the playground approach: At first, keep it simple. For your purpose it is
correct to split the table. What about a partial index over 1 table? This is possible with Postgres.
Am 19.11.10 11:03, schrieb Sven Geggus:
Andreas Kalschandreaskal...@gmx.de wrote:
Example
Am 20.11.10 01:38, schrieb Brett Henderson:
On Fri, Nov 19, 2010 at 7:37 PM, Andreas Kalsch andreaskal...@gmx.de
mailto:andreaskal...@gmx.de wrote:
Hi Brett,
thanks for your elaborate answer! Now I am up to date. Some ideas regarding
my use case ...
Am 18.11.10 23:50, schrieb
Hi,
thank you, Brett, for me this is the perfect setup. I hope that others will find it useful as well.
The names are OK. Two problems:
1) I read XML or PBF, dump it to CSV and then read the dump, but now my feature tables are blank
because of these errors:
ERROR: extra data after last
...@denofr.de wrote:
On Fri, Nov 19, 2010 at 09:37:33AM +0100, Andreas Kalsch wrote:
If you're applying diffs to the database you can enhance the
osmosisUpdate() function (initially empty, but can be customised) to
keep your separate tags tables up to date during each diff
application. You will need to run
+1 from here. I wonder that the bing imagery is not scaled on higher zoom levels in both JSOM and
Potlatch2. The imagery will not be better by scaling, but the point is that mappers can make more
detailed edits then.
Andi
Am 06.12.10 13:16, schrieb Chris Browet:
Hi,
It looks like Bing is
1) What does the data_type U in the actions table mean?
2) It seems that Osmosis violates pk_aktions (primary key), so it would be better to replace it by a
simple index.
3) This command does not seem to retrieve all changesets. After call, the latest timestamp was
20.11.2010.
This problem occurs with the current germany.osm AND pbf ( 11-Dec-2010 ) in the latest developer
releases (24xxx):
/home/andi/.libraries/osmosis-SNAPSHOT-r24679/bin/osmosis --read-pbf
file=/backup/downloads/zltl/germany.osm.pbf --ws database=de validateSchemaVersion=yes user=...
password=...
I started the script with yesterday's Great Britain PBF and today's Germany PBF, and there was no
error. It seems there was an error in yesterday's Germany data. It would be worse if that happended
with changesets.
Am 12.12.10 12:07, schrieb Frederik Ramm:
Hi,
Andreas Kalsch wrote
Can you help me applying a changeset with the latest snapshots? This is my
command:
/home/andi/.libraries/osmosis-SNAPSHOT-r24679/bin/osmosis --read-replication-interval
/backup/downloads/zltl/replicate/rlp/ --write-pgsimp-change database=rlp validateSchemaVersion=yes
user=andi password=...
I read all data using dumping and copy (fastest way).
1) For the PBF version this failed, because of 2 errors (!) in the nodes dump. 2 timestamps were
malformed like this: 2005-08-01 21:11:38)0200 instead of 2005-08-01 21:11:38+0200
So all tables were populated but the nodes table. From the
server caused some errors.
What do I need to do to run memcheck on debian?
Am 13.12.10 15:23, schrieb Frederik Ramm:
Hi,
Andreas Kalsch wrote:
1) For the PBF version this failed, because of 2 errors (!) in the nodes dump. 2 timestamps were
malformed like this: 2005-08-01 21:11:38)0200 instead
It would be very helpful for me to know, if anybody has the same issue:
changesets are not applied:
/home/andi/.libraries/osmosis-SNAPSHOT-r24679/bin/osmosis --read-replication-interval
/backup/downloads/osm/replicate/rlp/ --simplify-change --write-pgsimp-change database=rlp
How I would do post-processing OSM tags:
name: 'München'
name:en: 'Munich'
noexit: 'yes'
addr:street: 'Weg'
date:'2011-01-02'
becomes:
name: {
_: 'München'
en: 'Munich'
}
noexit: true
addr: {
street: 'Weg'
}
date: new Date(...)
This could be the result of a post-processing script so that your
I am currently reading this Google paper, which is probably interesting for indexing OSM data:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.1480rep=rep1type=pdf
Best,
Andi
___
dev mailing list
dev@openstreetmap.org
Probably this is useful for you, too, so let me announce osmo, a performant way to populate MongoDB
with OSM data: https://github.com/akidee/osmo
Please give me some feedback, or you even want to contribute.
Andi
___
dev mailing list
Currently I have no app that is ready to show an example. Once the data is in Mongo, you can do a
lot with it with simple means since it's no SQL and you have all data in 1 collection. Example of my
consecutive tool chain that I use to make some experiments:
I use node.js and Christian
My motivation to post them here was that some cloudmade developrs read them and put them on a high
position on their to do list, because what I propose is not really a technical challenge. Some
simple changes to get a great result ...
Am 02.11.11 16:47, schrieb Richard Fairhurst:
Andreas
88 matches
Mail list logo
