On Thu, Nov 12, 2009 at 12:40 PM, Brett Henderson <[email protected]> wrote:
> > > On Thu, Nov 12, 2009 at 9:36 AM, Frederik Ramm <[email protected]>wrote: > >> Hi, >> >> we just had a problem with diff updates because someone created >> relations with more than 65k members and Osmosis tries to store them as >> a char which raised an exception. >> >> I did a quick hack that would simply drop the excessive members with a >> warning: >> >> http://www.remote.org/frederik/tmp/remove_excessive_members.diff >> >> I'm not checking that in. >> >> I thought about simply patching Osmosis to use an integer but was >> informed by Jon Burgess that osm2pgsql (which is next in my toolchain) >> would carp on > 65k members as well so I didn't bother - especially as >> it turned out that the relations in question were an error in somebody's >> automated import and have been meanwhile fixed by Lennard (they >> contained the same element over and over). >> > > If > 65k members is legitimate then feel free to change Osmosis. I only > picked smaller data types to save some memory when doing things like large > sorts. If the API allows it then perhaps Osmosis should too. > > Another one that will have to be changed eventually is entity ids. They'll > only allow up to 2^31 currently (signed integer) which will probably break > eventually given the rapid growth of the database. > > I've updated the Relation class to use a 32-bit int instead of 16-bit char when persisting temp data to disk (which I assume is where it broke). This occurs in tasks like --sort which performs a disk based merge sort. The only reason for using a smaller data type was to reduce the amount of data being persisted which is largely unimportant for Relations because there aren't a lot of them anyway. It's only checked into svn for now and will show up in nightly builds. I'm overdue for a proper release so will have to do that at some stage. Brett
_______________________________________________ osmosis-dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/osmosis-dev
