Hi Mike,
good catch, I forgot to add the ...Factory.java file. Hopefully I
haven't forgotten any more files.
Cheers
On 22 May 2010 04:08, Mike N. <[email protected]> wrote:
> Hi andrzej,
>
> I tried to apply the patch, but I get an undefined symbol when trying to
> build 'FlattenFilterFactory'. Am I missing part of the patch or is there
> another setting to create the Factory?
>
> Thanks,
>
> Mike Nice
>
> --------------------------------------------------
> From: "andrzej zaborowski" <[email protected]>
> Sent: Friday, May 21, 2010 2:40 PM
> To: "David Carmean" <[email protected]>
> Cc: <[email protected]>; "OSM US Talk"
> <[email protected]>
> Subject: Re: [Talk-us] Cloudmade california.osm: dups or osmosis bug?
>
>> Hi,
>>
>> On 21 May 2010 01:54, David Carmean <[email protected]> wrote:
>>>
>>> Is it the dataset or osmosis that is giving me a single 2-node duplicate
>>> for each postGIS table created by osmosis?
>>
>> Not sure if it's related, but I noticed that many of the CloudMade
>> extracts can't be processed using osmosis (even though they're
>> generated with osmosis), except for the operations that don't try to
>> validate the order of elements. It seems they often contain
>> consecutive elements with the same id, but possibly different version.
>> I have a little patch (attached) to add a new filter (--ff) that
>> "flattens" the file. It does the same operation on sorted entity
>> streams as --simplify-change does on change streams, unfortunately
>> "--simplify" is taken by a different operation now. I decided it was
>> faster to modify osmosis than to download the planet and make my own
>> un-broken extracts.
>>
>> Cheers
>> (I'm not subscribed to osmosis-dev)
>>
>
>
>
>> _______________________________________________
>> Talk-us mailing list
>> [email protected]
>> http://lists.openstreetmap.org/listinfo/talk-us
>>
>
Index: src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java
===================================================================
--- src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java (revision 0)
+++ src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilter.java (revision 0)
@@ -0,0 +1,91 @@
+/* This software is released into the Public Domain.
+ * See copying.txt for details. */
+package org.openstreetmap.osmosis.core.filter.v0_6;
+
+import org.openstreetmap.osmosis.core.container.v0_6.BoundContainer;
+import org.openstreetmap.osmosis.core.container.v0_6.EntityContainer;
+import org.openstreetmap.osmosis.core.container.v0_6.EntityProcessor;
+import org.openstreetmap.osmosis.core.domain.v0_6.Entity;
+import org.openstreetmap.osmosis.core.task.v0_6.Sink;
+import org.openstreetmap.osmosis.core.task.v0_6.SinkSource;
+
+
+/**
+ * Flatten / simplify a sorted entity stream.
+ * (similar to --simplify-change)
+ */
+public class FlattenFilter implements SinkSource {
+ private Sink sink;
+ private EntityContainer previous_container;
+
+ /**
+ * Creates a new instance.
+ */
+ public FlattenFilter() {
+ }
+
+ /**
+ * Process a node, way or relation.
+ *
+ * @param current_container
+ * The entity container to be processed.
+ */
+ public void process(EntityContainer current_container) {
+ if (previous_container == null) {
+ previous_container = current_container;
+ return;
+ }
+
+ Entity current = current_container.getEntity();
+ Entity previous = previous_container.getEntity();
+
+ if (current.getId() != previous.getId() ||
+ current.getClass() != previous.getClass()) {
+ sink.process(previous_container);
+ previous_container = current_container;
+ return;
+ }
+
+ if (current.getVersion() > previous.getVersion())
+ previous_container = current_container;
+ }
+
+ /**
+ * Process the bound.
+ *
+ * @param boundContainer
+ * The bound to be processed.
+ */
+ public void process(BoundContainer boundContainer) {
+ /* By default, pass it on unchanged */
+ sink.process(boundContainer);
+ }
+
+ /**
+ * {...@inheritdoc}
+ */
+ public void complete() {
+ /*
+ * If we've stored entities temporarily, we now need to
+ * forward the stored ones to the output.
+ */
+ if (previous_container != null)
+ sink.process(previous_container);
+
+ sink.complete();
+ }
+
+ /**
+ * {...@inheritdoc}
+ */
+ public void release() {
+ sink.release();
+ }
+
+ /**
+ * {...@inheritdoc}
+ */
+ public void setSink(Sink sink) {
+ this.sink = sink;
+ }
+}
Index: src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java
===================================================================
--- src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java (revision 0)
+++ src/org/openstreetmap/osmosis/core/filter/v0_6/FlattenFilterFactory.java (revision 0)
@@ -0,0 +1,26 @@
+/* This software is released into the Public Domain.
+ * See copying.txt for details. */
+package org.openstreetmap.osmosis.core.filter.v0_6;
+
+import org.openstreetmap.osmosis.core.pipeline.common.TaskConfiguration;
+import org.openstreetmap.osmosis.core.pipeline.common.TaskManager;
+import org.openstreetmap.osmosis.core.pipeline.common.TaskManagerFactory;
+import org.openstreetmap.osmosis.core.pipeline.v0_6.SinkSourceManager;
+
+/**
+ * The task manager factory for a flatten/simplify filter.
+ */
+public class FlattenFilterFactory extends TaskManagerFactory {
+ /**
+ * {...@inheritdoc}
+ */
+ @Override
+ protected TaskManager createTaskManagerImpl(
+ TaskConfiguration taskConfig) {
+ return new SinkSourceManager(
+ taskConfig.getId(),
+ new FlattenFilter(),
+ taskConfig.getPipeArgs()
+ );
+ }
+}
Index: src/org/openstreetmap/osmosis/core/TaskRegistrar.java
===================================================================
--- src/org/openstreetmap/osmosis/core/TaskRegistrar.java (revision 21311)
+++ src/org/openstreetmap/osmosis/core/TaskRegistrar.java (working copy)
@@ -45,6 +45,7 @@
import org.openstreetmap.osmosis.core.filter.v0_6.UsedNodeFilterFactory;
import org.openstreetmap.osmosis.core.filter.v0_6.WayKeyFilterFactory;
import org.openstreetmap.osmosis.core.filter.v0_6.WayKeyValueFilterFactory;
+import org.openstreetmap.osmosis.core.filter.v0_6.FlattenFilterFactory;
import org.openstreetmap.osmosis.core.merge.v0_6.ChangeAppenderFactory;
import org.openstreetmap.osmosis.core.merge.v0_6.ChangeMergerFactory;
import org.openstreetmap.osmosis.core.merge.v0_6.ChangeSimplifierFactory;
@@ -153,8 +154,10 @@
factoryRegister.register("read-xml", new XmlReaderFactory());
factoryRegister.register("fast-read-xml", new FastXmlReaderFactory());
factoryRegister.register("rx", new XmlReaderFactory());
- factoryRegister.register("read-xml-change", new XmlChangeReaderFactory());
- factoryRegister.register("upload-xml-change", new XmlChangeUploaderFactory());
+ factoryRegister.register("flatten", new FlattenFilterFactory());
+ factoryRegister.register("ff", new FlattenFilterFactory());
+ factoryRegister.register("read-xml-change", new XmlChangeReaderFactory());
+ factoryRegister.register("upload-xml-change", new XmlChangeUploaderFactory());
factoryRegister.register("rxc", new XmlChangeReaderFactory());
factoryRegister.register("sort", entitySorterFactory06);
factoryRegister.register("s", entitySorterFactory06);
_______________________________________________
osmosis-dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/osmosis-dev
