Hello, I would prefer Zoi's option B, as doing it as a plugin allows users to pull when needed, but it won't be part of any core modules of wayang.
The dependencies could then also just reside in that plugin "wayang-spatial" and would be pulled through maven alongside. I don't have a strong opinion about adding the other platforms - the more the merrier. Best, Juri On 2026/01/15 11:37:59 Zoi Kaoudi via dev wrote: > Hello, > thank you for the detailed email. > I am more inclined to having it as a plugin. > One question I have though is whether one can enable/disable platforms within > the plugin. For instance, can I declare to use the spatial plugin but without > using the Spark platform. Is that possible? > Without plugins I would just not pass the ".withPlugin(Spark.basicPlugin())" > in my job, and then Wayang would execute the job in the rest of the platforms > I defined. If this is also possible with defining a new plugin then I would > vote that that's the best approach. > What do you think? > > Best > -- > Zoi > Στις Τετάρτη 14 Ιανουαρίου 2026 στις 02:37:57 μ.μ. CET, ο χρήστης Speer, > Maximilian <[email protected]> έγραψε: > > Hello everyone, > > as part of our master’s project in cooperation with Zoi Kaoudi, we are > extending Apache Wayang to support spatial data processing. > > So far, we implemented: > > * SpatialFilterOperator and SpatialJoinOperator > * Support for Java, Spark, and Postgres/JDBC > * A Geometry abstraction to translate between platform-specific geometry > representations (WKT, WBK, geoJson, JTS Geometry) > > We are preparing a PR to the main repository and would like guidance on where > these new classes and their dependencies should be placed. > > The core issue: dependencies > > Our spatial operators require additional dependencies: > > * JTS (small, lightweight) > * Apache Sedona (larger; needed for Spark-side spatial processing) > > Since many Wayang users likely don’t need spatial functionality, we’re > hesitant to add these dependencies directly into the existing platform > modules’ pom.xml files. > > Two structure options we see > > Option A: Add spatial operators like regular operators > > Place everything alongside existing operators and add the required > dependencies to the respective platform modules (Java/Spark/JDBC). This > approach is implemented in the repo linked below. > > Option B: Implement spatial support as a plugin > > Keep spatial functionality isolated as a plugin, including its dependencies, > e.g.: > > wayang-plugins/ > wayang-spatial/ > src/main/java/.../spatial/ > data/ > mapping/ > operators/ > pom.xml > > > This approach seems clean, but we are not sure whether adding plugins is a > viable approach. Currently the only existing Plugin appears to be the IEJoin. > > What we’d like your input on > > 1. Where should the spatial dependencies go? > > Should we accept adding JTS/Sedona to the platform modules, or keep them > contained via a plugin? > > 2. Where should shared spatial types live? > > Specifically: > > * the Geometry class (used across platforms) > * the spatial predicate enum (filter/join predicates), used in API to > specify spatial operator type (eg. intersect, overlaps) > > The PR in our own repository (operators added as normal operators) is here: > > https://github.com/Spatial-Data-MP/incubator-wayang/pull/1 > > Please feel free to also comment on our general approach. Thanks a lot for > your feedback. > > Best, > > Anton, Felix, Max >
