Hi, Good points. +1
—Alex > On Jan 19, 2026, at 14:48, Juri Petersen <[email protected]> wrote: > > Hello, > I would prefer Zoi's option B, as doing it as a plugin allows users to pull > when needed, but it won't be part of any core modules of wayang. > > The dependencies could then also just reside in that plugin "wayang-spatial" > and would be pulled through maven alongside. > > I don't have a strong opinion about adding the other platforms - the more the > merrier. > > Best, > Juri > > On 2026/01/15 11:37:59 Zoi Kaoudi via dev wrote: >> Hello, >> thank you for the detailed email. >> I am more inclined to having it as a plugin. >> One question I have though is whether one can enable/disable platforms >> within the plugin. For instance, can I declare to use the spatial plugin but >> without using the Spark platform. Is that possible? >> Without plugins I would just not pass the ".withPlugin(Spark.basicPlugin())" >> in my job, and then Wayang would execute the job in the rest of the >> platforms I defined. If this is also possible with defining a new plugin >> then I would vote that that's the best approach. >> What do you think? >> >> Best >> -- >> Zoi >> Στις Τετάρτη 14 Ιανουαρίου 2026 στις 02:37:57 μ.μ. CET, ο χρήστης Speer, >> Maximilian <[email protected]> έγραψε: >> >> Hello everyone, >> >> as part of our master’s project in cooperation with Zoi Kaoudi, we are >> extending Apache Wayang to support spatial data processing. >> >> So far, we implemented: >> >> * SpatialFilterOperator and SpatialJoinOperator >> * Support for Java, Spark, and Postgres/JDBC >> * A Geometry abstraction to translate between platform-specific geometry >> representations (WKT, WBK, geoJson, JTS Geometry) >> >> We are preparing a PR to the main repository and would like guidance on >> where these new classes and their dependencies should be placed. >> >> The core issue: dependencies >> >> Our spatial operators require additional dependencies: >> >> * JTS (small, lightweight) >> * Apache Sedona (larger; needed for Spark-side spatial processing) >> >> Since many Wayang users likely don’t need spatial functionality, we’re >> hesitant to add these dependencies directly into the existing platform >> modules’ pom.xml files. >> >> Two structure options we see >> >> Option A: Add spatial operators like regular operators >> >> Place everything alongside existing operators and add the required >> dependencies to the respective platform modules (Java/Spark/JDBC). This >> approach is implemented in the repo linked below. >> >> Option B: Implement spatial support as a plugin >> >> Keep spatial functionality isolated as a plugin, including its dependencies, >> e.g.: >> >> wayang-plugins/ >> wayang-spatial/ >> src/main/java/.../spatial/ >> data/ >> mapping/ >> operators/ >> pom.xml >> >> >> This approach seems clean, but we are not sure whether adding plugins is a >> viable approach. Currently the only existing Plugin appears to be the IEJoin. >> >> What we’d like your input on >> >> 1. Where should the spatial dependencies go? >> >> Should we accept adding JTS/Sedona to the platform modules, or keep them >> contained via a plugin? >> >> 2. Where should shared spatial types live? >> >> Specifically: >> >> * the Geometry class (used across platforms) >> * the spatial predicate enum (filter/join predicates), used in API to >> specify spatial operator type (eg. intersect, overlaps) >> >> The PR in our own repository (operators added as normal operators) is here: >> >> https://www.google.com/url?q=https://github.com/Spatial-Data-MP/incubator-wayang/pull/1&source=gmail-imap&ust=1769435296000000&usg=AOvVaw1N5_4SgpVXa_TV-6x3OUO3 >> >> Please feel free to also comment on our general approach. Thanks a lot for >> your feedback. >> >> Best, >> >> Anton, Felix, Max >> -- *Scalytics Connect* The foundation for secure, scalable, and transparent AI. www.scalytics.io <http://www.scalytics.io> -- Please consider the environment before printing this email -- Disclaimer: The content of this message is confidential. If you have received it by mistake, please inform us by an email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message.
