Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r191502285
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
<version>1.10.19</version>
<scope>test</scope>
</dependency>
+ <dependency>
--- End diff --
> I think I'm a bit concerned adding rxjava to the top level pom and to
dev/deps/spark-deps-hadoop-*
can it be just a `<arrow.version>0.8.0</arrow.version>` thing and not a
dependency?
Unsure what you mean here - we're using rxjava itself specifically to do
the event handling in this PR. See
https://github.com/apache/spark/pull/21366/files#diff-ae4cd884779fb4c3db58958ab984db59R40.
If we wanted an alternative approach we can build something from first
principles (executor service / manual linked blocking queues) but I like the
elegance that rx-java buys us here. The code we'd save building ourselves seems
worthwhile.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
