Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r194560151
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
<version>1.10.19</version>
<scope>test</scope>
</dependency>
+ <dependency>
--- End diff --
@felixcheung (sorry for the delayed answer), @mccheah a few
thoughts/observations:
a) If not mistaken Java 9 has introduced the related interfaces, so rx-java
might not be needed in the future.
b) Some more scala centric implementations: [akka
streams](https://doc.akka.io/docs/akka/current/stream/stream-integrations.html#integrating-with-reactive-streams)
instead, a Reactive Streams and JDK 9+ java.util.concurrent.Flow-compliant
implementation. This does not depend on rx-java.
Also there is [monix] (https://github.com/monix/monix), and
(RxScala)[https://github.com/ReactiveX/RxScala] seems outdated. A comparison
between monix and others
(here)[https://monix.io/docs/2x/reactive/observable-comparisons.html].
One question is how about performance at scale when you get events from
hundreds of executors at once which framework should work best? Should we worry
about this?
Rx-java is light but also monix might be light at the end of the day or
akka streams, although its purpose goes beyond observables etc.
c) I built the distro from the PR, so rxjava-2.1.13.jar,
reactive-streams-1.0.2.jar are added to the jars. Spark core does not seem to
have any classes in it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
