sid created SPARK-8133:
--------------------------
Summary: sticky partitions
Key: SPARK-8133
URL: https://issues.apache.org/jira/browse/SPARK-8133
Project: Spark
Issue Type: New Feature
Components: Streaming
Affects Versions: 1.3.1
Reporter: sid
We are trying to replace Apache Storm with Apache Spark streaming.
In storm; we partitioned stream based on "Customer ID" so that msgs with a
range of "customer IDs" will be routed to same bolt (worker).
We do this because each worker will cache customer details (from DB).
So we split into 4 partitions and each bolt (worker) will have 1/4 of the
entire range.
I am hoping we have a solution to this in Spark Streaming
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
