Re: Apache Spark example split/merge shards

2017-01-16 Thread Takeshi Yamamuro
Hi,

It seems you hit this issue:
https://issues.apache.org/jira/browse/SPARK-18020

// maropu

On Tue, Jan 17, 2017 at 11:51 AM, noppanit <noppani...@gmail.com> wrote:

> I'm totally new to Spark and I'm trying to learn from the example. I'm
> following this example
>
> https://github.com/apache/spark/blob/master/external/
> kinesis-asl/src/main/scala/org/apache/spark/examples/
> streaming/KinesisWordCountASL.scala.
>
> It works well. But I do have one question. Every time I split/merge Kinesis
> shards from the Amazon API, I have to restart the application. Is there a
> way for Spark application to automatically rebalance or to notify the
> application that Kinesis shards have been split or merged? I have to
> restart
> the application so I can see more shards in DynamoDB.
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Apache-Spark-example-split-merge-shards-tp28311.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>


-- 
---
Takeshi Yamamuro


Apache Spark example split/merge shards

2017-01-16 Thread noppanit
I'm totally new to Spark and I'm trying to learn from the example. I'm
following this example 

https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/examples/streaming/KinesisWordCountASL.scala.

It works well. But I do have one question. Every time I split/merge Kinesis
shards from the Amazon API, I have to restart the application. Is there a
way for Spark application to automatically rebalance or to notify the
application that Kinesis shards have been split or merged? I have to restart
the application so I can see more shards in DynamoDB. 



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Apache-Spark-example-split-merge-shards-tp28311.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org