Hi tdas, is it possible to run spark 24/7, i am using updateStateByKey and i
am streaming 3lac records in 1/2 hr, i am not getting the correct result
also i am not not able to run spark streaming for 24/7 after hew hrs i get
array out of bound exception even if i am not streaming anything? btw will
the streaming end if i am not streaming anything for a few minutes? Please
help me out here
Also is it possible to delete state? since its growing exponentially, also
not all the data are updated.. at some point we have to reset it na? how to
do that.. i am able to work with batch processing using spark successfully
but streaming is quite a mystery for me
i am submitting spark application in the following fashion
bin/spark-submit --class "NetworkCount" --master spark://abc.test.com:7077
try/simple-project/target/simple-project-1.0-jar-with-dependencies.jar
But is there any other way to submit spark application through the code?
like for example i am checking for a condition if true i wanted to run the
spark application
(isConditionTrue){
runSpark("NetworkCount","masterurl","jar")
}
I am aware we can set the jar and master url with spark context, but how to
run it from code automatically when a condition comes true without actually
using spark-submit
Is it possible?
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Applications-tp16976p17453.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
