Hello guys,
I faced one problem that I cannot pass my data inside rdd partition when I was
trying to develop spark streaming feature.I'm the newcomer of Spark, could you
please give me any suggestion on this problem?
The figure in the attachment is the code I used in my program:
After I run my code, I found these logs:Inside foreachRDD scope I can get the
right indexGeneratorMap value which includes right kva instance,
"CassandraAccessDatastax@5adc7292"
15/06/16 12:07:35 INFO streaming.AbstractSparkStreamingMain: [RDD][Index
Generator Map]: {class
com.worksap.company.framework.autoindex.edp.job.SparkStreamingTestEntity=AbstractIndexGenerator{kva=com.worksap.company.access.cassandra.CassandraAccessDatastax@5adc7292,
indexName='spark_streaming_test_index'}}
But in the foreachPartition scope, I found my kva instance is null. I totally
have no idea about it.
15/06/16 12:07:28 INFO streaming.AbstractSparkStreamingMain: [Index Generator
Map]: {class
com.worksap.company.framework.autoindex.edp.job.SparkStreamingTestEntity=AbstractIndexGenerator{kva=null,
indexName='spark_streaming_test_index'}}
If someone can give me any tips, I will really appreciate your help.
Thanks in advance.
Best Regards,
--
Gabriel Zhang
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
