Integrating Kafka 0.10 or higher with Spark 2.1.1 -- required jars

2017-07-07 Thread mahendra
Hi, After finding that support for streaming api in python is not present for kafka brokers version > 0.10 . I was trying to run a kafka example in scala. I used following spark submit command : /usr/local/spark/bin/spark-submit --jars /home/mahendra/spark-streaming-kafka-0-10_2.11-2.1.1.

Integrating Kafka 0.10 or higher with Spark 2.1.1 -- required jars

2017-07-07 Thread mahendra singh meena
Hi, After finding that support for streaming api in python is not present for kafka brokers version > 0.10 . I was trying to run a kafka example in scala. I used following spark submit command : /usr/local/spark/bin/spark-submit --jars /home/mahendra/spark- > streaming-kafka-0-10_2.11

Integrating Kafka 0.10 or higher with Spark 2.1.1 -- required jars

2017-07-07 Thread mahendra singh meena
Hi, After finding that support for streaming api in python is not present for kafka brokers version > 0.10 . I was trying to run a kafka example in scala. I used following spark submit command : /usr/local/spark/bin/spark-submit --jars > /home/mahendra/spark-streaming-kafka-0-10_2.11

Writing/Saving RDD to HDFS using saveAsTextFile

2016-10-07 Thread Mahendra Kutare
the help in advance, Mahendra about.me/mahendrakutare <https://about.me/mahendrakutare?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> ~~ Only those who will risk going too far can pos

Re: How to Disable or do minimal Logging for apache spark client Driver program?

2016-10-06 Thread Mahendra Kutare
import ch.qos.logback.classic.Level; sc.setLogLevel(Level.INFO.levelStr); //Change the level to an appropriate level for your application. Mahendra about.me/mahendrakutare <https://about.me/mahendrakutare?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign

Re: Restful WS for Spark

2016-09-30 Thread Mahendra Kutare
Try Cloudera Livy https://github.com/cloudera/livy It may be helpful for your requirement. Cheers, Mahendra about.me/mahendrakutare <https://about.me/mahendrakutare?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign