It seems you are usibg kafka 0.10. See my comments below.
---Original--- From: "shyla deshpande"<[email protected]> Date: 2017/7/10 08:17:10 To: "user"<[email protected]>; Subject: Spark streaming giving me a bunch of WARNINGS, please help meunderstand them WARN Use an existing SparkContext, some configuration may not take effect. I wanted to restart the spark streaming app, so stopped the running and issued a new spark submit. Why and how it will use a existing SparkContext? => you are using checkpoint to restore the sparkcontext. WARN Spark is not running in local mode, therefore the checkpoint directory must not be on the local filesystem. Directory 'file:/efs/checkpoint' appears to be on the local filesystem. =>the CP path should be HDFSand so on. If you want to use local path, the cluster model should be local. WARN overriding enable.auto.commit to false for executor =>stop the executor to commit the offset auto. WARN overriding auto.offset.reset to none for executor =>it set the index where the executor read msg WARN overriding executor group.id to spark-executor-mygroupid => set the groupid of consumer. If you do not set, it will set a default and give a warning. WARN overriding receive.buffer.bytes to 65536 see KAFKA-3135 WARN overriding enable.auto.commit to false for executor WARN overriding auto.offset.reset to none for executor
