[jira] [Created] (BAHIR-183) Using HDFS for saving message for mqtt source

[Wang Yanlin (JIRA)]

Wang Yanlin created BAHIR-183:
---------------------------------

             Summary: Using HDFS for saving message for mqtt source
                 Key: BAHIR-183
                 URL: https://issues.apache.org/jira/browse/BAHIR-183
             Project: Bahir
          Issue Type: Improvement
          Components: Spark Structured Streaming Connectors
    Affects Versions: Spark-2.2.0
            Reporter: Wang Yanlin
             Fix For: Spark-2.2.1


Currently in spark-sql-streaming-mqtt, the received mqtt message is saved in a 
local file by driver, this will have the risks of losing data for cluster mode 
when application master failover occurs. So saving in-coming mqtt messages 
using a director in checkpoint will solve this problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)