[ https://issues.apache.org/jira/browse/BAHIR-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959299#comment-15959299 ]
Francesco Beneventi edited comment on BAHIR-104 at 4/6/17 5:01 PM: ------------------------------------------------------------------- [~anntinutj], I fixed the wrong method call before running my tests. You can check it in my test repo [Github|https://github.com/fbeneventi/bahir/commit/3755ecc56e19ac58fe20089a475253cc0bd19486#diff-f63d51c23efe7ce897bcc35fc52039f7] Even using the correct API (helper.createPairedStream) I have the same issue. I'm not really a java/scala programmer but can you help me figure out how to address this problem? After solving this problem I can provide a simple test case to use this API in python. was (Author: fbeneventi): [~anntinutj], I fixed the wrong method call before running my tests. You can check it in my test repo [Github|https://github.com/fbeneventi/bahir/commit/3755ecc56e19ac58fe20089a475253cc0bd19486#diff-f63d51c23efe7ce897bcc35fc52039f7] Even with the correct API (helper.createPairedStream) I have the same issue. I'm not really a java/scala programmer but can you help me figure out how to address this problem? After solving this problem I can provide a simple test case to use this API in python. > MQTT Dstream returned by the new multi topic support API is not a pairRDD > ------------------------------------------------------------------------- > > Key: BAHIR-104 > URL: https://issues.apache.org/jira/browse/BAHIR-104 > Project: Bahir > Issue Type: Bug > Components: Spark Streaming Connectors > Affects Versions: Spark-2.1.0 > Reporter: Francesco Beneventi > Labels: MQTT, SPARK > > The new multi topic support API added with [BAHIR-89], when used in pyspark, > does not return a Dstream of <topic,message> tuples. > Example: > In pyspark, when creating a Dstream using the new API ( mqttstream = > MQTTUtils.createPairedStream(ssc, brokerUrl, topics) ) the expected contents > of mqttstream should be a collections of tuples: > (topic,message) , (topic,message) , (topic,message) , ... > Instead, the current content is a flattened list: > topic, message, topic, message, topic, message, ... > that is hard to use. -- This message was sent by Atlassian JIRA (v6.3.15#6346)