[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094057#comment-16094057
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user lresende commented on a diff in the pull request:

https://github.com/apache/bahir/pull/47#discussion_r128406934
  
--- Diff: streaming-mqtt/README.md ---
@@ -52,12 +52,14 @@ this actor can be configured to handle failures, etc.
 
 val lines = MQTTUtils.createStream(ssc, brokerUrl, topic)
 val lines = MQTTUtils.createPairedStream(ssc, brokerUrl, topic)
+val lines = MQTTUtils.createPairedByteArrayStreamStream(ssc, 
brokerUrl, topic)
--- End diff --

Fixed the typo. Validated there was no other StreamStream


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
> Fix For: Spark-2.2.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094009#comment-16094009
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user ckadner commented on a diff in the pull request:

https://github.com/apache/bahir/pull/47#discussion_r128401599
  
--- Diff: streaming-mqtt/README.md ---
@@ -52,12 +52,14 @@ this actor can be configured to handle failures, etc.
 
 val lines = MQTTUtils.createStream(ssc, brokerUrl, topic)
 val lines = MQTTUtils.createPairedStream(ssc, brokerUrl, topic)
+val lines = MQTTUtils.createPairedByteArrayStreamStream(ssc, 
brokerUrl, topic)
--- End diff --

@davidrosenstark -- I assume the `StreamStream` word duplication is a 
copy-paste error?

~`val lines = MQTTUtils.createPairedByteArrayStreamStream(ssc, brokerUrl, 
topic)`~
`val lines = MQTTUtils.createPairedByteArrayStream(ssc, brokerUrl, topic)`


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
> Fix For: Spark-2.2.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[Luciano Resende (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luciano Resende resolved BAHIR-100.
---
   Resolution: Fixed
Fix Version/s: Spark-2.2.0

> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
> Fix For: Spark-2.2.0
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093975#comment-16093975
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user asfgit closed the pull request at:

https://github.com/apache/bahir/pull/47


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093494#comment-16093494
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user lresende commented on the issue:

https://github.com/apache/bahir/pull/47
  
LGTM, merging if there are no more comments.


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081749#comment-16081749
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user ApacheBahir commented on the issue:

https://github.com/apache/bahir/pull/47
  

Refer to this link for build results (access rights to CI server needed): 
http://169.45.79.58:8080/job/bahir_spark_pr_builder/72/



> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081738#comment-16081738
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user davidrosenstark commented on a diff in the pull request:

https://github.com/apache/bahir/pull/47#discussion_r126603788
  
--- Diff: 
streaming-mqtt/src/main/scala/org/apache/spark/streaming/mqtt/MQTTPairedByteArrayInputDStream.scala
 ---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.streaming.mqtt
+
+import org.eclipse.paho.client.mqttv3._
+import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence
+
+import org.apache.spark.storage.StorageLevel
+import org.apache.spark.streaming.StreamingContext
+import org.apache.spark.streaming.dstream._
+import org.apache.spark.streaming.receiver.Receiver
+
+/**
+ * Input stream that subscribe messages from a Mqtt Broker.
+ * Uses eclipse paho as MqttClient http://www.eclipse.org/paho/
+ * @param brokerUrl  Url of remote mqtt publisher
--- End diff --

this was missing in all three classes. adding to all 3


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> ------
>
> Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081553#comment-16081553
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user tedyu commented on a diff in the pull request:

https://github.com/apache/bahir/pull/47#discussion_r126583567
  
--- Diff: streaming-mqtt/README.md ---
@@ -52,12 +52,14 @@ this actor can be configured to handle failures, etc.
 
 val lines = MQTTUtils.createStream(ssc, brokerUrl, topic)
 val lines = MQTTUtils.createPairedStream(ssc, brokerUrl, topic)
+val lines = MQTTUtils.createPairedByteArrayStreamStream(ssc, 
brokerUrl, topic)
--- End diff --

Where is createPairedByteArrayStreamStream defined ?


> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

BAHIR-100

[Rosenstark, David]

Hi I have sent PR for BAHIR-100 and it now passes. What are the next steps 
needed to get it merged?
-
Intel Electronics Ltd.

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: failing build question BAHIR-100 PR

[Ted Yu]

Maybe we should upgrade Java 8 - I see 8.0_91 and latest is 8u131.

Other than that, someone with admin privilege can add the suggested ulimit
command at the beginning of the QA run.

Cheers

On Thu, Jul 6, 2017 at 11:16 PM, Rosenstark, David <
david.rosenst...@intel.com> wrote:

> http://169.45.79.58:8080/job/bahir_spark_pr_builder/69/console
>
> On 06/07/2017, 23:08, "Ted Yu"  wrote:
>
> Can you give the URL where the error was reported ?
>
> Thanks
>
> On Thu, Jul 6, 2017 at 1:05 PM, Rosenstark, David <
> david.rosenst...@intel.com> wrote:
>
> > Hi my build passes fine on my machine running oracle java 8.
> > But I see it failed on a different sub-project not affected by
> my PR
> > on Jenkins. But this is the output: (see below)
> > Is this a known intermittent issue? Is there some issue on build
> > machine? Any other hints?
> >
> > 12:48:40 - Send & Receive messages of size 128000 bytes.
> > 12:48:40 AkkaStreamSourceSuite:
> > 12:48:40 BasicAkkaSourceSuite:
> > 12:48:50 - basic usage
> > 12:49:00 - Send and receive 100 messages.
> > 12:49:00 - params not provided
> > 12:49:10 - Recovering offset from the last processed offset
> > 112:49:10 - Recovering offset from the last processed offset
> > 12:49:10 #
> > 12:49:10 # A fatal error has been detected by the Java Runtime
> > Environment:
> > 12:49:10 #
> > 12:49:10 #  SIGSEGV (0xb) at pc=0x7f3970105120, pid=24784,
> > tid=139896151623424
> > 12:49:10 #
> > 12:49:10 # JRE version: OpenJDK Runtime Environment (8.0_91-b14)
> > (build 1.8.0_91-b14)
> > 12:49:10 # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed
> mode
> > linux-amd64 compressed oops)
> > 12:49:10 # Problematic frame:
> > 12:49:10 # C  0x7f3970105120
> > 12:49:10 #
> > 12:49:10 # Failed to write core dump. Core dumps have been
> disabled.
> > To enable core dumping, try "ulimit -c unlimited" before starting
> Java again
> > 12:49:10 #
> > 12:49:10 # An error report file with more information is saved
> as:
> > 12:49:10 # /var/lib/jenkins/workspace/
> bahir_spark_pr_builder/sql-
> > streaming-akka/hs_err_pid24784.log
> > 12:49:10 #
> > 12:49:10 # If you would like to submit a bug report, please
> visit:
> > 12:49:10 #   http://bugreport.java.com/bugreport/crash.jsp
> > 12:49:10 # The crash happened outside the Java Virtual Machine in
> > native code.
> > 12:49:10 # See problematic frame for where to report the bug.
> >
> >
> >
> >
> >
> > 
> -
> > Intel Electronics Ltd.
> >
> > This e-mail and any attachments may contain confidential material for
> > the sole use of the intended recipient(s). Any review or distribution
> > by others is strictly prohibited. If you are not the intended
> > recipient, please contact the sender and delete all copies.
> >
>
>
> -
> Intel Electronics Ltd.
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077686#comment-16077686
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user ApacheBahir commented on the issue:

https://github.com/apache/bahir/pull/47
  

Refer to this link for build results (access rights to CI server needed): 
http://169.45.79.58:8080/job/bahir_spark_pr_builder/71/



> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077685#comment-16077685
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user ApacheBahir commented on the issue:

https://github.com/apache/bahir/pull/47
  
Build successful
 



> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

failing build question BAHIR-100 PR

[Rosenstark, David]

Hi my build passes fine on my machine running oracle java 8.
But I see it failed on a different sub-project not affected by my PR on 
Jenkins. But this is the output: (see below)
Is this a known intermittent issue? Is there some issue on build machine? 
Any other hints?

12:48:40 - Send & Receive messages of size 128000 bytes.
12:48:40 AkkaStreamSourceSuite:
12:48:40 BasicAkkaSourceSuite:
12:48:50 - basic usage
12:49:00 - Send and receive 100 messages.
12:49:00 - params not provided
12:49:10 - Recovering offset from the last processed offset
112:49:10 - Recovering offset from the last processed offset
12:49:10 #
12:49:10 # A fatal error has been detected by the Java Runtime Environment:
12:49:10 #
12:49:10 #  SIGSEGV (0xb) at pc=0x7f3970105120, pid=24784, 
tid=139896151623424
12:49:10 #
12:49:10 # JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 
1.8.0_91-b14)
12:49:10 # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode 
linux-amd64 compressed oops)
12:49:10 # Problematic frame:
12:49:10 # C  0x7f3970105120
12:49:10 #
12:49:10 # Failed to write core dump. Core dumps have been disabled. To 
enable core dumping, try "ulimit -c unlimited" before starting Java again
12:49:10 #
12:49:10 # An error report file with more information is saved as:
12:49:10 # 
/var/lib/jenkins/workspace/bahir_spark_pr_builder/sql-streaming-akka/hs_err_pid24784.log
12:49:10 #
12:49:10 # If you would like to submit a bug report, please visit:
12:49:10 #   http://bugreport.java.com/bugreport/crash.jsp
12:49:10 # The crash happened outside the Java Virtual Machine in native 
code.
12:49:10 # See problematic frame for where to report the bug.





-
Intel Electronics Ltd.

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077124#comment-16077124
 ] 

ASF GitHub Bot commented on BAHIR-100:
--

Github user ApacheBahir commented on the issue:

https://github.com/apache/bahir/pull/47
  

Refer to this link for build results (access rights to CI server needed): 
http://169.45.79.58:8080/job/bahir_spark_pr_builder/69/



> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[Anntinu Josy (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009477#comment-16009477
 ] 

Anntinu Josy commented on BAHIR-100:


Sorry for the delay, was busy with some personal things. I will create the PR 
in a week

> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[Christian Kadner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004023#comment-16004023
 ] 

Christian Kadner commented on BAHIR-100:


[~anntinutj] were you able to make any progress on this?

> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>Assignee: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encoded Byte[] message without corruption

[Christian Kadner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947787#comment-15947787
 ] 

Christian Kadner commented on BAHIR-100:


[~anntinutj] I was not aware of the issue you describe, feel free to create a 
PR, but please elaborate on a concrete example scenario that produces the 
corrupt result you described and make sure to add a test case that fails before 
your change and succeeds after your change. Thank you!

> Providing MQTT Spark Streaming to return encoded Byte[] message without 
> corruption
> --
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, Once this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BAHIR-100) Providing MQTT Spark Streaming to return encorded Byte[] message without corruption

[Anntinu Josy (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947644#comment-15947644
 ] 

Anntinu Josy commented on BAHIR-100:


Can I start working on this [~ckadner] and [~luciano resende] ?


> Providing MQTT Spark Streaming to return encorded Byte[] message without 
> corruption
> ---
>
>         Key: BAHIR-100
> URL: https://issues.apache.org/jira/browse/BAHIR-100
> Project: Bahir
>  Issue Type: New Feature
>  Components: Spark Streaming Connectors
>Reporter: Anntinu Josy
>  Labels: mqtt, spark, streaming
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Now a days Network bandwidth is becoming a serious resource that need to be 
> conserver in IoT ecosystem, For this puropse we are using different byte[] 
> based encoding such as Protocol Buffer and flat Buffer, One this encoded 
> message is converted into string the data becomes corrupted, So same byte[] 
> format need to be preserved when forwarded.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)