Are you sure that the data can be saved as strings? Another, more controlled approach is use DStream.foreachRDD , which takes a Function2 parameter, RDD and Time. There you can explicitly do stuff with the RDD, save it to separate files (separated by time), or whatever. Might help you to debug what is going on. Might help if you shows the streaming program in a pastebin.
TD On Fri, May 29, 2015 at 12:55 AM, Chaudhary, Umesh < umesh.chaudh...@searshc.com> wrote: > Hi, > > I have written manual receiver for Websphere MQ and its working fine. > > If I am doing *JavaDStream.SaveAsTextFile(“/home/user/out.txt”) * then > its generating a directory naming *out.txt* appending its timestamp. > > In this directory only *_SUCCESS* file is present. I can see data on > console while running in local mode but not able to save it as text file. > > Is there any other way for saving streaming data? > > > > *From:* Chaudhary, Umesh > *Sent:* Tuesday, May 26, 2015 2:39 AM > *To:* 'Arush Kharbanda'; user@spark.apache.org > *Subject:* RE: Websphere MQ as a data source for Apache Spark Streaming > > > > Thanks for the suggestion, I will try and post the outcome. > > > > *From:* Arush Kharbanda [mailto:ar...@sigmoidanalytics.com > <ar...@sigmoidanalytics.com>] > *Sent:* Monday, May 25, 2015 12:24 PM > *To:* Chaudhary, Umesh; user@spark.apache.org > *Subject:* Re: Websphere MQ as a data source for Apache Spark Streaming > > > > Hi Umesh, > > > > You can write a customer receiver for Websphere MQ, using the API for > websphere MQ. > > > > https://spark.apache.org/docs/latest/streaming-custom-receivers.html > > > > Thanks > > Arush > > > > On Mon, May 25, 2015 at 8:04 PM, Chaudhary, Umesh < > umesh.chaudh...@searshc.com> wrote: > > I have seen it but it has different configuration for connecting the MQ. > > I mean for Websphere MQ we need Host, Queue Manager, Channel And Queue > Name but here according to MQTT protocol > > > > client = new MqttClient(brokerUrl, MqttClient.generateClientId(), > persistence) > > > > It only expects Broker URL which is in appropriate for establishing > connection with Websphere MQ. > > > > Please Suggest ! > > > > > > *From:* Arush Kharbanda [mailto:ar...@sigmoidanalytics.com] > *Sent:* Monday, May 25, 2015 6:29 AM > *To:* Chaudhary, Umesh > *Cc:* user@spark.apache.org > *Subject:* Re: Websphere MQ as a data source for Apache Spark Streaming > > > > Hi Umesh, > > > > You can connect to Spark Streaming with MQTT refer to the example. > > > > > https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/MQTTWordCount.scala > > > > > > > > Thanks > > Arush > > > > > > > > On Mon, May 25, 2015 at 3:43 PM, umesh9794 <umesh.chaudh...@searshc.com> > wrote: > > I was digging into the possibilities for Websphere MQ as a data source for > spark-streaming becuase it is needed in one of our use case. I got to know > that MQTT <http://mqtt.org/> is the protocol that supports the > communication from MQ data structures but since I am a newbie to spark > streaming I need some working examples for the same. Did anyone try to > connect the MQ with spark streaming. Please devise the best way for doing > so. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Websphere-MQ-as-a-data-source-for-Apache-Spark-Streaming-tp23013.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > -- > > [image: Image removed by sender. Sigmoid Analytics] > <http://htmlsig.com/www.sigmoidanalytics.com> > > *Arush Kharbanda* || Technical Teamlead > > ar...@sigmoidanalytics.com || www.sigmoidanalytics.com > > This message, including any attachments, is the property of Sears Holdings > Corporation and/or one of its subsidiaries. It is confidential and may > contain proprietary or legally privileged information. If you are not the > intended recipient, please delete it without reading the contents. Thank > you. > > > > > > -- > > [image: Image removed by sender. Sigmoid Analytics] > <http://htmlsig.com/www.sigmoidanalytics.com> > > *Arush Kharbanda* || Technical Teamlead > > ar...@sigmoidanalytics.com || www.sigmoidanalytics.com > This message, including any attachments, is the property of Sears > Holdings Corporation and/or one of its subsidiaries. It is confidential and > may contain proprietary or legally privileged information. If you are not > the intended recipient, please delete it without reading the contents. > Thank you. >
