RE: Getting parent RDD

2015-09-16 Thread Samya MAITI
m: Akhil Das [mailto:ak...@sigmoidanalytics.com] Sent: Wednesday, September 16, 2015 12:24 PM To: Samya MAITI <samya.ma...@amadeus.com> Cc: user@spark.apache.org Subject: Re: Getting parent RDD ​How many RDDs are you having in that stream? If its a single RDD then you could do a .f

Re: Getting parent RDD

2015-09-16 Thread Akhil Das
ot > sure how), so as to refer to that in catch block. > > > > Regards, > > Sam > > > > *From:* Akhil Das [mailto:ak...@sigmoidanalytics.com] > *Sent:* Wednesday, September 16, 2015 12:24 PM > *To:* Samya MAITI <samya.ma...@amadeus.com> >

Re: Getting parent RDD

2015-09-16 Thread Akhil Das
​How many RDDs are you having in that stream? If its a single RDD then you could do a .foreach and log the message, something like: val ssc = val msgStream = . //SparkKafkaDirectAPI val wordCountPair = TransformStream.transform(msgStream) /wordCountPair.foreach( ​msg​ => try{