I have an RDD created as follows:
*JavaPairRDD inputDataFiles =
sparkContext.wholeTextFiles("hdfs://ip:8020/user/cdhuser/inputFolder/");*
On this RDD I perform a map to process individual files and invoke a
foreach to trigger the same map.
* JavaRDD
he worker logs or the driver?
>
>
> On Thursday, September 8, 2016, Nisha Menon <nisha.meno...@gmail.com>
> wrote:
>
>> I have an RDD created as follows:
>>
>> *JavaPairRDD<String,String> inputDataFiles =
>> sparkContext.wholeTextFiles("
a.a...@gmail.com> wrote:
> On yarn, logs are aggregated from each containers to hdfs. You can use
> yarn CLI or ui to view. For spark, you would have a history server which
> consolidate s the logs
> On 21 Sep 2016 19:03, "Nisha Menon" <nisha.meno...@gmail.com> wrote