Re: distinct query getting stuck at ShuffleBlockFetcherIterator

[Ramesh Krishnan]

Hi Yash,
Thank you for the response.
Sorry it was not at distinct but it was at a join stage.
It was a self join. There were no errors and the jobs were stuck at the
step for a around 7 hrs, the last message that came through was .

*ShuffleBlockFetcherIterator: Started 4 remote fetches*
Thanks,
Ramesh

On Fri, Apr 7, 2017 at 10:05 AM, Yash Sharma  wrote:

> Hi Ramesh,
> Could you share some logs please? pastebin ? dag view ?
> Did you check for GC pauses if any.
>
> On Thu, 6 Apr 2017 at 21:55 Ramesh Krishnan 
> wrote:
>
>> I have a use case of distinct on a dataframe. When i run the application
>> is getting stuck at  LINE *ShuffleBlockFetcherIterator: Started 4 remote
>> fetches *forever.
>>
>> Can someone help .
>>
>>
>> Thanks
>> Ramesh
>>
>

Re: distinct query getting stuck at ShuffleBlockFetcherIterator

[Yash Sharma]

Hi Ramesh,
Could you share some logs please? pastebin ? dag view ?
Did you check for GC pauses if any.

On Thu, 6 Apr 2017 at 21:55 Ramesh Krishnan  wrote:

> I have a use case of distinct on a dataframe. When i run the application
> is getting stuck at  LINE *ShuffleBlockFetcherIterator: Started 4 remote
> fetches *forever.
>
> Can someone help .
>
>
> Thanks
> Ramesh
>