Re: org.apache.spark.shuffle.FetchFailedException: Too large frame:

>> group by a. >> >> rb >> ​ >> >> On Tue, May 1, 2018 at 4:21 AM, Pralabh Kumar >> wrote: >> >>> Hi >>> >>> I am getting the above error in Spark SQL . I have increase (using 5000 >>> ) number of partitions but

Re: org.apache.spark.shuffle.FetchFailedException: Too large frame:

QL . I have increase (using 5000 ) >> number of partitions but still getting the same error . >> >> My data most probably is skew. >> >> >> >> org.apache.spark.shuffle.FetchFailedException: Too large frame: 4247124829 >> at >> org.ap

Re: org.apache.spark.shuffle.FetchFailedException: Too large frame:

t; number of partitions but still getting the same error . > > My data most probably is skew. > > > > org.apache.spark.shuffle.FetchFailedException: Too large frame: 4247124829 > at > org.apache.spark.storage.ShuffleBlockFetcherIterator.throwFetchFailedException(S

org.apache.spark.shuffle.FetchFailedException: Too large frame:

Hi I am getting the above error in Spark SQL . I have increase (using 5000 ) number of partitions but still getting the same error . My data most probably is skew. org.apache.spark.shuffle.FetchFailedException: Too large frame: 4247124829 at