RE: [EXT] handling skewness issues

2019-04-30 Thread email
Please share the links if they are publicly available. Otherwise please share the name of the talks. Thank you From: Jules Damji Sent: Monday, April 29, 2019 8:04 PM To: Michael Mansour Cc: rajat kumar ; user@spark.apache.org Subject: Re: [EXT] handling skewness issues Yes, indeed! A

Re: [EXT] handling skewness issues

2019-04-29 Thread Jules Damji
2019 at 9:34 AM > To: "user@spark.apache.org" > Subject: [EXT] handling skewness issues > > Hi All, > > How to overcome skewness issues in spark ? > > I read that we can add some randomness to key column before join and remove > that random part after

Re: [EXT] handling skewness issues

2019-04-29 Thread Michael Mansour
e.org" Subject: [EXT] handling skewness issues Hi All, How to overcome skewness issues in spark ? I read that we can add some randomness to key column before join and remove that random part after join. is there any better way ? Above method seems to be a workaround. thanks rajat