On 30 Nov 2012, at 16:46, Mark Grover wrote:
Hi David, It seems like Hive is unable to find the skewed keys on
HDFS. Did you set *hive.skewjoin.key property? If so, to what value?*
Hey Mark,
thanks for answering!
I didn't set it to anything, but left it at its default value (100,000
IIRC).
Hey David,
Sure thing. Play around with that property's value, see if that makes any
difference.
Also, if you could search to see if a file with a name like *hive_skew_join_
**bigkeys* exists on HDFS. Perhaps, it's looking at a different path. If
so, we can figure out how to fix that.
Mark
On
Hi,
I am trying to solve the last reducer hangs because of GC because of
truckloads of data issue that I have on some queries, by using SET
hive.optimize.skewjoin=true; Unfortunately, every time I try this, I
encounter an error of the form:
...
2012-11-30 10:42:39,181 Stage-10 map = 100%,
Hi David,
It seems like Hive is unable to find the skewed keys on HDFS.
Did you set *hive.skewjoin.key property? If so, to what value?*
Mark
On Fri, Nov 30, 2012 at 2:10 AM, David Morel david.mo...@amakuru.netwrote:
Hi,
I am trying to solve the last reducer hangs because of GC because of
