Hi Experts,
I am running a distinct query on parquet. I stuck after some time. Here is the
log. I have allocated Java direct memory 32GB and Heap size 24GB. There are
around 50 million record in parquet file.
uckets in hash table = 524288.
2014-10-31 02:06:09,717 [b54b8800-910c-4a01-a052-533def304e84:frag:2:6] DEBUG
o.a.d.e.p.impl.common.HashTable - Rehashing entries within the batch: 61;
batchStartIdx = 3997696, total numBuckets in hash table = 1048576.
2014-10-31 02:06:09,718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:1] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:4] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,718 [b54b8800-910c-4a01-a052-533def304e84:frag:2:0] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,719 [b54b8800-910c-4a01-a052-533def304e84:frag:2:2] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,724 [b54b8800-910c-4a01-a052-533def304e84:frag:2:9] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,725 [b54b8800-910c-4a01-a052-533def304e84:frag:2:10] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,730 [b54b8800-910c-4a01-a052-533def304e84:frag:2:8] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
2014-10-31 02:06:09,739 [b54b8800-910c-4a01-a052-533def304e84:frag:2:5] DEBUG
o.a.d.e.p.impl.common.HashTable - Put into hash table failed .. Retrying with
new batch holder...
Any pointer?
Thanks & regards
Tridib
