[
https://issues.apache.org/jira/browse/HIVE-15565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831308#comment-15831308
]
Hive QA commented on HIVE-15565:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12848220/HIVE-15565.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10963 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[specialChar] (batchId=22)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop]
(batchId=226)
org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path]
(batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape1]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape2]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval]
(batchId=87)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe]
(batchId=87)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3060/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3060/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3060/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12848220 - PreCommit-HIVE-Build
> LLAP: GroupByOperator flushes hash table too frequently
> -------------------------------------------------------
>
> Key: HIVE-15565
> URL: https://issues.apache.org/jira/browse/HIVE-15565
> Project: Hive
> Issue Type: Bug
> Components: llap
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-15565.1.patch, HIVE-15565.2.patch
>
>
> {{GroupByOperator::isTez}} would be true in LLAP mode. Current memory
> computations can go wrong with {{isTez}} checks in {{GroupByOperator}}. For
> e.g, in a LLAP instance with Xmx128G and 12 executors, it would start
> flushing hash table for every record once it reaches around 42GB
> (hive.tez.container.size=7100, hive.map.aggr.hash.percentmemory=0.5).
> {noformat}
> 2017-01-08T23:40:21,339 INFO [TezTaskRunner
> (1480722417364_1922_7_03_000004_1)]
> org.apache.hadoop.hive.ql.exec.GroupByOperator: Hash Table flushed: new size
> = 0
> 2017-01-08T23:40:21,339 INFO [TezTaskRunner
> (1480722417364_1922_7_03_000012_1)]
> org.apache.hadoop.hive.ql.exec.GroupByOperator: Hash Table flushed: new size
> = 0
> 2017-01-08T23:40:21,339 INFO [TezTaskRunner
> (1480722417364_1922_7_03_000004_1)]
> org.apache.hadoop.hive.ql.exec.GroupByOperator: Hash Tbl flush: #hash table =
> 1
> 2017-01-08T23:40:21,339 INFO [TezTaskRunner
> (1480722417364_1922_7_03_000012_1)]
> org.apache.hadoop.hive.ql.exec.GroupByOperator: Hash Tbl flush: #hash table =
> 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
