[
https://issues.apache.org/jira/browse/HIVE-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839762#comment-15839762
]
Hive QA commented on HIVE-15700:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12849487/HIVE-15700.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11001 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=93)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorStringExpressions.testLoadBytesColumnVectorByValueLargeData
(batchId=266)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3197/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3197/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3197/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12849487 - PreCommit-HIVE-Build
> BytesColumnVector can get stuck trying to resize byte buffer
> ------------------------------------------------------------
>
> Key: HIVE-15700
> URL: https://issues.apache.org/jira/browse/HIVE-15700
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
> Reporter: Jason Dere
> Assignee: Jason Dere
> Attachments: HIVE-15700.1.patch, HIVE-15700.2.patch
>
>
> While looking at HIVE-15698, hit an issue where one of the reducers was stuck
> in the following stack trace:
> {noformat}
> Thread 12735: (state = IN_JAVA)
> -
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.increaseBufferSpace(int)
> @bci=22, line=245 (Compiled frame; information may be imprecise)
> - org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(int,
> byte[], int, int) @bci=18, line=150 (Interpreted frame)
> -
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.storeRowColumn(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch,
> int, int, boolean) @bci=536, line=442 (Compiled frame)
> -
> org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch,
> int) @bci=110, line=761 (Interpreted frame)
> -
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(org.apache.hadoop.io.BytesWritable,
> java.lang.Iterable, byte) @bci=184, line=444 (Interpreted frame)
> - org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector()
> @bci=119, line=388 (Interpreted frame)
> - org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord() @bci=8,
> line=239 (Interpreted frame)
> - org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run() @bci=124,
> line=319 (Interpreted frame)
> -
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(java.util.Map,
> java.util.Map) @bci=30, line=185 (Interpreted frame)
> - org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(java.util.Map,
> java.util.Map) @bci=159, line=168 (Interpreted frame)
> - org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run() @bci=65,
> line=370 (Interpreted frame)
> - org.apache.tez.runtime.task.TaskRunner2Callable$1.run() @bci=133, line=73
> (Interpreted frame)
> - org.apache.tez.runtime.task.TaskRunner2Callable$1.run() @bci=1, line=61
> (Interpreted frame)
> -
> java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction,
> java.security.AccessControlContext) @bci=0 (Compiled frame)
> - javax.security.auth.Subject.doAs(javax.security.auth.Subject,
> java.security.PrivilegedExceptionAction) @bci=42, line=422 (Interpreted frame)
> -
> org.apache.hadoop.security.UserGroupInformation.doAs(java.security.PrivilegedExceptionAction)
> @bci=14, line=1724 (Interpreted frame)
> - org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() @bci=38,
> line=61 (Interpreted frame)
> - org.apache.tez.runtime.task.TaskRunner2Callable.callInternal() @bci=1,
> line=37 (Interpreted frame)
> - org.apache.tez.common.CallableWithNdc.call() @bci=8, line=36 (Interpreted
> frame)
> - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
> -
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
> @bci=95, line=1142 (Interpreted frame)
> - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617
> (Interpreted frame)
> - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)
> {noformat}
> The reducer's input was 167 9MB binary values coming from the previous map
> job. Per [~gopalv] the BytesColumnVector is stuck trying to reallocate/copy
> all of these values into the same memory buffer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
