[
https://issues.apache.org/jira/browse/HIVE-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085489#comment-17085489
]
Hive QA commented on HIVE-23218:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000179/HIVE-23218.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17128 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21715/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21715/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21715/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000179 - PreCommit-HIVE-Build
> LlapRecordReader queue limit computation is not optimal
> -------------------------------------------------------
>
> Key: HIVE-23218
> URL: https://issues.apache.org/jira/browse/HIVE-23218
> Project: Hive
> Issue Type: Improvement
> Components: llap
> Reporter: Rajesh Balamohan
> Assignee: Ramesh Kumar Thangarajan
> Priority: Major
> Attachments: HIVE-23218.2.patch, HIVE-23218.3.patch
>
>
> After decoding {{OrcEncodedDataConsumer::decodeBatch}}, data is enqueued into
> a queue in LlapRecordReader. Queue limit for this queue is determined in
> LlapRecordReader. If it is minimal, it ends up waiting for 100ms until it
> gets capacity.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L168
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L590
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L260
> {{determineQueueLimit}} takes into consideration all columns though only few
> columns are needed for projection. Here is an example.
> {noformat}
> create table test_acid(a1 string, a2 string, a3 string, a4 string, a5 string,
> a6 string, a7 string, a8 string, a9 string, a10 string,
> a11 string, a22 string, a33 string, a44 string, a55 string, a66 string, a77
> string, a88 string, a99 string, a100 string,
> a111 decimal(25,2), a222 decimal(25,2), a333 decimal(25,2), a444
> decimal(25,2), a555 decimal(25,2), a666 decimal(25,2), a777 decimal(25,2),
> a888 decimal(25,2), a999 decimal(25,2), a1000 decimal(25,2)) stored as orc;
> insert into table test_acid values
> ("a1","a2","a3","a4","a5","a6","a7","a8","a9","a10",
> "a11","a22","a33","a44","a55","a66","a77","a88","a99","a100",
> 10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23
> );
> select a44, count(*) from test_acid where a44 like "a4%" group by a44 order
> by a44;
> {noformat}
> For this query, queue size predicted would be "138" as it takes into account
> all fields instead of just 2. This would causes unwanted delays in adding
> data to the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
