[jira] [Commented] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791342#comment-16791342
]
Hive QA commented on HIVE-21434:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16479/dev-support/hive-personality.sh
|
| git revision | master / 13938db |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16479/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21434.patch
>
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(\*) on Aggregate and applying Filter on top
> of it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21435) LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if present, while building SubmitWorkRequestProto
[ https://issues.apache.org/jira/browse/HIVE-21435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chaurasia updated HIVE-21435: - Description: Currently we always pass [split number here as taskNum|[https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]]. However it should be taken from TASK_ATTEMPT_ID conf if it is present. (was: Currently we always pass [split number here as taskNum |[https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185],]. However it should be taken from TASK_ATTEMPT_ID conf if it is present.) > LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if > present, while building SubmitWorkRequestProto > -- > > Key: HIVE-21435 > URL: https://issues.apache.org/jira/browse/HIVE-21435 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Shubham Chaurasia >Assignee: Shubham Chaurasia >Priority: Major > > Currently we always pass [split number here as > taskNum|[https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]]. > However it should be taken from TASK_ATTEMPT_ID conf if it is present. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21435) LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if present, while building SubmitWorkRequestProto
[ https://issues.apache.org/jira/browse/HIVE-21435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chaurasia updated HIVE-21435: - Description: Currently we always pass [split number here as taskNum|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]. However it should be taken from TASK_ATTEMPT_ID conf if it is present. (was: Currently we always pass [split number here as taskNum|[https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]]. However it should be taken from TASK_ATTEMPT_ID conf if it is present.) > LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if > present, while building SubmitWorkRequestProto > -- > > Key: HIVE-21435 > URL: https://issues.apache.org/jira/browse/HIVE-21435 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Shubham Chaurasia >Assignee: Shubham Chaurasia >Priority: Major > > Currently we always pass [split number here as > taskNum|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]. > However it should be taken from TASK_ATTEMPT_ID conf if it is present. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21435) LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if present, while building SubmitWorkRequestProto
[ https://issues.apache.org/jira/browse/HIVE-21435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shubham Chaurasia reassigned HIVE-21435: > LlapBaseInputFormat should get task number from TASK_ATTEMPT_ID conf if > present, while building SubmitWorkRequestProto > -- > > Key: HIVE-21435 > URL: https://issues.apache.org/jira/browse/HIVE-21435 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Shubham Chaurasia >Assignee: Shubham Chaurasia >Priority: Major > > Currently we always pass [split number here as taskNum > |[https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185]|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapBaseInputFormat.java#L185],]. > However it should be taken from TASK_ATTEMPT_ID conf if it is present. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21338) Remove order by and limit for aggregates
[
https://issues.apache.org/jira/browse/HIVE-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791307#comment-16791307
]
Hive QA commented on HIVE-21338:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962197/HIVE-21338.8.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15824 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16478/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16478/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16478/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962197 - PreCommit-HIVE-Build
> Remove order by and limit for aggregates
>
>
> Key: HIVE-21338
> URL: https://issues.apache.org/jira/browse/HIVE-21338
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21338.1.patch, HIVE-21338.2.patch,
> HIVE-21338.3.patch, HIVE-21338.4.patch, HIVE-21338.5.patch,
> HIVE-21338.6.patch, HIVE-21338.7.patch, HIVE-21338.8.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a query is guaranteed to produce at most one row LIMIT and ORDER BY could
> be removed. This saves unnecessary vertex for LIMIT/ORDER BY.
> {code:sql}
> explain select count(*) cs from store_sales where ss_ext_sales_price > 100.00
> order by cs limit 100
> {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> DagId: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Edges:
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: store_sales
> filterExpr: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Group By Operator
> aggregations: count()
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: bigint)
> Execution mode: vectorized
> Reducer 2
> Execution mode: vectorized
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(VALUE._col0)
> mode: mergepartial
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: bigint)
> sort order: +
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> TopN Hash Memory Usage: 0.1
> Reducer 3
> Execution mode: vectorized
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: bigint)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 100
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Fix Version/s: 4.0.0 > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Resolution: Fixed Status: Resolved (was: Patch Available) 02.patch committed to master. Thanks [~maheshk114] for the review! > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21338) Remove order by and limit for aggregates
[
https://issues.apache.org/jira/browse/HIVE-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791291#comment-16791291
]
Hive QA commented on HIVE-21338:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16478/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16478/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove order by and limit for aggregates
>
>
> Key: HIVE-21338
> URL: https://issues.apache.org/jira/browse/HIVE-21338
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21338.1.patch, HIVE-21338.2.patch,
> HIVE-21338.3.patch, HIVE-21338.4.patch, HIVE-21338.5.patch,
> HIVE-21338.6.patch, HIVE-21338.7.patch, HIVE-21338.8.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a query is guaranteed to produce at most one row LIMIT and ORDER BY could
> be removed. This saves unnecessary vertex for LIMIT/ORDER BY.
> {code:sql}
> explain select count(*) cs from store_sales where ss_ext_sales_price > 100.00
> order by cs limit 100
> {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> DagId: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Edges:
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: store_sales
> filterExpr: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: (ss_ext_sales_price > 100) (type: boolean)
>
[jira] [Commented] (HIVE-21401) Break up DDLTask - extract Table related operations
[
https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791259#comment-16791259
]
Hive QA commented on HIVE-21401:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962163/HIVE-21401.14.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16477/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16477/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16477/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12962163/HIVE-21401.14.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962163 - PreCommit-HIVE-Build
> Break up DDLTask - extract Table related operations
> ---
>
> Key: HIVE-21401
> URL: https://issues.apache.org/jira/browse/HIVE-21401
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch,
> HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch,
> HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch,
> HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch,
> HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #2: extract all the table related operations from the old DDLTask except
> alter table, and move them under the new package. Also create the new
> internal framework.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21399) Adjust hive.map.aggr.hash.min.reduction statically depending on group by statistics
[ https://issues.apache.org/jira/browse/HIVE-21399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21399: --- Attachment: HIVE-21399.04.patch > Adjust hive.map.aggr.hash.min.reduction statically depending on group by > statistics > --- > > Key: HIVE-21399 > URL: https://issues.apache.org/jira/browse/HIVE-21399 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-21399.01.patch, HIVE-21399.02.patch, > HIVE-21399.03.patch, HIVE-21399.04.patch > > > Currently, the value is set statically from config variable. If stats are > available, we could try to adjust this value at optimization time to favor > turning off hash aggregation earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21401) Break up DDLTask - extract Table related operations
[
https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791232#comment-16791232
]
Hive QA commented on HIVE-21401:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} hcatalog/core in master has 29 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 17 new + 1650 unchanged - 141
fixed = 1667 total (was 1791) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hcatalog/core: The patch generated 1 new + 40
unchanged - 8 fixed = 41 total (was 48) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
26s{color} | {color:red} ql generated 1 new + 2256 unchanged - 2 fixed = 2257
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} contrib in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} hcatalog/core generated 0 new + 28 unchanged - 1
fixed = 28 total (was 29) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
55s{color} | {color:green} util in the patch passed. {color} |
|
[jira] [Commented] (HIVE-21401) Break up DDLTask - extract Table related operations
[
https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791242#comment-16791242
]
Hive QA commented on HIVE-21401:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962163/HIVE-21401.14.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16476/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16476/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16476/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962163 - PreCommit-HIVE-Build
> Break up DDLTask - extract Table related operations
> ---
>
> Key: HIVE-21401
> URL: https://issues.apache.org/jira/browse/HIVE-21401
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch,
> HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch,
> HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch,
> HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch,
> HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #2: extract all the table related operations from the old DDLTask except
> alter table, and move them under the new package. Also create the new
> internal framework.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[
https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791177#comment-16791177
]
Hive QA commented on HIVE-21421:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962156/HIVE-21421.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15824 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16475/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16475/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16475/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962156 - PreCommit-HIVE-Build
> HiveStatement.getQueryId throws NPE when query is not running.
> ---
>
> Key: HIVE-21421
> URL: https://issues.apache.org/jira/browse/HIVE-21421
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HiveStatement.getQueryId throws NullPointerException if it invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21434:
---
Status: Patch Available (was: In Progress)
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(*) on Aggregate and applying Filter on top of
> it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[
https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791146#comment-16791146
]
Hive QA commented on HIVE-21421:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} jdbc: The patch generated 0 new + 5 unchanged - 1
fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16475/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: jdbc itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16475/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveStatement.getQueryId throws NPE when query is not running.
> ---
>
> Key: HIVE-21421
> URL: https://issues.apache.org/jira/browse/HIVE-21421
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HiveStatement.getQueryId throws NullPointerException if it invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21368) Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
[
https://issues.apache.org/jira/browse/HIVE-21368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791123#comment-16791123
]
Hive QA commented on HIVE-21368:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962146/HIVE-21368.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15826 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=190)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16474/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16474/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16474/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962146 - PreCommit-HIVE-Build
> Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
> --
>
> Key: HIVE-21368
> URL: https://issues.apache.org/jira/browse/HIVE-21368
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-21368.1.patch, HIVE-21368.2.patch
>
>
> Joins projecting Decimal64 have a suspicious cast in the inner loop
> {code}
> ConvertDecimal64ToDecimal(col 14:decimal(7,2)/DECIMAL_64) -> 24:decimal(7,2)'
> {code}
> {code}
> create temporary table foo(x int , y decimal(7,2));
> create temporary table bar(x int , y decimal(7,2));
> set hive.explain.user=false;
> explain vectorization detail select sum(foo.y) from foo, bar where foo.x =
> bar.x;
> {code}
> {code}
> ' Map Join Operator'
> 'condition map:'
> ' Inner Join 0 to 1'
> 'keys:'
> ' 0 _col0 (type: int)'
> ' 1 _col0 (type: int)'
> 'Map Join Vectorization:'
> 'bigTableKeyColumnNums: [0]'
> 'bigTableRetainedColumnNums: [3]'
> 'bigTableValueColumnNums: [3]'
> 'bigTableValueExpressions:
> ConvertDecimal64ToDecimal(col 1:decimal(7,2)/DECIMAL_64) -> 3:decimal(7,2)'
> 'className: VectorMapJoinInnerBigOnlyLongOperator'
> 'native: true'
> 'nativeConditionsMet:
> hive.mapjoin.optimized.hashtable IS true,
> hive.vectorized.execution.mapjoin.native.enabled IS true,
> hive.execution.engine tez IN [tez, spark] IS true, One MapJoin Condition IS
> true, No nullsafe IS true, Small table vectorizes IS true, Fast Hash Table
> and No Hybrid Hash Join IS true'
> 'projectedOutputColumnNums: [3]'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21368) Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
[
https://issues.apache.org/jira/browse/HIVE-21368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791095#comment-16791095
]
Hive QA commented on HIVE-21368:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16474/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16474/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16474/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
> --
>
> Key: HIVE-21368
> URL: https://issues.apache.org/jira/browse/HIVE-21368
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-21368.1.patch, HIVE-21368.2.patch
>
>
> Joins projecting Decimal64 have a suspicious cast in the inner loop
> {code}
> ConvertDecimal64ToDecimal(col 14:decimal(7,2)/DECIMAL_64) -> 24:decimal(7,2)'
> {code}
> {code}
> create temporary table foo(x int , y decimal(7,2));
> create temporary table bar(x int , y decimal(7,2));
> set hive.explain.user=false;
> explain vectorization detail select sum(foo.y) from foo, bar where foo.x =
> bar.x;
> {code}
> {code}
> ' Map Join Operator'
> 'condition map:'
> ' Inner Join 0 to 1'
> 'keys:'
> ' 0 _col0 (type: int)'
> ' 1 _col0 (type: int)'
> 'Map Join Vectorization:'
> '
[jira] [Updated] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21434:
---
Description:
Currently there is a limitation in the rule implementation.
{code}
...
// No room for optimization since we cannot convert from non-empty
// GROUP BY list to the empty one.
...
{code}
The rule extension should try to overcome the limitation by implementing this
specific case calculating count(\*) on Aggregate and applying Filter on top of
it.
was:
Currently there is a limitation in the rule implementation.
{code}
...
// No room for optimization since we cannot convert from non-empty
// GROUP BY list to the empty one.
...
{code}
The rule extension should try to overcome the limitation by implementing this
specific case calculating count(*) on Aggregate and applying Filter on top of
it.
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21434.patch
>
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(\*) on Aggregate and applying Filter on top
> of it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791091#comment-16791091
]
Jesus Camacho Rodriguez commented on HIVE-21434:
Initial patch to trigger ptest.
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21434.patch
>
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(\*) on Aggregate and applying Filter on top
> of it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-21434:
---
Attachment: HIVE-21434.patch
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-21434.patch
>
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(*) on Aggregate and applying Filter on top of
> it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21434 started by Jesus Camacho Rodriguez.
--
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(*) on Aggregate and applying Filter on top of
> it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21434) Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if possible
[
https://issues.apache.org/jira/browse/HIVE-21434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-21434:
--
> Extend HiveAggregatePullUpConstantsRule to remove all grouping columns if
> possible
> --
>
> Key: HIVE-21434
> URL: https://issues.apache.org/jira/browse/HIVE-21434
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently there is a limitation in the rule implementation.
> {code}
> ...
> // No room for optimization since we cannot convert from non-empty
> // GROUP BY list to the empty one.
> ...
> {code}
> The rule extension should try to overcome the limitation by implementing this
> specific case calculating count(*) on Aggregate and applying Filter on top of
> it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21424) Disable AggregateStatsCache by default
[
https://issues.apache.org/jira/browse/HIVE-21424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791081#comment-16791081
]
Hive QA commented on HIVE-21424:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962145/HIVE-21424.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15824 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=263)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=263)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=263)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=263)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16473/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16473/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16473/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962145 - PreCommit-HIVE-Build
> Disable AggregateStatsCache by default
> --
>
> Key: HIVE-21424
> URL: https://issues.apache.org/jira/browse/HIVE-21424
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21424.01.patch, HIVE-21424.02.patch,
> HIVE-21424.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21338) Remove order by and limit for aggregates
[
https://issues.apache.org/jira/browse/HIVE-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21338:
---
Status: Open (was: Patch Available)
> Remove order by and limit for aggregates
>
>
> Key: HIVE-21338
> URL: https://issues.apache.org/jira/browse/HIVE-21338
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21338.1.patch, HIVE-21338.2.patch,
> HIVE-21338.3.patch, HIVE-21338.4.patch, HIVE-21338.5.patch,
> HIVE-21338.6.patch, HIVE-21338.7.patch, HIVE-21338.8.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a query is guaranteed to produce at most one row LIMIT and ORDER BY could
> be removed. This saves unnecessary vertex for LIMIT/ORDER BY.
> {code:sql}
> explain select count(*) cs from store_sales where ss_ext_sales_price > 100.00
> order by cs limit 100
> {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> DagId: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Edges:
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: store_sales
> filterExpr: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Group By Operator
> aggregations: count()
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: bigint)
> Execution mode: vectorized
> Reducer 2
> Execution mode: vectorized
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(VALUE._col0)
> mode: mergepartial
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: bigint)
> sort order: +
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> TopN Hash Memory Usage: 0.1
> Reducer 3
> Execution mode: vectorized
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: bigint)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 100
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde:
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 100
> Processor Tree:
> ListSink
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21338) Remove order by and limit for aggregates
[
https://issues.apache.org/jira/browse/HIVE-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21338:
---
Attachment: HIVE-21338.8.patch
> Remove order by and limit for aggregates
>
>
> Key: HIVE-21338
> URL: https://issues.apache.org/jira/browse/HIVE-21338
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21338.1.patch, HIVE-21338.2.patch,
> HIVE-21338.3.patch, HIVE-21338.4.patch, HIVE-21338.5.patch,
> HIVE-21338.6.patch, HIVE-21338.7.patch, HIVE-21338.8.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a query is guaranteed to produce at most one row LIMIT and ORDER BY could
> be removed. This saves unnecessary vertex for LIMIT/ORDER BY.
> {code:sql}
> explain select count(*) cs from store_sales where ss_ext_sales_price > 100.00
> order by cs limit 100
> {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> DagId: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Edges:
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: store_sales
> filterExpr: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Group By Operator
> aggregations: count()
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: bigint)
> Execution mode: vectorized
> Reducer 2
> Execution mode: vectorized
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(VALUE._col0)
> mode: mergepartial
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: bigint)
> sort order: +
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> TopN Hash Memory Usage: 0.1
> Reducer 3
> Execution mode: vectorized
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: bigint)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 100
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde:
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 100
> Processor Tree:
> ListSink
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21338) Remove order by and limit for aggregates
[
https://issues.apache.org/jira/browse/HIVE-21338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-21338:
---
Status: Patch Available (was: Open)
> Remove order by and limit for aggregates
>
>
> Key: HIVE-21338
> URL: https://issues.apache.org/jira/browse/HIVE-21338
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21338.1.patch, HIVE-21338.2.patch,
> HIVE-21338.3.patch, HIVE-21338.4.patch, HIVE-21338.5.patch,
> HIVE-21338.6.patch, HIVE-21338.7.patch, HIVE-21338.8.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a query is guaranteed to produce at most one row LIMIT and ORDER BY could
> be removed. This saves unnecessary vertex for LIMIT/ORDER BY.
> {code:sql}
> explain select count(*) cs from store_sales where ss_ext_sales_price > 100.00
> order by cs limit 100
> {code}
> {code}
> STAGE PLANS:
> Stage: Stage-1
> Tez
> DagId: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Edges:
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> DagName: vgarg_20190227131959_2914830f-eab6-425d-b9f0-b8cb56f8a1e9:4
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: store_sales
> filterExpr: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Filter Operator
> predicate: (ss_ext_sales_price > 100) (type: boolean)
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Select Operator
> Statistics: Num rows: 1 Data size: 112 Basic stats:
> COMPLETE Column stats: NONE
> Group By Operator
> aggregations: count()
> mode: hash
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> Reduce Output Operator
> sort order:
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> value expressions: _col0 (type: bigint)
> Execution mode: vectorized
> Reducer 2
> Execution mode: vectorized
> Reduce Operator Tree:
> Group By Operator
> aggregations: count(VALUE._col0)
> mode: mergepartial
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Reduce Output Operator
> key expressions: _col0 (type: bigint)
> sort order: +
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> TopN Hash Memory Usage: 0.1
> Reducer 3
> Execution mode: vectorized
> Reduce Operator Tree:
> Select Operator
> expressions: KEY.reducesinkkey0 (type: bigint)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 120 Basic stats: COMPLETE
> Column stats: NONE
> Limit
> Number of rows: 100
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> File Output Operator
> compressed: false
> Statistics: Num rows: 1 Data size: 120 Basic stats:
> COMPLETE Column stats: NONE
> table:
> input format:
> org.apache.hadoop.mapred.SequenceFileInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
> serde:
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> Stage: Stage-0
> Fetch Operator
> limit: 100
> Processor Tree:
> ListSink
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791038#comment-16791038
]
Hive QA commented on HIVE-15406:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962140/HIVE-15406.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_ppr] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1] (batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_trunc]
(batchId=8)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16472/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16472/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16472/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962140 - PreCommit-HIVE-Build
> Consider vectorizing the new 'trunc' function
> -
>
> Key: HIVE-15406
> URL: https://issues.apache.org/jira/browse/HIVE-15406
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Matt McCline
>Assignee: Laszlo Bodor
>Priority: Critical
> Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch
>
>
> Rounding function 'trunc' added by HIVE-14582.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791025#comment-16791025
]
Vineet Garg commented on HIVE-21382:
[~jcamachorodriguez] Can you take a look:
https://github.com/apache/hive/pull/567
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?focusedWorklogId=212055=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-212055
]
ASF GitHub Bot logged work on HIVE-21382:
-
Author: ASF GitHub Bot
Created on: 12/Mar/19 21:21
Start Date: 12/Mar/19 21:21
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #567:
HIVE-21382: Group by keys reduction optimization - keys are not reduced in
query23
URL: https://github.com/apache/hive/pull/567
[HIVE-21382](https://issues.apache.org/jira/browse/HIVE-21382)
Further enhancements to Group by key reduction in presence of PK/UNIQUE
constraints. This patch enables reducing keys in following cases:
* If group by keys originate from multiple tables (e.g. GBY after JOIN)
* If group by keys contain deterministic expression
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 212055)
Time Spent: 10m
Remaining Estimate: 0h
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21382) Group by keys reduction optimization - keys are not reduced in query23
[
https://issues.apache.org/jira/browse/HIVE-21382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-21382:
--
Labels: pull-request-available (was: )
> Group by keys reduction optimization - keys are not reduced in query23
> --
>
> Key: HIVE-21382
> URL: https://issues.apache.org/jira/browse/HIVE-21382
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21382.1.patch, HIVE-21382.2.patch,
> HIVE-21382.2.patch, HIVE-21382.3.patch
>
>
> {code:sql}
> explain cbo with frequent_ss_items as
> (select substr(i_item_desc,1,30) itemdesc,i_item_sk item_sk,d_date
> solddate,count(*) cnt
> from store_sales
> ,date_dim
> ,item
> where ss_sold_date_sk = d_date_sk
> and ss_item_sk = i_item_sk
> and d_year in (1999,1999+1,1999+2,1999+3)
> group by substr(i_item_desc,1,30),i_item_sk,d_date
> having count(*) >4)
> select sum(sales)
> from ((select cs_quantity*cs_list_price sales
>from catalog_sales
>,date_dim
>where d_year = 1999
> and d_moy = 1
> and cs_sold_date_sk = d_date_sk
> and cs_item_sk in (select item_sk from frequent_ss_items))) subq
> limit 100;
> {code}
> {code:sql}
> HiveSortLimit(fetch=[100])
> HiveProject($f0=[$0])
> HiveAggregate(group=[{}], agg#0=[sum($0)])
> HiveProject(sales=[*(CAST($2):DECIMAL(10, 0), $3)])
> HiveSemiJoin(condition=[=($1, $5)], joinType=[inner])
> HiveJoin(condition=[=($0, $4)], joinType=[inner], algorithm=[none],
> cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(cs_sold_date_sk=[$0], cs_item_sk=[$15],
> cs_quantity=[$18], cs_list_price=[$20])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints, catalog_sales]],
> table:alias=[catalog_sales])
> HiveProject(d_date_sk=[$0])
> HiveFilter(condition=[AND(=($6, 1999), =($8, 1))])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$1])
> HiveFilter(condition=[>($3, 4)])
> HiveProject(substr=[$2], i_item_sk=[$1], d_date=[$0], $f3=[$3])
> HiveAggregate(group=[{3, 4, 5}], agg#0=[count()])
> HiveJoin(condition=[=($1, $4)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($0, $2)], joinType=[inner],
> algorithm=[none], cost=[{2.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(ss_sold_date_sk=[$0], ss_item_sk=[$2])
> HiveFilter(condition=[IS NOT NULL($0)])
> HiveTableScan(table=[[perf_constraints,
> store_sales]], table:alias=[store_sales])
> HiveProject(d_date_sk=[$0], d_date=[$2])
> HiveFilter(condition=[IN($6, 1999, 2000, 2001, 2002)])
> HiveTableScan(table=[[perf_constraints, date_dim]],
> table:alias=[date_dim])
> HiveProject(i_item_sk=[$0], substr=[substr($4, 1, 30)])
> HiveTableScan(table=[[perf_constraints, item]],
> table:alias=[item])
> {code}
> Right side of HiveSemiJoin has an aggregate which could be reduce to have
> only {{i_item_sk}} as group by key since {{i_item_sk}} is primary key.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-17061) Add Support for Column List in Insert Clause
[
https://issues.apache.org/jira/browse/HIVE-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan resolved HIVE-17061.
-
Resolution: Fixed
Fix Version/s: 4.0.0
Resolved via HIVE-20590
> Add Support for Column List in Insert Clause
>
>
> Key: HIVE-17061
> URL: https://issues.apache.org/jira/browse/HIVE-17061
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Shawn Weeks
>Priority: Minor
> Fix For: 4.0.0
>
>
> Include support for a list of columns in the insert clause of the merge
> statement. This helps when you may not know or care about the order of
> columns in the target table or if you don't want to have to insert values
> into all of the columns.
> {code}
> MERGE INTO target
> USING source ON b = y
> WHEN MATCHED AND c + 1 + z > 0
> THEN UPDATE SET a = 1, c = z
> WHEN NOT MATCHED AND z IS NULL
> THEN INSERT(a,b) VALUES(z, 7)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21424) Disable AggregateStatsCache by default
[ https://issues.apache.org/jira/browse/HIVE-21424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791010#comment-16791010 ] Ashutosh Chauhan commented on HIVE-21424: - +1 pending tests. > Disable AggregateStatsCache by default > -- > > Key: HIVE-21424 > URL: https://issues.apache.org/jira/browse/HIVE-21424 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21424.01.patch, HIVE-21424.02.patch, > HIVE-21424.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791006#comment-16791006
]
Hive QA commented on HIVE-15406:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 16 new + 4 unchanged - 1 fixed
= 20 total (was 5) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
22s{color} | {color:red} ql generated 2 new + 2258 unchanged - 0 fixed = 2260
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 7 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Found reliance on default encoding in new
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp(int,
byte[], int):in new
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp(int,
byte[], int): new String(byte[]) At TruncDateFromTimestamp.java:[line 33] |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp.processDate(BytesColumnVector,
int, Date):in
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp.processDate(BytesColumnVector,
int, Date): String.getBytes() At TruncDateFromTimestamp.java:[line 145] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16472/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16472/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16472/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16472/yetus/new-findbugs-ql.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16472/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16472/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Consider vectorizing the new 'trunc' function
> -
>
> Key: HIVE-15406
> URL: https://issues.apache.org/jira/browse/HIVE-15406
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Matt McCline
>Assignee: Laszlo Bodor
>Priority: Critical
>
[jira] [Commented] (HIVE-21430) INSERT into a dynamically partitioned table with hive.stats.autogather = false throws a MetaException
[
https://issues.apache.org/jira/browse/HIVE-21430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791005#comment-16791005
]
Ashutosh Chauhan commented on HIVE-21430:
-
IMHO, we shall remove the config {{hive.stats.autogather}} since a) overhead to
collect stats is negligible, so its always on. b) StatsTask always need to be
present since we need to invalidate stats when we are not collecting them. So,
better to remove this config and simplify the codebase (and avoid issues like
this).
> INSERT into a dynamically partitioned table with hive.stats.autogather =
> false throws a MetaException
> -
>
> Key: HIVE-21430
> URL: https://issues.apache.org/jira/browse/HIVE-21430
> Project: Hive
> Issue Type: Bug
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Attachments: metaexception_repro.patch,
> org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread-output.txt
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When the test TestStatsUpdaterThread#testTxnDynamicPartitions added in the
> attached patch is run it throws exception (full logs attached.)
> org.apache.hadoop.hive.metastore.api.MetaException: Cannot change stats state
> for a transactional table default.simple_stats without providing the
> transactional write state for verification (new write ID 5, valid write IDs
> null; current state \{"BASIC_STATS":"true","COLUMN_STATS":{"s":"true"}}; new
> state null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.alterPartitionNoTxn(ObjectStore.java:4328)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
[
https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790982#comment-16790982
]
Hive QA commented on HIVE-20580:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962134/HIVE-20580.4.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
org.apache.hadoop.hive.ql.TestAcidOnTez.testIsOriginal (batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16471/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16471/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16471/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962134 - PreCommit-HIVE-Build
> OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
> --
>
> Key: HIVE-20580
> URL: https://issues.apache.org/jira/browse/HIVE-20580
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20580.2.patch, HIVE-20580.3.patch,
> HIVE-20580.4.patch, HIVE-20580.patch
>
>
> {{org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isOriginal()}} is checking
> for presence of {{hive.acid.key.index}} in the footer. This is only created
> when the file is written by {{OrcRecordUpdater}}. It should instead check
> for presence of Acid metadata columns so that a file can be produced by
> something other than {{OrcRecordUpater}}.
> Also, {{hive.acid.key.index}} counts number of different type of events which
> is not really useful for Acid V2 (as of Hive 3) since each file only has 1
> type of event.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
[
https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790941#comment-16790941
]
Hive QA commented on HIVE-20580:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} ql: The patch generated 0 new + 327 unchanged - 1
fixed = 327 total (was 328) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16471/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16471/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
> --
>
> Key: HIVE-20580
> URL: https://issues.apache.org/jira/browse/HIVE-20580
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20580.2.patch, HIVE-20580.3.patch,
> HIVE-20580.4.patch, HIVE-20580.patch
>
>
> {{org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isOriginal()}} is checking
> for presence of {{hive.acid.key.index}} in the footer. This is only created
> when the file is written by {{OrcRecordUpdater}}. It should instead check
> for presence of Acid metadata columns so that a file can be produced by
> something other than {{OrcRecordUpater}}.
> Also, {{hive.acid.key.index}} counts number of different type of events which
> is not
[jira] [Commented] (HIVE-21425) Use DirectExecutorService for getInputSummary
[
https://issues.apache.org/jira/browse/HIVE-21425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790894#comment-16790894
]
Hive QA commented on HIVE-21425:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962128/HIVE-21425.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15824 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16470/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16470/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16470/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962128 - PreCommit-HIVE-Build
> Use DirectExecutorService for getInputSummary
> -
>
> Key: HIVE-21425
> URL: https://issues.apache.org/jira/browse/HIVE-21425
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21425.1.patch, HIVE-21425.1.patch,
> HIVE-21425.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|title=Utilities.java}
> int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
> pathNeedProcess.size());
> if (numExecutors > 1) {
> LOG.info("Using {} threads for getContentSummary", numExecutors);
> executor = Executors.newFixedThreadPool(numExecutors,
> new ThreadFactoryBuilder().setDaemon(true)
> .setNameFormat("Get-Input-Summary-%d").build());
> } else {
> executor = null;
> }
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2482-L2490
> Instead of using a 'null' {{ExecutorService}}, use Guava's
> {{DirectExecutorService}} and remove special casing for a 'null' value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21425) Use DirectExecutorService for getInputSummary
[
https://issues.apache.org/jira/browse/HIVE-21425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790855#comment-16790855
]
Hive QA commented on HIVE-21425:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 112 unchanged - 1
fixed = 112 total (was 113) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16470/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16470/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use DirectExecutorService for getInputSummary
> -
>
> Key: HIVE-21425
> URL: https://issues.apache.org/jira/browse/HIVE-21425
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21425.1.patch, HIVE-21425.1.patch,
> HIVE-21425.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|title=Utilities.java}
> int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
> pathNeedProcess.size());
> if (numExecutors > 1) {
> LOG.info("Using {} threads for getContentSummary", numExecutors);
> executor = Executors.newFixedThreadPool(numExecutors,
> new ThreadFactoryBuilder().setDaemon(true)
> .setNameFormat("Get-Input-Summary-%d").build());
> } else {
> executor = null;
> }
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2482-L2490
> Instead of using a 'null' {{ExecutorService}}, use Guava's
> {{DirectExecutorService}} and remove special casing for a 'null' value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21426) Remove Utilities Global Random
[
https://issues.apache.org/jira/browse/HIVE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790820#comment-16790820
]
Hive QA commented on HIVE-21426:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962129/HIVE-21426.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15824 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16469/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16469/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16469/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962129 - PreCommit-HIVE-Build
> Remove Utilities Global Random
> --
>
> Key: HIVE-21426
> URL: https://issues.apache.org/jira/browse/HIVE-21426
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21426.1.patch, HIVE-21426.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L253
> Remove global {{Random}} object in favor of {{ThreadLocalRandom}}.
> {quote}
> ThreadLocalRandom is initialized with an internally generated seed that may
> not otherwise be modified. When applicable, use of ThreadLocalRandom rather
> than shared Random objects in concurrent programs will typically encounter
> much less overhead and contention.
> {quote}
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadLocalRandom.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790790#comment-16790790 ] mahesh kumar behera commented on HIVE-21421: [^HIVE-21421.02.patch] looks fine to me +1 pending test > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21426) Remove Utilities Global Random
[
https://issues.apache.org/jira/browse/HIVE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790773#comment-16790773
]
Hive QA commented on HIVE-21426:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 139 unchanged - 1
fixed = 139 total (was 140) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
32s{color} | {color:green} ql generated 0 new + 2256 unchanged - 2 fixed = 2256
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16469/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16469/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove Utilities Global Random
> --
>
> Key: HIVE-21426
> URL: https://issues.apache.org/jira/browse/HIVE-21426
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21426.1.patch, HIVE-21426.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L253
> Remove global {{Random}} object in favor of {{ThreadLocalRandom}}.
> {quote}
> ThreadLocalRandom is initialized with an internally generated seed that may
> not otherwise be modified. When applicable, use of ThreadLocalRandom rather
> than shared Random objects in concurrent programs will typically encounter
> much less overhead and contention.
> {quote}
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadLocalRandom.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Patch Available (was: Open) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Attachment: HIVE-21401.14.patch > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Open (was: Patch Available) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Open (was: Patch Available) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Attachment: (was: HIVE-21401.14.patch) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Attachment: HIVE-21401.14.patch > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Patch Available (was: Open) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch, HIVE-21401.14.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21404) MSSQL upgrade script alters the wrong column
[
https://issues.apache.org/jira/browse/HIVE-21404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790711#comment-16790711
]
Zoltan Haindrich commented on HIVE-21404:
-
+1
> MSSQL upgrade script alters the wrong column
>
>
> Key: HIVE-21404
> URL: https://issues.apache.org/jira/browse/HIVE-21404
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.2.0
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.0
>
> Attachments: HIVE-21404.1.patch, HIVE-21404.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-20221 changes PARTITION_PARAMS, so the following command is modifying
> the wrong table:
> {{ALTER TABLE "SERDE_PARAMS" ALTER COLUMN "PARAM_VALUE" nvarchar(MAX);}}
> https://github.com/apache/hive/blob/d3b036920acde7bb04840697eb13038103b062b4/standalone-metastore/metastore-server/src/main/sql/mssql/upgrade-3.1.0-to-3.2.0.mssql.sql#L21
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21401) Break up DDLTask - extract Table related operations
[
https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790724#comment-16790724
]
Hive QA commented on HIVE-21401:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962127/HIVE-21401.13.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=337)
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=220)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16468/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16468/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16468/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962127 - PreCommit-HIVE-Build
> Break up DDLTask - extract Table related operations
> ---
>
> Key: HIVE-21401
> URL: https://issues.apache.org/jira/browse/HIVE-21401
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 3.1.1
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch,
> HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch,
> HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch,
> HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch,
> HIVE-21401.12.patch, HIVE-21401.13.patch
>
>
> DDLTask is a huge class, more than 5000 lines long. The related DDLWork is
> also a huge class, which has a field for each DDL operation it supports. The
> goal is to refactor these in order to have everything cut into more
> handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each operation
> * have a package for each operation group (database ddl, table ddl, etc), so
> the amount of classes under a package is more manageable
> * make all the requests (DDLDesc subclasses) immutable
> * DDLTask should be agnostic to the actual operations
> * right now let's ignore the issue of having some operations handled by
> DDLTask which are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there are two DDLTask and DDLWork classes in the
> code base the new ones in the new package are called DDLTask2 and DDLWork2
> thus avoiding the usage of fully qualified class names where both the old and
> the new classes are in use.
> Step #2: extract all the table related operations from the old DDLTask except
> alter table, and move them under the new package. Also create the new
> internal framework.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Attachment: HIVE-21421.02.patch > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Status: Patch Available (was: Open) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Attachment: (was: HIVE-21421.02.patch) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Status: Open (was: Patch Available) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21401) Break up DDLTask - extract Table related operations
[
https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790709#comment-16790709
]
Hive QA commented on HIVE-21401:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} contrib in master has 10 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} hcatalog/core in master has 29 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 18 new + 1650 unchanged - 141
fixed = 1668 total (was 1791) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hcatalog/core: The patch generated 1 new + 40
unchanged - 8 fixed = 41 total (was 48) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
24s{color} | {color:red} ql generated 1 new + 2256 unchanged - 2 fixed = 2257
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} contrib in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} hcatalog/core generated 0 new + 28 unchanged - 1
fixed = 28 total (was 29) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}
[jira] [Updated] (HIVE-21368) Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
[
https://issues.apache.org/jira/browse/HIVE-21368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Teddy Choi updated HIVE-21368:
--
Attachment: HIVE-21368.2.patch
> Vectorization: Unnecessary Decimal64 -> HiveDecimal conversion
> --
>
> Key: HIVE-21368
> URL: https://issues.apache.org/jira/browse/HIVE-21368
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-21368.1.patch, HIVE-21368.2.patch
>
>
> Joins projecting Decimal64 have a suspicious cast in the inner loop
> {code}
> ConvertDecimal64ToDecimal(col 14:decimal(7,2)/DECIMAL_64) -> 24:decimal(7,2)'
> {code}
> {code}
> create temporary table foo(x int , y decimal(7,2));
> create temporary table bar(x int , y decimal(7,2));
> set hive.explain.user=false;
> explain vectorization detail select sum(foo.y) from foo, bar where foo.x =
> bar.x;
> {code}
> {code}
> ' Map Join Operator'
> 'condition map:'
> ' Inner Join 0 to 1'
> 'keys:'
> ' 0 _col0 (type: int)'
> ' 1 _col0 (type: int)'
> 'Map Join Vectorization:'
> 'bigTableKeyColumnNums: [0]'
> 'bigTableRetainedColumnNums: [3]'
> 'bigTableValueColumnNums: [3]'
> 'bigTableValueExpressions:
> ConvertDecimal64ToDecimal(col 1:decimal(7,2)/DECIMAL_64) -> 3:decimal(7,2)'
> 'className: VectorMapJoinInnerBigOnlyLongOperator'
> 'native: true'
> 'nativeConditionsMet:
> hive.mapjoin.optimized.hashtable IS true,
> hive.vectorized.execution.mapjoin.native.enabled IS true,
> hive.execution.engine tez IN [tez, spark] IS true, One MapJoin Condition IS
> true, No nullsafe IS true, Small table vectorizes IS true, Fast Hash Table
> and No Hybrid Hash Join IS true'
> 'projectedOutputColumnNums: [3]'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21424) Disable AggregateStatsCache by default
[ https://issues.apache.org/jira/browse/HIVE-21424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790666#comment-16790666 ] Zoltan Haindrich commented on HIVE-21424: - note: aggregatestatscache is highly broken; the variance filter is practically doesn't work because the cached/requested are integers => the division always result in 0 value; and since maxVariance is a floating point .01 ; the condition is never satisified https://github.com/apache/hive/blob/9f2f101feec4283f020b72b918e0961215e76789/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/AggregateStatsCache.java#L224 > Disable AggregateStatsCache by default > -- > > Key: HIVE-21424 > URL: https://issues.apache.org/jira/browse/HIVE-21424 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21424.01.patch, HIVE-21424.02.patch, > HIVE-21424.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21424) Disable AggregateStatsCache by default
[ https://issues.apache.org/jira/browse/HIVE-21424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21424: Attachment: HIVE-21424.03.patch > Disable AggregateStatsCache by default > -- > > Key: HIVE-21424 > URL: https://issues.apache.org/jira/browse/HIVE-21424 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21424.01.patch, HIVE-21424.02.patch, > HIVE-21424.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790631#comment-16790631
]
Hive QA commented on HIVE-15406:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962126/HIVE-15406.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_ppr] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1] (batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_udf_trunc]
(batchId=8)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=230)
org.apache.hive.hcatalog.mapreduce.TestHCatMutableNonPartitioned.testHCatNonPartitionedTable[7]
(batchId=214)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16467/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16467/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16467/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962126 - PreCommit-HIVE-Build
> Consider vectorizing the new 'trunc' function
> -
>
> Key: HIVE-15406
> URL: https://issues.apache.org/jira/browse/HIVE-15406
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 4.0.0
>Reporter: Matt McCline
>Assignee: Laszlo Bodor
>Priority: Critical
> Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch
>
>
> Rounding function 'trunc' added by HIVE-14582.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: HIVE-15406.02.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch, HIVE-15406.02.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[
https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790583#comment-16790583
]
Hive QA commented on HIVE-15406:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
22s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 16 new + 4 unchanged - 1 fixed
= 20 total (was 5) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
24s{color} | {color:red} ql generated 6 new + 2258 unchanged - 0 fixed = 2264
total (was 2258) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 7 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromDate
defines non-transient non-serializable instance field date In
TruncDateFromDate.java:instance field date In TruncDateFromDate.java |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromString defines
non-transient non-serializable instance field date In
TruncDateFromString.java:instance field date In TruncDateFromString.java |
| | Found reliance on default encoding in new
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp(int,
byte[], int):in new
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp(int,
byte[], int): new String(byte[]) At TruncDateFromTimestamp.java:[line 33] |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp.processDate(BytesColumnVector,
int, Date):in
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncDateFromTimestamp.processDate(BytesColumnVector,
int, Date): String.getBytes() At TruncDateFromTimestamp.java:[line 145] |
| | Field TruncDecimalNoScale.colNum masks field in superclass
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncFloat In
TruncDecimalNoScale.java:superclass
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncFloat In
TruncDecimalNoScale.java |
| | Field TruncFloatNoScale.colNum masks field in superclass
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncFloat In
TruncFloatNoScale.java:superclass
org.apache.hadoop.hive.ql.exec.vector.expressions.TruncFloat In
TruncFloatNoScale.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16467/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16467/yetus/diff-checkstyle-ql.txt
|
| whitespace |
[jira] [Updated] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
[
https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-20580:
--
Attachment: HIVE-20580.4.patch
> OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
> --
>
> Key: HIVE-20580
> URL: https://issues.apache.org/jira/browse/HIVE-20580
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20580.2.patch, HIVE-20580.3.patch,
> HIVE-20580.4.patch, HIVE-20580.patch
>
>
> {{org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isOriginal()}} is checking
> for presence of {{hive.acid.key.index}} in the footer. This is only created
> when the file is written by {{OrcRecordUpdater}}. It should instead check
> for presence of Acid metadata columns so that a file can be produced by
> something other than {{OrcRecordUpater}}.
> Also, {{hive.acid.key.index}} counts number of different type of events which
> is not really useful for Acid V2 (as of Hive 3) since each file only has 1
> type of event.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[
https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790552#comment-16790552
]
Hive QA commented on HIVE-21421:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962114/HIVE-21421.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15824 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[test_teradatabinaryfile]
(batchId=2)
org.apache.hive.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable[1]
(batchId=209)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16466/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16466/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16466/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962114 - PreCommit-HIVE-Build
> HiveStatement.getQueryId throws NPE when query is not running.
> ---
>
> Key: HIVE-21421
> URL: https://issues.apache.org/jira/browse/HIVE-21421
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HiveStatement.getQueryId throws NullPointerException if it invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by not set utf-8 character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() example: executing sql: hive> select id from db.table1 where id != '你好'; show locks sql: hive> show locks db.table1 extended; OK db@table1 SHARED LOCK_QUERYID:hive_2019031221_05425a80-c6f7-43e3-a784-ad690b469b4d LOCK_TIME:1552396936424 LOCK_MODE:IMPLICIT LOCK_QUERYSTRING:select id from db.table1 where id != '`}' Time taken: 0.039 seconds, Fetched: 5 row(s) was: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by not set utf-8 character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() example: executing sql: hive> select id from db.table1 – 搜索 group by id; show locks sql: hive> show locks db.table1 extended; OK db@table1 SHARED LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 LOCK_TIME:1552384291462 LOCK_MODE:IMPLICIT LOCK_QUERYSTRING:select id from db.table1 group by id – " * Time taken: 0.028 seconds, Fetched: 10 row(s) > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when the show locks sql contains chinese character, the result will show > unreadable codes, this is caused by not set utf-8 character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() > > example: > executing sql: > hive> select id from db.table1 where id != '你好'; > show locks sql: > hive> show locks db.table1 extended; > OK > db@table1 SHARED > LOCK_QUERYID:hive_2019031221_05425a80-c6f7-43e3-a784-ad690b469b4d > LOCK_TIME:1552396936424 > LOCK_MODE:IMPLICIT > LOCK_QUERYSTRING:select id from db.table1 where id != '`}' > Time taken: 0.039 seconds, Fetched: 5 row(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Attachment: HIVE-21432.patch > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > Attachments: HIVE-21432.patch > > > when the show locks sql contains chinese character, the result will show > unreadable codes, this is caused by not set utf-8 character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() > > example: > executing sql: > hive> select id from db.table1 where id != '你好'; > show locks sql: > hive> show locks db.table1 extended; > OK > db@table1 SHARED > LOCK_QUERYID:hive_2019031221_05425a80-c6f7-43e3-a784-ad690b469b4d > LOCK_TIME:1552396936424 > LOCK_MODE:IMPLICIT > LOCK_QUERYSTRING:select id from db.table1 where id != '`}' > Time taken: 0.039 seconds, Fetched: 5 row(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21426) Remove Utilities Global Random
[
https://issues.apache.org/jira/browse/HIVE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21426:
--
Status: Patch Available (was: Open)
> Remove Utilities Global Random
> --
>
> Key: HIVE-21426
> URL: https://issues.apache.org/jira/browse/HIVE-21426
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21426.1.patch, HIVE-21426.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L253
> Remove global {{Random}} object in favor of {{ThreadLocalRandom}}.
> {quote}
> ThreadLocalRandom is initialized with an internally generated seed that may
> not otherwise be modified. When applicable, use of ThreadLocalRandom rather
> than shared Random objects in concurrent programs will typically encounter
> much less overhead and contention.
> {quote}
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadLocalRandom.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21426) Remove Utilities Global Random
[
https://issues.apache.org/jira/browse/HIVE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21426:
--
Attachment: HIVE-21426.1.patch
> Remove Utilities Global Random
> --
>
> Key: HIVE-21426
> URL: https://issues.apache.org/jira/browse/HIVE-21426
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21426.1.patch, HIVE-21426.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L253
> Remove global {{Random}} object in favor of {{ThreadLocalRandom}}.
> {quote}
> ThreadLocalRandom is initialized with an internally generated seed that may
> not otherwise be modified. When applicable, use of ThreadLocalRandom rather
> than shared Random objects in concurrent programs will typically encounter
> much less overhead and contention.
> {quote}
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadLocalRandom.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by not set utf-8 character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() example: executing sql: hive> select id from db.table1 – 搜索 group by id; show locks sql: hive> show locks db.table1 extended; OK db@table1 SHARED LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 LOCK_TIME:1552384291462 LOCK_MODE:IMPLICIT LOCK_QUERYSTRING:select id from db.table1 group by id – " * Time taken: 0.028 seconds, Fetched: 10 row(s) was: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() example: executing sql: hive> select id from db.table1 -- 搜索 group by id; show locks sql: hive> show locks db.table1 extended; OK db@table1 SHARED LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 LOCK_TIME:1552384291462 LOCK_MODE:IMPLICIT LOCK_QUERYSTRING:select id from db.table1 group by id -- " * Time taken: 0.028 seconds, Fetched: 10 row(s) > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when the show locks sql contains chinese character, the result will show > unreadable codes, this is caused by not set utf-8 character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() > > example: > executing sql: > hive> select id from db.table1 – 搜索 > group by id; > show locks sql: > hive> show locks db.table1 extended; > OK > db@table1 SHARED > LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 > LOCK_TIME:1552384291462 > LOCK_MODE:IMPLICIT > LOCK_QUERYSTRING:select id from db.table1 group by id – " * > Time taken: 0.028 seconds, Fetched: 10 row(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Component/s: Vectorization > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Affects Version/s: 4.0.0 > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: (was: HIVE-15406.01.patch) > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21425) Use DirectExecutorService for getInputSummary
[
https://issues.apache.org/jira/browse/HIVE-21425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21425:
--
Status: Open (was: Patch Available)
> Use DirectExecutorService for getInputSummary
> -
>
> Key: HIVE-21425
> URL: https://issues.apache.org/jira/browse/HIVE-21425
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21425.1.patch, HIVE-21425.1.patch,
> HIVE-21425.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|title=Utilities.java}
> int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
> pathNeedProcess.size());
> if (numExecutors > 1) {
> LOG.info("Using {} threads for getContentSummary", numExecutors);
> executor = Executors.newFixedThreadPool(numExecutors,
> new ThreadFactoryBuilder().setDaemon(true)
> .setNameFormat("Get-Input-Summary-%d").build());
> } else {
> executor = null;
> }
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2482-L2490
> Instead of using a 'null' {{ExecutorService}}, use Guava's
> {{DirectExecutorService}} and remove special casing for a 'null' value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21426) Remove Utilities Global Random
[
https://issues.apache.org/jira/browse/HIVE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21426:
--
Status: Open (was: Patch Available)
> Remove Utilities Global Random
> --
>
> Key: HIVE-21426
> URL: https://issues.apache.org/jira/browse/HIVE-21426
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21426.1.patch, HIVE-21426.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L253
> Remove global {{Random}} object in favor of {{ThreadLocalRandom}}.
> {quote}
> ThreadLocalRandom is initialized with an internally generated seed that may
> not otherwise be modified. When applicable, use of ThreadLocalRandom rather
> than shared Random objects in concurrent programs will typically encounter
> much less overhead and contention.
> {quote}
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadLocalRandom.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21425) Use DirectExecutorService for getInputSummary
[
https://issues.apache.org/jira/browse/HIVE-21425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21425:
--
Attachment: HIVE-21425.1.patch
> Use DirectExecutorService for getInputSummary
> -
>
> Key: HIVE-21425
> URL: https://issues.apache.org/jira/browse/HIVE-21425
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21425.1.patch, HIVE-21425.1.patch,
> HIVE-21425.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|title=Utilities.java}
> int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
> pathNeedProcess.size());
> if (numExecutors > 1) {
> LOG.info("Using {} threads for getContentSummary", numExecutors);
> executor = Executors.newFixedThreadPool(numExecutors,
> new ThreadFactoryBuilder().setDaemon(true)
> .setNameFormat("Get-Input-Summary-%d").build());
> } else {
> executor = null;
> }
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2482-L2490
> Instead of using a 'null' {{ExecutorService}}, use Guava's
> {{DirectExecutorService}} and remove special casing for a 'null' value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21425) Use DirectExecutorService for getInputSummary
[
https://issues.apache.org/jira/browse/HIVE-21425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21425:
--
Status: Patch Available (was: Open)
> Use DirectExecutorService for getInputSummary
> -
>
> Key: HIVE-21425
> URL: https://issues.apache.org/jira/browse/HIVE-21425
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21425.1.patch, HIVE-21425.1.patch,
> HIVE-21425.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java|title=Utilities.java}
> int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(),
> pathNeedProcess.size());
> if (numExecutors > 1) {
> LOG.info("Using {} threads for getContentSummary", numExecutors);
> executor = Executors.newFixedThreadPool(numExecutors,
> new ThreadFactoryBuilder().setDaemon(true)
> .setNameFormat("Get-Input-Summary-%d").build());
> } else {
> executor = null;
> }
> {code}
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2482-L2490
> Instead of using a 'null' {{ExecutorService}}, use Guava's
> {{DirectExecutorService}} and remove special casing for a 'null' value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Patch Available (was: Open) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Attachment: HIVE-21401.13.patch > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch, HIVE-21401.13.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790529#comment-16790529 ] Laszlo Bodor commented on HIVE-15406: - Could someone take a look at this patch? https://reviews.apache.org/r/70193/ [~teddy.choi]: I've marked you as a reviewer on review board, as it's vectorization related...could you take a look or give and advice who could review this? cc: [~ashutoshc] > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() example: executing sql: hive> select id from db.table1 -- 搜索 group by id; show locks sql: hive> show locks db.table1 extended; OK db@table1 SHARED LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 LOCK_TIME:1552384291462 LOCK_MODE:IMPLICIT LOCK_QUERYSTRING:select id from db.table1 group by id -- " * Time taken: 0.028 seconds, Fetched: 10 row(s) was:when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when the show locks sql contains chinese character, the result will show > unreadable codes, this is caused by wrong character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() > > example: > executing sql: > hive> select id from db.table1 -- 搜索 > group by id; > show locks sql: > hive> show locks db.table1 extended; > OK > db@table1 SHARED > LOCK_QUERYID:hive_20190312175151_3df05c4b-0402-459b-ac82-67b15ac45d57 > LOCK_TIME:1552384291462 > LOCK_MODE:IMPLICIT > LOCK_QUERYSTRING:select id from db.table1 group by id -- " * > Time taken: 0.028 seconds, Fetched: 10 row(s) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when sql contains chinese character, the result of show locks sql has > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when sql contains chinese character, the result of show locks sql has -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21401) Break up DDLTask - extract Table related operations
[ https://issues.apache.org/jira/browse/HIVE-21401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21401: -- Status: Open (was: Patch Available) > Break up DDLTask - extract Table related operations > --- > > Key: HIVE-21401 > URL: https://issues.apache.org/jira/browse/HIVE-21401 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 3.1.1 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21401.01.patch, HIVE-21401.02.patch, > HIVE-21401.03.patch, HIVE-21401.04.patch, HIVE-21401.05.patch, > HIVE-21401.06.patch, HIVE-21401.07.patch, HIVE-21401.08.patch, > HIVE-21401.09.patch, HIVE-21401.10.patch, HIVE-21401.11.patch, > HIVE-21401.12.patch > > > DDLTask is a huge class, more than 5000 lines long. The related DDLWork is > also a huge class, which has a field for each DDL operation it supports. The > goal is to refactor these in order to have everything cut into more > handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each operation > * have a package for each operation group (database ddl, table ddl, etc), so > the amount of classes under a package is more manageable > * make all the requests (DDLDesc subclasses) immutable > * DDLTask should be agnostic to the actual operations > * right now let's ignore the issue of having some operations handled by > DDLTask which are not actual DDL operations (lock, unlock, desc...) > In the interim time when there are two DDLTask and DDLWork classes in the > code base the new ones in the new package are called DDLTask2 and DDLWork2 > thus avoiding the usage of fully qualified class names where both the old and > the new classes are in use. > Step #2: extract all the table related operations from the old DDLTask except > alter table, and move them under the new package. Also create the new > internal framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Status: Patch Available (was: In Progress) > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: HIVE-15406.01.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when the show locks sql contains chinese character, the result will show unreadable codes, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() (was: when sql contains chinese character, the result of show locks sql has unreadable code, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks()) > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when the show locks sql contains chinese character, the result will show > unreadable codes, this is caused by wrong character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when sql contains chinese character, the result of show locks sql has unreadable code, this is caused by wrong character encoding in org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() (was: when sql contains chinese character, the result of show locks sql has unreadable code, this is caused by ) > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when sql contains chinese character, the result of show locks sql has > unreadable code, this is caused by wrong character encoding in > org.apache.hadoop.hive.ql.exec.DDLTask.showLocks() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21432) unreadable code in show locks sql result caused by wrong character encoding
[ https://issues.apache.org/jira/browse/HIVE-21432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yx91490 updated HIVE-21432: --- Description: when sql contains chinese character, the result of show locks sql has unreadable code, this is caused by (was: when sql contains chinese character, the result of show locks sql has ) > unreadable code in show locks sql result caused by wrong character encoding > > > Key: HIVE-21432 > URL: https://issues.apache.org/jira/browse/HIVE-21432 > Project: Hive > Issue Type: Bug >Reporter: yx91490 >Priority: Major > > when sql contains chinese character, the result of show locks sql has > unreadable code, this is caused by -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17061) Add Support for Column List in Insert Clause
[
https://issues.apache.org/jira/browse/HIVE-17061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790519#comment-16790519
]
Shawn Weeks commented on HIVE-17061:
Looks like HIVE-20590 may have been a duplicate of this.
> Add Support for Column List in Insert Clause
>
>
> Key: HIVE-17061
> URL: https://issues.apache.org/jira/browse/HIVE-17061
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Shawn Weeks
>Priority: Minor
>
> Include support for a list of columns in the insert clause of the merge
> statement. This helps when you may not know or care about the order of
> columns in the target table or if you don't want to have to insert values
> into all of the columns.
> {code}
> MERGE INTO target
> USING source ON b = y
> WHEN MATCHED AND c + 1 + z > 0
> THEN UPDATE SET a = 1, c = z
> WHEN NOT MATCHED AND z IS NULL
> THEN INSERT(a,b) VALUES(z, 7)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20590) Allow merge statement to have column schema
[ https://issues.apache.org/jira/browse/HIVE-20590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790517#comment-16790517 ] Shawn Weeks commented on HIVE-20590: Isn't this a duplicate of HIVE-17061? > Allow merge statement to have column schema > --- > > Key: HIVE-20590 > URL: https://issues.apache.org/jira/browse/HIVE-20590 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Vineet Garg >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20590.01.patch, HIVE-20590.03.patch, > HIVE-20590.04.patch, HIVE-20590.05.patch, HIVE-20590.06.patch > > > Currently MERGE statement doesn't let user specify column schema with INSERT > statements, therefore DEFAULT constraint are not applicable with it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[
https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790512#comment-16790512
]
Hive QA commented on HIVE-21421:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} jdbc: The patch generated 0 new + 5 unchanged - 1
fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16466/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: jdbc itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16466/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveStatement.getQueryId throws NPE when query is not running.
> ---
>
> Key: HIVE-21421
> URL: https://issues.apache.org/jira/browse/HIVE-21421
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HiveStatement.getQueryId throws NullPointerException if it invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
[
https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790484#comment-16790484
]
Hive QA commented on HIVE-20580:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12962111/HIVE-20580.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 15825 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=230)
org.apache.hadoop.hive.ql.TestAcidOnTez.testIsOriginal (batchId=241)
org.apache.hadoop.hive.ql.io.orc.TestOrcRawRecordMerger.testNewBase
(batchId=301)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/16465/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/16465/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-16465/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12962111 - PreCommit-HIVE-Build
> OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
> --
>
> Key: HIVE-20580
> URL: https://issues.apache.org/jira/browse/HIVE-20580
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20580.2.patch, HIVE-20580.3.patch, HIVE-20580.patch
>
>
> {{org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isOriginal()}} is checking
> for presence of {{hive.acid.key.index}} in the footer. This is only created
> when the file is written by {{OrcRecordUpdater}}. It should instead check
> for presence of Acid metadata columns so that a file can be produced by
> something other than {{OrcRecordUpater}}.
> Also, {{hive.acid.key.index}} counts number of different type of events which
> is not really useful for Acid V2 (as of Hive 3) since each file only has 1
> type of event.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21316) Comparision of varchar column and string literal should happen in varchar
[ https://issues.apache.org/jira/browse/HIVE-21316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790467#comment-16790467 ] Zoltan Haindrich commented on HIVE-21316: - [~ashutoshc] opened rb: https://reviews.apache.org/r/70190/ > Comparision of varchar column and string literal should happen in varchar > - > > Key: HIVE-21316 > URL: https://issues.apache.org/jira/browse/HIVE-21316 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21316.01.patch, HIVE-21316.02.patch, > HIVE-21316.03.patch > > > this is most probably the root cause behind HIVE-21310 as well -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: HIVE-15406.01.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: HIVE-15406.01.patch > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15406) Consider vectorizing the new 'trunc' function
[ https://issues.apache.org/jira/browse/HIVE-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-15406: Attachment: (was: HIVE-15406.01.patch) > Consider vectorizing the new 'trunc' function > - > > Key: HIVE-15406 > URL: https://issues.apache.org/jira/browse/HIVE-15406 > Project: Hive > Issue Type: Bug >Reporter: Matt McCline >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-15406.01.patch > > > Rounding function 'trunc' added by HIVE-14582. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20580) OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
[
https://issues.apache.org/jira/browse/HIVE-20580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790466#comment-16790466
]
Hive QA commented on HIVE-20580:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2258 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 12 new + 327 unchanged - 1
fixed = 339 total (was 328) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 181
unchanged - 0 fixed = 183 total (was 181) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-16465/dev-support/hive-personality.sh
|
| git revision | master / 9f2f101 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16465/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16465/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-16465/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> OrcInputFormat.isOriginal() should not rely on hive.acid.key.index
> --
>
> Key: HIVE-20580
> URL: https://issues.apache.org/jira/browse/HIVE-20580
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20580.2.patch, HIVE-20580.3.patch, HIVE-20580.patch
>
>
> {{org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.isOriginal()}} is checking
> for presence of {{hive.acid.key.index}} in the footer. This is only created
> when the file is written by
[jira] [Commented] (HIVE-14430) More instances of HiveConf and the associated UDFClassLoader than expected
[ https://issues.apache.org/jira/browse/HIVE-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790464#comment-16790464 ] chenjianxun commented on HIVE-14430: is there any progess for this issue? i have got the same problem, the instances of "org.apache.hadoop.hive.conf.HiveConf" keep growing. and the dump file of hiveserver2 shows that there is a memory leak problem i am using hive 2.3.4 and hadoop-2.6.0-cdh5.16.1 Leak Suspects 1,041 instances of *"org.apache.hadoop.hive.conf.HiveConf"*, loaded by *"sun.misc.Launcher$AppClassLoader @ 0xeaaa0058"* occupy *270,268,056 (52.70%)* bytes. These instances are referenced from one instance of *"java.util.HashMap$Node[]"*, loaded by *""* > More instances of HiveConf and the associated UDFClassLoader than expected > -- > > Key: HIVE-14430 > URL: https://issues.apache.org/jira/browse/HIVE-14430 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2 >Reporter: Siddharth Seth >Assignee: Vaibhav Gumashta >Priority: Critical > > 841 instances of HiveConf. > 831 instances of UDFClassLoader > This is on a HS2 instance configured to run 10 concurrent queries with LLAP. > 10 SessionState instances. Something is holding on to the additional > HiveConf, UDFClassLoaders - potentially HMSHandler. > This is with an embedded metastore. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21411) LEFT JOIN CONVERT TO INNER JOIN LEAD TO WRONG RESULT
[
https://issues.apache.org/jira/browse/HIVE-21411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790449#comment-16790449
]
xialu commented on HIVE-21411:
--
set hive.auto.convert.join=false;
> LEFT JOIN CONVERT TO INNER JOIN LEAD TO WRONG RESULT
>
>
> Key: HIVE-21411
> URL: https://issues.apache.org/jira/browse/HIVE-21411
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.1, 2.2.0, 2.3.0
>Reporter: xialu
>Assignee: Ashutosh Chauhan
>Priority: Critical
>
> when i have not assign alias table name to the left side table , left join
> convert to inner join. left side table's alias in ast tree called left.
> for example
> select nvl(ss_wholesale_cost, 10), d_quarter_name from lulu.store_sales left
> join lulu.date_dim on ss_sold_date_sk = d_date_sk limit 10;
> {noformat}
> | STAGE DEPENDENCIES: |
> | Stage-1 is a root stage |
> | Stage-0 depends on stages: Stage-1 |
> | |
> | STAGE PLANS: |
> | Stage: Stage-1 |
> | Map Reduce |
> | Map Operator Tree: |
> | TableScan |
> | alias: left |
> | Statistics: Num rows: 200 Data size: 28499 Basic stats: COMPLETE Column
> stats: NONE |
> | Filter Operator |
> | predicate: ss_sold_date_sk is not null (type: boolean) |
> | Statistics: Num rows: 200 Data size: 28499 Basic stats: COMPLETE Column
> stats: NONE |
> | Select Operator |
> | expressions: ss_wholesale_cost (type: decimal(7,2)), ss_sold_date_sk (type:
> bigint) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 200 Data size: 28499 Basic stats: COMPLETE Column
> stats: NONE |
> | Reduce Output Operator |
> | key expressions: _col1 (type: bigint) |
> | sort order: + |
> | Map-reduce partition columns: _col1 (type: bigint) |
> | Statistics: Num rows: 200 Data size: 28499 Basic stats: COMPLETE Column
> stats: NONE |
> | value expressions: _col0 (type: decimal(7,2)) |
> | TableScan |
> | alias: date_dim |
> | Statistics: Num rows: 200 Data size: 25639 Basic stats: COMPLETE Column
> stats: NONE |
> | Filter Operator |
> | predicate: d_date_sk is not null (type: boolean) |
> | Statistics: Num rows: 200 Data size: 25639 Basic stats: COMPLETE Column
> stats: NONE |
> | Select Operator |
> | expressions: d_date_sk (type: bigint), d_quarter_name (type: string) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 200 Data size: 25639 Basic stats: COMPLETE Column
> stats: NONE |
> | Reduce Output Operator |
> | key expressions: _col0 (type: bigint) |
> | sort order: + |
> | Map-reduce partition columns: _col0 (type: bigint) |
> | Statistics: Num rows: 200 Data size: 25639 Basic stats: COMPLETE Column
> stats: NONE |
> | value expressions: _col1 (type: string) |
> | Reduce Operator Tree: |
> | Join Operator |
> | condition map: |
> | Inner Join 0 to 1 |
> | keys: |
> | 0 _col1 (type: bigint) |
> | 1 _col0 (type: bigint) |
> | outputColumnNames: _col0, _col3 |
> | Statistics: Num rows: 220 Data size: 31348 Basic stats: COMPLETE Column
> stats: NONE |
> | Select Operator |
> | expressions: NVL(_col0,10) (type: decimal(12,2)), _col3 (type: string) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 220 Data size: 31348 Basic stats: COMPLETE Column
> stats: NONE |
> | Limit |
> | Number of rows: 100 |
> | Statistics: Num rows: 100 Data size: 14200 Basic stats: COMPLETE Column
> stats: NONE |
> | File Output Operator |
> | compressed: false |
> | Statistics: Num rows: 100 Data size: 14200 Basic stats: COMPLETE Column
> stats: NONE |
> | table: |
> | input format: org.apache.hadoop.mapred.SequenceFileInputFormat |
> | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat |
> | serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe |
> | |
> | Stage: Stage-0 |
> | Fetch Operator |
> | limit: 100 |
> | Processor Tree: |
> | ListSink |
> | |
> ++--+
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Attachment: (was: HIVE-21421.02.patch) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Status: Open (was: Patch Available) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21421) HiveStatement.getQueryId throws NPE when query is not running.
[ https://issues.apache.org/jira/browse/HIVE-21421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21421: Status: Patch Available (was: Open) > HiveStatement.getQueryId throws NPE when query is not running. > --- > > Key: HIVE-21421 > URL: https://issues.apache.org/jira/browse/HIVE-21421 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21421.01.patch, HIVE-21421.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > HiveStatement.getQueryId throws NullPointerException if it invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
