[jira] [Updated] (HIVE-16976) DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
[
https://issues.apache.org/jira/browse/HIVE-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal updated HIVE-16976:
--
Attachment: HIVE-16976.11.patxh
> DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
>
>
> Key: HIVE-16976
> URL: https://issues.apache.org/jira/browse/HIVE-16976
> Project: Hive
> Issue Type: Improvement
> Components: Tez
>Affects Versions: 2.1.1, 3.0.0
>Reporter: Gopal V
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-16976.1.patch, HIVE-16976.10.patch,
> HIVE-16976.11.patxh, HIVE-16976.2.patch, HIVE-16976.3.patch,
> HIVE-16976.4.patch, HIVE-16976.5.patch, HIVE-16976.6.patch,
> HIVE-16976.7.patch, HIVE-16976.8.patch, HIVE-16976.9.patch
>
>
> Tez DPP does not kick in for scenarios where a user wants to run a comparison
> clause instead of a JOIN/IN clause.
> {code}
> explain select count(1) from store_sales where ss_sold_date_sk > (select
> max(d_Date_sk) from date_dim where d_year = 2017);
> Warning: Map Join MAPJOIN[21][bigTable=?] in task 'Map 1' is a cross product
> OK
> Plan optimized by CBO.
> Vertex dependency in root stage
> Map 1 <- Reducer 4 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
> Stage-0
> Fetch Operator
> limit:-1
> Stage-1
> Reducer 2 vectorized, llap
> File Output Operator [FS_36]
> Group By Operator [GBY_35] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(VALUE._col0)"]
> <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_34]
> Group By Operator [GBY_33] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(1)"]
> Select Operator [SEL_32] (rows=9600142089 width=16)
> Filter Operator [FIL_31] (rows=9600142089 width=16)
> predicate:(_col0 > _col1)
> Map Join Operator [MAPJOIN_30] (rows=28800426268 width=16)
> Conds:(Inner),Output:["_col0","_col1"]
> <-Reducer 4 [BROADCAST_EDGE] vectorized, llap
> BROADCAST [RS_28]
> Group By Operator [GBY_27] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(VALUE._col0)"]
> <-Map 3 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_26]
> Group By Operator [GBY_25] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(d_date_sk)"]
> Select Operator [SEL_24] (rows=652 width=12)
> Output:["d_date_sk"]
> Filter Operator [FIL_23] (rows=652 width=12)
> predicate:(d_year = 2017)
> TableScan [TS_2] (rows=73049 width=12)
>
> tpcds_bin_partitioned_newschema_orc_1@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
> <-Select Operator [SEL_29] (rows=28800426268 width=8)
> Output:["_col0"]
> TableScan [TS_0] (rows=28800426268 width=172)
>
> tpcds_bin_partitioned_newschema_orc_1@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE
> {code}
> The SyntheticJoinPredicate is only injected for equi joins, not for < or >
> scalar subqueries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21103: - Attachment: HIVE-21103.2.patch > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-21103.1.patch, HIVE-21103.2.patch > > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[
https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737916#comment-16737916
]
Hive QA commented on HIVE-21082:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} hplsql in master has 176 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} hplsql: The patch generated 3 new + 271 unchanged - 2
fixed = 274 total (was 273) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15557/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15557/yetus/diff-checkstyle-hplsql.txt
|
| modules | C: hplsql U: hplsql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15557/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> In HPL/SQL, declare statement does not support variable of type character
> --
>
> Key: HIVE-21082
> URL: https://issues.apache.org/jira/browse/HIVE-21082
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Baoning He
>Assignee: Baoning He
>Priority: Major
> Attachments: HIVE-21082.1.patch
>
>
> In the following HPL/SQL programs:
> DECLARE a character(5); SET a = 'b';
> when the type of variable 'a' is CHARACTER, it cannot be assigned a value
> successfully. The support for the character type should be added to DECLARE
> statement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737899#comment-16737899
]
Hive QA commented on HIVE-20170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954256/HIVE-20170.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15556/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15556/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15556/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954256 - PreCommit-HIVE-Build
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch, HIVE-20170.8.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16907) "INSERT INTO" overwrite old data when destination table encapsulated by backquote
[
https://issues.apache.org/jira/browse/HIVE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737901#comment-16737901
]
Zoltan Haindrich commented on HIVE-16907:
-
[~vgarg], [~jcamachorodriguez]: Could you please take a look?
> "INSERT INTO" overwrite old data when destination table encapsulated by
> backquote
>
>
> Key: HIVE-16907
> URL: https://issues.apache.org/jira/browse/HIVE-16907
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 1.1.0, 2.1.1
>Reporter: Nemon Lou
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-16907.02.patch, HIVE-16907.03.patch,
> HIVE-16907.03.patch, HIVE-16907.03.patch, HIVE-16907.03.patch,
> HIVE-16907.1.patch
>
>
> A way to reproduce:
> {noformat}
> create database tdb;
> use tdb;
> create table t1(id int);
> create table t2(id int);
> explain insert into `tdb.t1` select * from t2;
> {noformat}
> {noformat}
> +---+
> |
> Explain |
> +---+
> | STAGE DEPENDENCIES:
> |
> | Stage-1 is a root stage
> |
> | Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2,
> Stage-4 |
> | Stage-3
> |
> | Stage-0 depends on stages: Stage-3, Stage-2, Stage-5
> |
> | Stage-2
> |
> | Stage-4
> |
> | Stage-5 depends on stages: Stage-4
> |
> |
> |
> | STAGE PLANS:
> |
> | Stage: Stage-1
> |
> | Map Reduce
> |
> | Map Operator Tree:
> |
> | TableScan
> |
> | alias: t2
> |
> | Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column
> stats: NONE |
> | Select Operator
> |
> | expressions: id (type: int)
> |
> | outputColumnNames: _col0
> |
> | Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column
> stats: NONE |
> | File Output Operator
> |
> | compressed: false
>
[jira] [Work started] (HIVE-8346) MapRedLocalTask Error Handling
[
https://issues.apache.org/jira/browse/HIVE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-8346 started by Mani M.
> MapRedLocalTask Error Handling
> --
>
> Key: HIVE-8346
> URL: https://issues.apache.org/jira/browse/HIVE-8346
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 0.14.0
>Reporter: Szehon Ho
>Assignee: Mani M
>Priority: Minor
> Labels: newbie
> Attachments: image-2019-01-09-17-31-15-247.png
>
>
> If there are any exceptions trying to fork a local task, the exception
> message is logged but not the stack trace. There can be a lot of issues
> forking a process, so we should log the stack trace for better debuggability.
> Code in MapRedLocalTask.executeInChildJVM(DriverContext ctx):
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> LOG.error("Exception: " + e.getMessage());
> return (1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-8346) MapRedLocalTask Error Handling
[
https://issues.apache.org/jira/browse/HIVE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737895#comment-16737895
]
Mani M commented on HIVE-8346:
--
HI [~pvary],
As of the ticket is raised only the error message was written in the log
Component. But after subsequent fix this had been changed to include the full
exception in the log file
changed as the part of HIVE-9860
and subsequently changed in HIVE-13183
!image-2019-01-09-17-31-15-247.png!
Screenshot is attached herewith.
So, I think, this Jira ticket is no-longer valid, we can close this.
> MapRedLocalTask Error Handling
> --
>
> Key: HIVE-8346
> URL: https://issues.apache.org/jira/browse/HIVE-8346
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 0.14.0
>Reporter: Szehon Ho
>Assignee: Mani M
>Priority: Minor
> Labels: newbie
> Attachments: image-2019-01-09-17-31-15-247.png
>
>
> If there are any exceptions trying to fork a local task, the exception
> message is logged but not the stack trace. There can be a lot of issues
> forking a process, so we should log the stack trace for better debuggability.
> Code in MapRedLocalTask.executeInChildJVM(DriverContext ctx):
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> LOG.error("Exception: " + e.getMessage());
> return (1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-8346) MapRedLocalTask Error Handling
[
https://issues.apache.org/jira/browse/HIVE-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-8346:
-
Attachment: image-2019-01-09-17-31-15-247.png
> MapRedLocalTask Error Handling
> --
>
> Key: HIVE-8346
> URL: https://issues.apache.org/jira/browse/HIVE-8346
> Project: Hive
> Issue Type: Bug
> Components: Logging
>Affects Versions: 0.14.0
>Reporter: Szehon Ho
>Assignee: Mani M
>Priority: Minor
> Labels: newbie
> Attachments: image-2019-01-09-17-31-15-247.png
>
>
> If there are any exceptions trying to fork a local task, the exception
> message is logged but not the stack trace. There can be a lot of issues
> forking a process, so we should log the stack trace for better debuggability.
> Code in MapRedLocalTask.executeInChildJVM(DriverContext ctx):
> {code}
> } catch (Exception e) {
> e.printStackTrace();
> LOG.error("Exception: " + e.getMessage());
> return (1);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Affects Version/s: 2.1.1
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Attachment: HIVE-2.1.patch
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737878#comment-16737878 ] Daniel Dai commented on HIVE-21082: --- +1 pending test. > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21082.1.patch > > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Component/s: Physical Optimizer
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei reassigned HIVE-2:
-
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-21099:
--
Release Note: Do Not Print StackTraces to STDERR in
ConditionalResolverMergeFiles
Status: Patch Available (was: In Progress)
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737879#comment-16737879 ] Baoning He commented on HIVE-21082: --- [~daijy] Thank you for reminding, I have attached a new patch with a test. > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21082.1.patch > > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-21099:
--
Attachment: HIVE.21099.patch
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737874#comment-16737874
]
Hive QA commented on HIVE-20170:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15556/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15556/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch, HIVE-20170.8.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21099 started by Mani M.
-
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoning He updated HIVE-21082: -- Attachment: (was: HIVE-21082.patch) > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoning He updated HIVE-21082: -- Attachment: HIVE-21082.1.patch Status: Patch Available (was: In Progress) > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21082.1.patch > > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoning He updated HIVE-21082: -- Status: In Progress (was: Patch Available) > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[
https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737866#comment-16737866
]
Hive QA commented on HIVE-21107:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954252/HIVE-21107.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=307)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/1/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954252 - PreCommit-HIVE-Build
> Cannot find field" error during dynamically partitioned hash join
> -
>
> Key: HIVE-21107
> URL: https://issues.apache.org/jira/browse/HIVE-21107
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21107.1.patch
>
>
> This occurs in non-CBO path with dynamic partitioned join + constant
> propagation ON.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737863#comment-16737863 ] Sankar Hariappan commented on HIVE-21103: - +1 > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-21103.1.patch > > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[
https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737844#comment-16737844
]
Hive QA commented on HIVE-21107:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 48 unchanged - 0 fixed
= 49 total (was 48) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-1/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-1/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-1/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Cannot find field" error during dynamically partitioned hash join
> -
>
> Key: HIVE-21107
> URL: https://issues.apache.org/jira/browse/HIVE-21107
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-21107.1.patch
>
>
> This occurs in non-CBO path with dynamic partitioned join + constant
> propagation ON.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21104) PTF with nested structure throws ClassCastException
[
https://issues.apache.org/jira/browse/HIVE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737846#comment-16737846
]
Rajesh Balamohan commented on HIVE-21104:
-
When GenericUDAFRank is initialized, it inits correctly with input as
"org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector"
output as
"org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaIntObjectInspector".
But somewhere in the processing, it is changed to "LazyBinaryStruct"
> PTF with nested structure throws ClassCastException
> ---
>
> Key: HIVE-21104
> URL: https://issues.apache.org/jira/browse/HIVE-21104
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Major
>
> {noformat}
> DROP TABLE IF EXISTS dummy;
> CREATE TABLE dummy (i int);
> INSERT INTO TABLE dummy VALUES (1);
> DROP TABLE IF EXISTS struct_table_example;
> CREATE TABLE struct_table_example (a int, s1 struct f3: int, f4: int> ) STORED AS ORC;
> INSERT INTO TABLE struct_table_example SELECT 1, named_struct('f1', false,
> 'f2', 'test', 'f3', 3, 'f4', 4) FROM dummy;
> select s1.f1, s1.f2, rank() over (partition by s1.f2 order by s1.f4) from
> struct_table_example;
> {noformat}
> This would throw the following error
> {noformat}
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"test","reducesinkkey1":4},"value":{"_col1":{"f1":false,"f2":"test","f3":3,"f4":4}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:297)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"test","reducesinkkey1":4},"value":{"_col1":{"f1":false,"f2":"test","f3":3,"f4":4}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:287)
> ... 16 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.getPrimitiveJavaObject(WritableIntObjectInspector.java:46)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:412)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.copyToStandardObject(GenericUDAFRank.java:219)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank$GenericUDAFAbstractRankEvaluator.iterate(GenericUDAFRank.java:154)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
> at
> org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:407)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:325)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:139)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
> ... 17 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1546783872011_263870_1_01 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:196)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> (state=08S01,code=2)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737825#comment-16737825
]
Hive QA commented on HIVE-21077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954246/HIVE-21077.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15554/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15554/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15554/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12954246/HIVE-21077.04.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954246 - PreCommit-HIVE-Build
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737824#comment-16737824
]
Hive QA commented on HIVE-21077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954246/HIVE-21077.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15699 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testDatabaseOps
(batchId=230)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15553/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15553/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15553/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954246 - PreCommit-HIVE-Build
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737805#comment-16737805
]
Hive QA commented on HIVE-21077:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
17s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15553/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15553/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15553/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7:
[jira] [Commented] (HIVE-21104) PTF with nested structure throws ClassCastException
[
https://issues.apache.org/jira/browse/HIVE-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737797#comment-16737797
]
Gopal V commented on HIVE-21104:
Looks like it is trying to cast s1 as the type of s1.f4
{code}
rb.currVal =
GenericUDAFRank.copyToStandardObject(parameters, inputOI,
ObjectInspectorCopyOption.JAVA);
{code}
gets parameters[0] as a Struct instead of getting it as two different integer
values. The struct is not extracted into two values - but remains as a
LazyStruct.
> PTF with nested structure throws ClassCastException
> ---
>
> Key: HIVE-21104
> URL: https://issues.apache.org/jira/browse/HIVE-21104
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Major
>
> {noformat}
> DROP TABLE IF EXISTS dummy;
> CREATE TABLE dummy (i int);
> INSERT INTO TABLE dummy VALUES (1);
> DROP TABLE IF EXISTS struct_table_example;
> CREATE TABLE struct_table_example (a int, s1 struct f3: int, f4: int> ) STORED AS ORC;
> INSERT INTO TABLE struct_table_example SELECT 1, named_struct('f1', false,
> 'f2', 'test', 'f3', 3, 'f4', 4) FROM dummy;
> select s1.f1, s1.f2, rank() over (partition by s1.f2 order by s1.f4) from
> struct_table_example;
> {noformat}
> This would throw the following error
> {noformat}
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0)
> {"key":{"reducesinkkey0":"test","reducesinkkey1":4},"value":{"_col1":{"f1":false,"f2":"test","f3":3,"f4":4}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:297)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{"reducesinkkey0":"test","reducesinkkey1":4},"value":{"_col1":{"f1":false,"f2":"test","f3":3,"f4":4}}}
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:287)
> ... 16 more
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector.getPrimitiveJavaObject(WritableIntObjectInspector.java:46)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:412)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.copyToStandardObject(GenericUDAFRank.java:219)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank$GenericUDAFAbstractRankEvaluator.iterate(GenericUDAFRank.java:154)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
> at
> org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.processRow(WindowingTableFunction.java:407)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.processRow(PTFOperator.java:325)
> at
> org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:139)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at
> org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356)
> ... 17 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1
> killedTasks:0, Vertex vertex_1546783872011_263870_1_01 [Reducer 2]
> killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:0
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:196)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> (state=08S01,code=2)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21105) LLAP: provide an option to configure #retries and time between retries for LLAP-ZK SecretManager
[
https://issues.apache.org/jira/browse/HIVE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737789#comment-16737789
]
Hive QA commented on HIVE-21105:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954243/HIVE-21105.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hive.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable[2]
(batchId=209)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15552/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15552/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15552/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954243 - PreCommit-HIVE-Build
> LLAP: provide an option to configure #retries and time between retries for
> LLAP-ZK SecretManager
>
>
> Key: HIVE-21105
> URL: https://issues.apache.org/jira/browse/HIVE-21105
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1, 2.3.4
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-21105.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21110) Stats replication for materialized views
[ https://issues.apache.org/jira/browse/HIVE-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat reassigned HIVE-21110: - > Stats replication for materialized views > > > Key: HIVE-21110 > URL: https://issues.apache.org/jira/browse/HIVE-21110 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > > Check if materialized views have stats associated with them. If so, support > replicating those statistics. Most of this should be testing whether the code > for table level stats replication is working for materialized views as well. > But since materialized views are handled as views, they have slightly > different code path than normal tables e.g. creating a materialized view. > Those paths will need fixes along the lines of normal table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Open (was: Patch Available)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch, HIVE-20170.8.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21108) Assign writeId for stats update for a converted transactional table
[ https://issues.apache.org/jira/browse/HIVE-21108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat reassigned HIVE-21108: - > Assign writeId for stats update for a converted transactional table > --- > > Key: HIVE-21108 > URL: https://issues.apache.org/jira/browse/HIVE-21108 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > > When a non-ACID table on the source is converted to an ACID table on the > target, the subsequent statistics update (column as well as table level) > dumped on the source won't have writeId and snapshot associated with those. > When loading those updates on the target we need to associate an appropriate > writeId with them. This applies to both a bootstrap and an incremental dump > and load. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21109) Stats replication for ACID tables.
[ https://issues.apache.org/jira/browse/HIVE-21109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat reassigned HIVE-21109: - > Stats replication for ACID tables. > -- > > Key: HIVE-21109 > URL: https://issues.apache.org/jira/browse/HIVE-21109 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > > Transactional tables require a writeID associated with the stats update. This > writeId needs to be in sync with the writeId on the source and hence needs to > be replicated from the source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21108) Assign writeId for stats update for a converted transactional table
[ https://issues.apache.org/jira/browse/HIVE-21108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737771#comment-16737771 ] Ashutosh Bapat commented on HIVE-21108: --- When we start replicating the stats, the tests for migrated tables fail since there is not writeId associated with the stats update of non-ACID tables on the source. So, we require HIVE-21108 to be fixed before HIVE-21037 is complete. > Assign writeId for stats update for a converted transactional table > --- > > Key: HIVE-21108 > URL: https://issues.apache.org/jira/browse/HIVE-21108 > Project: Hive > Issue Type: Sub-task >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > > When a non-ACID table on the source is converted to an ACID table on the > target, the subsequent statistics update (column as well as table level) > dumped on the source won't have writeId and snapshot associated with those. > When loading those updates on the target we need to associate an appropriate > writeId with them. This applies to both a bootstrap and an incremental dump > and load. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Attachment: HIVE-20170.8.patch
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch, HIVE-20170.8.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Patch Available (was: Open)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch, HIVE-20170.8.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21105) LLAP: provide an option to configure #retries and time between retries for LLAP-ZK SecretManager
[
https://issues.apache.org/jira/browse/HIVE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737746#comment-16737746
]
Hive QA commented on HIVE-21105:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} llap-common in master has 76 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-common: The patch generated 1 new + 22 unchanged -
0 fixed = 23 total (was 22) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15552/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15552/yetus/diff-checkstyle-llap-common.txt
|
| modules | C: common llap-common U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15552/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP: provide an option to configure #retries and time between retries for
> LLAP-ZK SecretManager
>
>
> Key: HIVE-21105
> URL: https://issues.apache.org/jira/browse/HIVE-21105
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1, 2.3.4
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-21105.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[ https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737738#comment-16737738 ] Vineet Garg commented on HIVE-21107: Will add tests + fix checkstyle in next patch. > Cannot find field" error during dynamically partitioned hash join > - > > Key: HIVE-21107 > URL: https://issues.apache.org/jira/browse/HIVE-21107 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21107.1.patch > > > This occurs in non-CBO path with dynamic partitioned join + constant > propagation ON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[ https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21107: --- Attachment: HIVE-21107.1.patch > Cannot find field" error during dynamically partitioned hash join > - > > Key: HIVE-21107 > URL: https://issues.apache.org/jira/browse/HIVE-21107 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21107.1.patch > > > This occurs in non-CBO path with dynamic partitioned join + constant > propagation ON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[ https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-21107: -- > Cannot find field" error during dynamically partitioned hash join > - > > Key: HIVE-21107 > URL: https://issues.apache.org/jira/browse/HIVE-21107 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > This occurs in non-CBO path with dynamic partitioned join + constant > propagation ON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
[
https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman resolved HIVE-21106.
---
Resolution: Won't Fix
Fix Version/s: 4.0.0
Release Note: n/a
already addressed on master via another commit
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcAcidRowBatchReader.java#L1570-L1575
> Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
> --
>
> Key: HIVE-21106
> URL: https://issues.apache.org/jira/browse/HIVE-21106
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
> Fix For: 4.0.0
>
>
> {{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}}
> {noformat}
> AcidStats acidStats = OrcAcidUtils.parseAcidStats(deleteDeltaReader);
> if (acidStats.deletes == 0) {
> continue; // just a safe check to ensure that we are not reading empty
> delete files.
> }
> {noformat}
> If the {{delete_delta../bucket_x}} is empty, it may not have a
> {{hive.acid.index}} and {{OrcAcidUtils.parseAcidStats()}} will return null
> which causes NPE.
> Even though HIVE-20941 will ensure empty files are no longer created but
> empty files predating that fix may exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21107) Cannot find field" error during dynamically partitioned hash join
[ https://issues.apache.org/jira/browse/HIVE-21107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21107: --- Status: Patch Available (was: Open) > Cannot find field" error during dynamically partitioned hash join > - > > Key: HIVE-21107 > URL: https://issues.apache.org/jira/browse/HIVE-21107 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-21107.1.patch > > > This occurs in non-CBO path with dynamic partitioned join + constant > propagation ON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
[
https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman reassigned HIVE-21106:
-
Assignee: Eugene Koifman
> Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
> --
>
> Key: HIVE-21106
> URL: https://issues.apache.org/jira/browse/HIVE-21106
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Major
>
> {{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}}
> {noformat}
> AcidStats acidStats = OrcAcidUtils.parseAcidStats(deleteDeltaReader);
> if (acidStats.deletes == 0) {
> continue; // just a safe check to ensure that we are not reading empty
> delete files.
> }
> {noformat}
> If the {{delete_delta../bucket_x}} is empty, it may not have a
> {{hive.acid.index}} and {{OrcAcidUtils.parseAcidStats()}} will return null
> which causes NPE.
> Even though HIVE-20941 will ensure empty files are no longer created but
> empty files predating that fix may exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737734#comment-16737734
]
Mani M commented on HIVE-18884:
---
Hi Peter,
Again the test got failed in some other classes Which was changed by me.
I hv made changes point to the logging...
Any suggestions to avoid the this other than raising jira tickets for
disabling and enabling the pre commit testing.
With Regards
M.Mani
+61 432 461 087
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21106) Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
[
https://issues.apache.org/jira/browse/HIVE-21106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-21106:
--
Description:
{{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}}
{noformat}
AcidStats acidStats = OrcAcidUtils.parseAcidStats(deleteDeltaReader);
if (acidStats.deletes == 0) {
continue; // just a safe check to ensure that we are not reading empty delete
files.
}
{noformat}
If the {{delete_delta../bucket_x}} is empty, it may not have a
{{hive.acid.index}} and {{OrcAcidUtils.parseAcidStats()}} will return null
which causes NPE.
Even though HIVE-20941 will ensure empty files are no longer created but empty
files predating that fix may exist
was:
{{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}}
{noformat}
AcidStats acidStats = OrcAcidUtils.parseAcidStats(deleteDeltaReader);
if (acidStats.deletes == 0) {
continue; // just a safe check to ensure that we are not reading empty delete
files.
}
{noformat}
If the {{delete_delta../bucket_x}} is empty, it may not have a
{{hive.acid.index}} and {{OrcAcidUtils.parseAcidStats()}} will return null
which causes NPE.
> Potential NEP in VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry
> --
>
> Key: HIVE-21106
> URL: https://issues.apache.org/jira/browse/HIVE-21106
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Priority: Major
>
> {{VectorizedOrcAcidRowBatchReader.ColumnizedDeleteEventRegistry()}}
> {noformat}
> AcidStats acidStats = OrcAcidUtils.parseAcidStats(deleteDeltaReader);
> if (acidStats.deletes == 0) {
> continue; // just a safe check to ensure that we are not reading empty
> delete files.
> }
> {noformat}
> If the {{delete_delta../bucket_x}} is empty, it may not have a
> {{hive.acid.index}} and {{OrcAcidUtils.parseAcidStats()}} will return null
> which causes NPE.
> Even though HIVE-20941 will ensure empty files are no longer created but
> empty files predating that fix may exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737728#comment-16737728
]
Hive QA commented on HIVE-18884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954222/HIVE.18884.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=220)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954222 - PreCommit-HIVE-Build
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-21077:
---
Attachment: (was: HIVE-21077.04.patch)
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-21077:
---
Attachment: HIVE-21077.04.patch
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737704#comment-16737704
]
Vihang Karajgaonkar commented on HIVE-21077:
Latest patch should fix most of the test failures above. Also, rethought about
the default 0 which is not really what we want since it maps to a legitimate
time value (1st Jan 1970 0:0:0 GMT). I think its best to keep this column
allowing nulls to keep backwards compatibility with the existing database and
catalogs in the system.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-21077:
---
Attachment: HIVE-21077.04.patch
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737686#comment-16737686
]
Hive QA commented on HIVE-18884:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
27s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15551/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15551/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
>
[jira] [Updated] (HIVE-21105) LLAP: provide an option to configure #retries and time between retries for LLAP-ZK SecretManager
[ https://issues.apache.org/jira/browse/HIVE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-21105: Attachment: HIVE-21105.1.patch > LLAP: provide an option to configure #retries and time between retries for > LLAP-ZK SecretManager > > > Key: HIVE-21105 > URL: https://issues.apache.org/jira/browse/HIVE-21105 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.1.1, 2.3.4 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21105.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21105) LLAP: provide an option to configure #retries and time between retries for LLAP-ZK SecretManager
[ https://issues.apache.org/jira/browse/HIVE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-21105: Status: Patch Available (was: Open) > LLAP: provide an option to configure #retries and time between retries for > LLAP-ZK SecretManager > > > Key: HIVE-21105 > URL: https://issues.apache.org/jira/browse/HIVE-21105 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 2.3.4, 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-21105.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[
https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737673#comment-16737673
]
Hive QA commented on HIVE-21103:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954220/HIVE-21103.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15550/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15550/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15550/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954220 - PreCommit-HIVE-Build
> PartitionManagementTask should not modify DN configs to avoid closing
> persistence manager
> -
>
> Key: HIVE-21103
> URL: https://issues.apache.org/jira/browse/HIVE-21103
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-21103.1.patch
>
>
> HIVE-20707 added automatic partition management which uses thread pools to
> run parallel msck repair. It also modifies datanucleus connection pool size
> to avoid explosion of connections to backend database. But object store
> closes the persistence manager when it detects a change in datanuclues or jdo
> configs. So when PartitionManagementTask is running and when HS2 tries to
> connect to metastore HS2 will get persistence manager close exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21100) Allow flattening of table subdirectories resulted when using TEZ engine and UNION clause
[ https://issues.apache.org/jira/browse/HIVE-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737666#comment-16737666 ] Eugene Koifman commented on HIVE-21100: --- What is the motivation for this change? Doing FileSystem.rename() on a system like S3 is expensive. > Allow flattening of table subdirectories resulted when using TEZ engine and > UNION clause > > > Key: HIVE-21100 > URL: https://issues.apache.org/jira/browse/HIVE-21100 > Project: Hive > Issue Type: Improvement >Reporter: George Pachitariu >Assignee: George Pachitariu >Priority: Minor > Attachments: HIVE-21100.patch > > > Right now, when writing data into a table with Tez engine and the clause > UNION ALL is the last step of the query, Hive on Tez will create a > subdirectory for each branch of the UNION ALL. > With this patch the subdirectories are removed, and the files are renamed and > moved to the parent directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21105) LLAP: provide an option to configure #retries and time between retries for LLAP-ZK SecretManager
[ https://issues.apache.org/jira/browse/HIVE-21105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-21105: --- Assignee: Vaibhav Gumashta > LLAP: provide an option to configure #retries and time between retries for > LLAP-ZK SecretManager > > > Key: HIVE-21105 > URL: https://issues.apache.org/jira/browse/HIVE-21105 > Project: Hive > Issue Type: Bug > Components: llap >Affects Versions: 3.1.1, 2.3.4 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[
https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737631#comment-16737631
]
Hive QA commented on HIVE-21103:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
32s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
1s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15550/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15550/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PartitionManagementTask should not modify DN configs to avoid closing
> persistence manager
> -
>
> Key: HIVE-21103
> URL: https://issues.apache.org/jira/browse/HIVE-21103
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-21103.1.patch
>
>
> HIVE-20707 added automatic partition management which uses thread pools to
> run parallel msck repair. It also modifies datanucleus connection pool size
> to avoid explosion of connections to backend database. But object store
> closes the persistence manager when it detects a change in datanuclues or jdo
> configs. So when PartitionManagementTask is running and when HS2 tries to
> connect to metastore HS2 will get persistence manager close exception.
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-21082) In HPL/SQL, declare statement does not support variable of type character
[ https://issues.apache.org/jira/browse/HIVE-21082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737618#comment-16737618 ] Daniel Dai commented on HIVE-21082: --- I'm fine with the patch, but can you add a test to it as convention? > In HPL/SQL, declare statement does not support variable of type character > -- > > Key: HIVE-21082 > URL: https://issues.apache.org/jira/browse/HIVE-21082 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21082.patch > > > In the following HPL/SQL programs: > DECLARE a character(5); SET a = 'b'; > when the type of variable 'a' is CHARACTER, it cannot be assigned a value > successfully. The support for the character type should be added to DECLARE > statement. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16976) DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
[
https://issues.apache.org/jira/browse/HIVE-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737611#comment-16737611
]
Hive QA commented on HIVE-16976:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954219/HIVE-16976.10.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=170)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15549/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15549/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15549/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954219 - PreCommit-HIVE-Build
> DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
>
>
> Key: HIVE-16976
> URL: https://issues.apache.org/jira/browse/HIVE-16976
> Project: Hive
> Issue Type: Improvement
> Components: Tez
>Affects Versions: 2.1.1, 3.0.0
>Reporter: Gopal V
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-16976.1.patch, HIVE-16976.10.patch,
> HIVE-16976.2.patch, HIVE-16976.3.patch, HIVE-16976.4.patch,
> HIVE-16976.5.patch, HIVE-16976.6.patch, HIVE-16976.7.patch,
> HIVE-16976.8.patch, HIVE-16976.9.patch
>
>
> Tez DPP does not kick in for scenarios where a user wants to run a comparison
> clause instead of a JOIN/IN clause.
> {code}
> explain select count(1) from store_sales where ss_sold_date_sk > (select
> max(d_Date_sk) from date_dim where d_year = 2017);
> Warning: Map Join MAPJOIN[21][bigTable=?] in task 'Map 1' is a cross product
> OK
> Plan optimized by CBO.
> Vertex dependency in root stage
> Map 1 <- Reducer 4 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
> Stage-0
> Fetch Operator
> limit:-1
> Stage-1
> Reducer 2 vectorized, llap
> File Output Operator [FS_36]
> Group By Operator [GBY_35] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(VALUE._col0)"]
> <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_34]
> Group By Operator [GBY_33] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(1)"]
> Select Operator [SEL_32] (rows=9600142089 width=16)
> Filter Operator [FIL_31] (rows=9600142089 width=16)
> predicate:(_col0 > _col1)
> Map Join Operator [MAPJOIN_30] (rows=28800426268 width=16)
> Conds:(Inner),Output:["_col0","_col1"]
> <-Reducer 4 [BROADCAST_EDGE] vectorized, llap
> BROADCAST [RS_28]
> Group By Operator [GBY_27] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(VALUE._col0)"]
> <-Map 3 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_26]
> Group By Operator [GBY_25] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(d_date_sk)"]
> Select Operator [SEL_24] (rows=652 width=12)
> Output:["d_date_sk"]
> Filter Operator [FIL_23] (rows=652 width=12)
> predicate:(d_year = 2017)
> TableScan [TS_2] (rows=73049 width=12)
>
> tpcds_bin_partitioned_newschema_orc_1@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
> <-Select Operator [SEL_29] (rows=28800426268 width=8)
> Output:["_col0"]
> TableScan [TS_0] (rows=28800426268 width=172)
>
> tpcds_bin_partitioned_newschema_orc_1@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE
> {code}
> The
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737602#comment-16737602
]
Vihang Karajgaonkar commented on HIVE-21077:
test failures are related. Looking into these.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16976) DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
[
https://issues.apache.org/jira/browse/HIVE-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737573#comment-16737573
]
Hive QA commented on HIVE-16976:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 26 new + 130 unchanged - 3
fixed = 156 total (was 133) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
1s{color} | {color:red} ql generated 1 new + 2310 unchanged - 0 fixed = 2311
total (was 2310) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Possible null pointer dereference of table in
org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(Node,
Stack, NodeProcessorCtx, Object[]) Dereferenced at
DynamicPartitionPruningOptimization.java:table in
org.apache.hadoop.hive.ql.optimizer.DynamicPartitionPruningOptimization.process(Node,
Stack, NodeProcessorCtx, Object[]) Dereferenced at
DynamicPartitionPruningOptimization.java:[line 192] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15549/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15549/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15549/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15549/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
>
>
> Key: HIVE-16976
> URL: https://issues.apache.org/jira/browse/HIVE-16976
> Project: Hive
> Issue Type: Improvement
> Components: Tez
>Affects Versions: 2.1.1, 3.0.0
>Reporter: Gopal V
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-16976.1.patch, HIVE-16976.10.patch,
> HIVE-16976.2.patch, HIVE-16976.3.patch, HIVE-16976.4.patch,
> HIVE-16976.5.patch, HIVE-16976.6.patch, HIVE-16976.7.patch,
> HIVE-16976.8.patch, HIVE-16976.9.patch
>
>
> Tez DPP does not kick in for scenarios where a user wants to run a comparison
> clause instead of a JOIN/IN clause.
> {code}
> explain select count(1) from store_sales where ss_sold_date_sk > (select
> max(d_Date_sk) from date_dim
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737542#comment-16737542
]
Hive QA commented on HIVE-20170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954215/HIVE-20170.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15548/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15548/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15548/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12954215/HIVE-20170.7.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954215 - PreCommit-HIVE-Build
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737540#comment-16737540
]
Hive QA commented on HIVE-20170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954215/HIVE-20170.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954215 - PreCommit-HIVE-Build
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-18884:
--
Attachment: HIVE.18884.06.patch
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-18884:
--
Labels: logging newbie noob (was: noob)
Affects Version/s: 4.0.0
Status: Patch Available (was: Open)
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: noob, newbie, logging
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737507#comment-16737507 ] Prasanth Jayachandran commented on HIVE-21103: -- Thanks [~sankarh] for finding for the root cause of this issue! Since you have more context on this issue can you please review this change? It just removes the DN config change in PartitionManagementTask > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-21103.1.patch > > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21103: - Status: Patch Available (was: Open) > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-21103.1.patch > > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-21103: - Attachment: HIVE-21103.1.patch > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-21103.1.patch > > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737504#comment-16737504
]
Hive QA commented on HIVE-20170:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15547/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15547/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16976) DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
[
https://issues.apache.org/jira/browse/HIVE-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal updated HIVE-16976:
--
Attachment: HIVE-16976.10.patch
> DPP: SyntheticJoinPredicate transitivity for < > and BETWEEN
>
>
> Key: HIVE-16976
> URL: https://issues.apache.org/jira/browse/HIVE-16976
> Project: Hive
> Issue Type: Improvement
> Components: Tez
>Affects Versions: 2.1.1, 3.0.0
>Reporter: Gopal V
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-16976.1.patch, HIVE-16976.10.patch,
> HIVE-16976.2.patch, HIVE-16976.3.patch, HIVE-16976.4.patch,
> HIVE-16976.5.patch, HIVE-16976.6.patch, HIVE-16976.7.patch,
> HIVE-16976.8.patch, HIVE-16976.9.patch
>
>
> Tez DPP does not kick in for scenarios where a user wants to run a comparison
> clause instead of a JOIN/IN clause.
> {code}
> explain select count(1) from store_sales where ss_sold_date_sk > (select
> max(d_Date_sk) from date_dim where d_year = 2017);
> Warning: Map Join MAPJOIN[21][bigTable=?] in task 'Map 1' is a cross product
> OK
> Plan optimized by CBO.
> Vertex dependency in root stage
> Map 1 <- Reducer 4 (BROADCAST_EDGE)
> Reducer 2 <- Map 1 (CUSTOM_SIMPLE_EDGE)
> Reducer 4 <- Map 3 (CUSTOM_SIMPLE_EDGE)
> Stage-0
> Fetch Operator
> limit:-1
> Stage-1
> Reducer 2 vectorized, llap
> File Output Operator [FS_36]
> Group By Operator [GBY_35] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(VALUE._col0)"]
> <-Map 1 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_34]
> Group By Operator [GBY_33] (rows=1 width=8)
> Output:["_col0"],aggregations:["count(1)"]
> Select Operator [SEL_32] (rows=9600142089 width=16)
> Filter Operator [FIL_31] (rows=9600142089 width=16)
> predicate:(_col0 > _col1)
> Map Join Operator [MAPJOIN_30] (rows=28800426268 width=16)
> Conds:(Inner),Output:["_col0","_col1"]
> <-Reducer 4 [BROADCAST_EDGE] vectorized, llap
> BROADCAST [RS_28]
> Group By Operator [GBY_27] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(VALUE._col0)"]
> <-Map 3 [CUSTOM_SIMPLE_EDGE] vectorized, llap
> PARTITION_ONLY_SHUFFLE [RS_26]
> Group By Operator [GBY_25] (rows=1 width=8)
> Output:["_col0"],aggregations:["max(d_date_sk)"]
> Select Operator [SEL_24] (rows=652 width=12)
> Output:["d_date_sk"]
> Filter Operator [FIL_23] (rows=652 width=12)
> predicate:(d_year = 2017)
> TableScan [TS_2] (rows=73049 width=12)
>
> tpcds_bin_partitioned_newschema_orc_1@date_dim,date_dim,Tbl:COMPLETE,Col:COMPLETE,Output:["d_date_sk","d_year"]
> <-Select Operator [SEL_29] (rows=28800426268 width=8)
> Output:["_col0"]
> TableScan [TS_0] (rows=28800426268 width=172)
>
> tpcds_bin_partitioned_newschema_orc_1@store_sales,store_sales,Tbl:COMPLETE,Col:COMPLETE
> {code}
> The SyntheticJoinPredicate is only injected for equi joins, not for < or >
> scalar subqueries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21103) PartitionManagementTask should not modify DN configs to avoid closing persistence manager
[ https://issues.apache.org/jira/browse/HIVE-21103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-21103: > PartitionManagementTask should not modify DN configs to avoid closing > persistence manager > - > > Key: HIVE-21103 > URL: https://issues.apache.org/jira/browse/HIVE-21103 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Critical > > HIVE-20707 added automatic partition management which uses thread pools to > run parallel msck repair. It also modifies datanucleus connection pool size > to avoid explosion of connections to backend database. But object store > closes the persistence manager when it detects a change in datanuclues or jdo > configs. So when PartitionManagementTask is running and when HS2 tries to > connect to metastore HS2 will get persistence manager close exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21102) Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
[
https://issues.apache.org/jira/browse/HIVE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737478#comment-16737478
]
Hive QA commented on HIVE-21102:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954172/HIVE-21102.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_combine_equivalent_work]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_4]
(batchId=192)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=190)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join32]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin1]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[cbo_stats]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_empty]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[optimize_nullscan]
(batchId=148)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[pcr] (batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[runtime_skewjoin_mapjoin_spark]
(batchId=136)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_18]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats_noscan_2]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_view]
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=131)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15546/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15546/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15546/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954172 - PreCommit-HIVE-Build
> Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
>
>
> Key: HIVE-21102
> URL: https://issues.apache.org/jira/browse/HIVE-21102
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-21102.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Patch Available (was: Open)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Attachment: HIVE-20170.7.patch
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Open (was: Patch Available)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch,
> HIVE-20170.6.patch, HIVE-20170.7.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: HIVE-20699.3.patch
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: (was: HIVE-20699.3.patch)
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21102) Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
[
https://issues.apache.org/jira/browse/HIVE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737438#comment-16737438
]
Hive QA commented on HIVE-21102:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15546/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15546/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
>
>
> Key: HIVE-21102
> URL: https://issues.apache.org/jira/browse/HIVE-21102
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-21102.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: HIVE-20699.3.patch
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737411#comment-16737411
]
Hive QA commented on HIVE-20170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954178/HIVE-20170.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=230)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=230)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15545/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15545/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15545/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954178 - PreCommit-HIVE-Build
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737343#comment-16737343
]
Hive QA commented on HIVE-20170:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15545/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15545/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21100) Allow flattening of table subdirectories resulted when using TEZ engine and UNION clause
[
https://issues.apache.org/jira/browse/HIVE-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737327#comment-16737327
]
Hive QA commented on HIVE-21100:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954155/HIVE-21100.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15544/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15544/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15544/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954155 - PreCommit-HIVE-Build
> Allow flattening of table subdirectories resulted when using TEZ engine and
> UNION clause
>
>
> Key: HIVE-21100
> URL: https://issues.apache.org/jira/browse/HIVE-21100
> Project: Hive
> Issue Type: Improvement
>Reporter: George Pachitariu
>Assignee: George Pachitariu
>Priority: Minor
> Attachments: HIVE-21100.patch
>
>
> Right now, when writing data into a table with Tez engine and the clause
> UNION ALL is the last step of the query, Hive on Tez will create a
> subdirectory for each branch of the UNION ALL.
> With this patch the subdirectories are removed, and the files are renamed and
> moved to the parent directory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21100) Allow flattening of table subdirectories resulted when using TEZ engine and UNION clause
[
https://issues.apache.org/jira/browse/HIVE-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737289#comment-16737289
]
Hive QA commented on HIVE-21100:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} common: The patch generated 1 new + 454 unchanged - 0
fixed = 455 total (was 454) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 19 new + 42 unchanged - 0
fixed = 61 total (was 42) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15544/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15544/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15544/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15544/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15544/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow flattening of table subdirectories resulted when using TEZ engine and
> UNION clause
>
>
> Key: HIVE-21100
> URL: https://issues.apache.org/jira/browse/HIVE-21100
> Project: Hive
> Issue Type: Improvement
>Reporter: George Pachitariu
>Assignee: George Pachitariu
>Priority: Minor
> Attachments: HIVE-21100.patch
>
>
> Right now, when writing data into a table with Tez engine and the clause
> UNION ALL is the last step of the query, Hive on Tez will create a
> subdirectory for each branch of the UNION ALL.
> With
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737264#comment-16737264
]
Hive QA commented on HIVE-21095:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954145/HIVE-21095.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15695 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15543/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15543/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15543/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954145 - PreCommit-HIVE-Build
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21094: - Status: In Progress (was: Patch Available) > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch, HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. > This was discussed in the community doc [Consistent timestamp types in Hadoop > SQL > engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21094: - Description: TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local time, and the writer's zone is stored with it. When reading, the timestamp in reader local time + reader zone is displayed. This is misleading for the user, since it looks like all the data was written in the reader's time zone. TIMESTAMPTZ should be stored in UTC time and be displayed in reader local time (as it was before) but should not display the reader's time zone. This was discussed in the community doc [Consistent timestamp types in Hadoop SQL engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing] was: TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local time, and the writer's zone is stored with it. When reading, the timestamp in reader local time + reader zone is displayed. This is misleading for the user, since it looks like all the data was written in the reader's time zone. TIMESTAMPTZ should be stored in UTC time and be displayed in reader local time (as it was before) but should not display the reader's time zone. > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch, HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. > This was discussed in the community doc [Consistent timestamp types in Hadoop > SQL > engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Description:
SHOW CREATE TABLE shows the time zone that the table was created in (if it
contains a TIMESTAMPTZ column). This is also misleading, since it might have
nothing to do with the actual data.
e.g.
{code:java}
hive> set time zone America/Los_Angeles;
hive> create table text_local (ts timestamp with local time zone) stored as
textfile;
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone('America/Los_Angeles'))
{code}
should be:
{code:java}
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone)
{code}
This was discussed in the community doc [Consistent timestamp types in Hadoop
SQL
engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
was:
SHOW CREATE TABLE shows the time zone that the table was created in (if it
contains a TIMESTAMPTZ column). This is also misleading, since it might have
nothing to do with the actual data.
e.g.
{code:java}
hive> set time zone America/Los_Angeles;
hive> create table text_local (ts timestamp with local time zone) stored as
textfile;
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone('America/Los_Angeles'))
{code}
should be:
{code:java}
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone)
{code}
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737225#comment-16737225
]
Hive QA commented on HIVE-21095:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15543/dev-support/hive-personality.sh
|
| git revision | master / e92df8d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15543/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21094: - Attachment: HIVE-21094.1.patch Status: Patch Available (was: Open) > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch, HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21094: - Status: Open (was: Patch Available) > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Open (was: Patch Available)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Attachment: (was: HIVE-20170.6.patch)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Patch Available (was: Open)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Attachment: HIVE-20170.6.patch
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Open (was: Patch Available)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Attachment: HIVE-20170.6.patch
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20170) Improve JoinOperator "rows for join key" Logging
[
https://issues.apache.org/jira/browse/HIVE-20170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
BELUGA BEHR updated HIVE-20170:
---
Status: Patch Available (was: Open)
> Improve JoinOperator "rows for join key" Logging
>
>
> Key: HIVE-20170
> URL: https://issues.apache.org/jira/browse/HIVE-20170
> Project: Hive
> Issue Type: Improvement
> Components: Operators
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-20170.1.patch, HIVE-20170.2.patch,
> HIVE-20170.3.patch, HIVE-20170.4.patch, HIVE-20170.5.patch, HIVE-20170.6.patch
>
>
> {code:java}
> 2018-06-25 09:37:33,193 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5728000 rows
> for join key [333, 22]
> 2018-06-25 09:37:33,901 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5828000 rows
> for join key [333, 22]
> 2018-06-25 09:37:34,623 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 5928000 rows
> for join key [333, 22]
> 2018-06-25 09:37:35,342 INFO [main]
> org.apache.hadoop.hive.ql.exec.CommonJoinOperator: table 0 has 6028000 rows
> for join key [333, 22]
> {code}
> [https://github.com/apache/hive/blob/6d890faf22fd1ede3658a5eed097476eab3c67e9/ql/src/java/org/apache/hadoop/hive/ql/exec/JoinOperator.java#L120]
> This logging should use the same facilities as the other Operators for
> emitting this type of log message. HIVE-10078
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21102) Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
[ https://issues.apache.org/jira/browse/HIVE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-21102: Status: Patch Available (was: Open) > Optimize SparkPlanGenerator for getInputPaths (emptyFile checks) > > > Key: HIVE-21102 > URL: https://issues.apache.org/jira/browse/HIVE-21102 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-21102.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21102) Optimize SparkPlanGenerator for getInputPaths (emptyFile checks)
[ https://issues.apache.org/jira/browse/HIVE-21102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-21102: Attachment: HIVE-21102.1.patch > Optimize SparkPlanGenerator for getInputPaths (emptyFile checks) > > > Key: HIVE-21102 > URL: https://issues.apache.org/jira/browse/HIVE-21102 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-21102.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
