[jira] [Assigned] (HIVE-23303) Fix test failures of Hive ProxyFileSystem against Hadoop 2.10.0 on branch-2
[ https://issues.apache.org/jira/browse/HIVE-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki reassigned HIVE-23303: --- > Fix test failures of Hive ProxyFileSystem against Hadoop 2.10.0 on branch-2 > --- > > Key: HIVE-23303 > URL: https://issues.apache.org/jira/browse/HIVE-23303 > Project: Hive > Issue Type: Bug > Components: Shims >Affects Versions: 2.3.7 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > On bumping Hadoop version to 2.10.0, ProxyFileSystem needs fix due to the > change of FilterFileSystem in Hadoop. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093059#comment-17093059
]
Hive QA commented on HIVE-22737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001261/HIVE-22737.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17141 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithServiceDiscovery.testKillQueryWithDifferentServer
(batchId=215)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21962/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21962/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21962/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001261 - PreCommit-HIVE-Build
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23294) Remove sync bottleneck in TezConfigurationFactory
[ https://issues.apache.org/jira/browse/HIVE-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093054#comment-17093054 ] Rajesh Balamohan commented on HIVE-23294: - [~ashutoshc] : Thanks for sharing the patch. Impact with reflection approach is that, it ends up checking for member access every time it is accessed adding to the overhead. > Remove sync bottleneck in TezConfigurationFactory > - > > Key: HIVE-23294 > URL: https://issues.apache.org/jira/browse/HIVE-23294 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Attachments: HIVE-23294.1.patch, HIVE-23294.2.patch, Screenshot > 2020-04-24 at 1.53.20 PM.png > > > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java#L53] > [https://github.com/apache/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L1628] > It ends up locking for property names in the config. For short running > queries with concurrency, this is an issue. > > !Screenshot 2020-04-24 at 1.53.20 PM.png|width=1086,height=459! > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23206) Project not defined correctly after reordering a join
[ https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23206: -- Status: Patch Available (was: Open) > Project not defined correctly after reordering a join > - > > Key: HIVE-23206 > URL: https://issues.apache.org/jira/browse/HIVE-23206 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Steve Carlin >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch, > HIVE-23206.3.patch > > > The following highlighted line seems to be incorrect in the test suite: > [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89] > Note that the project takes all the columns from the table scan, yet it only > needs a couple of them. > I did some very small debugging on this. When I removed the > applyJoinOrderingTransform here: > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897] > ... the problem goes away. So presumably one of the rules in there is > causing the problem. > Here is a slightly simplified version of the query which has the same problem > (using the same tpc-ds database): > explain cbo with ws_wh as > (select ws1.ws_order_number > from web_sales ws1,web_returns wr2 > where ws1.ws_order_number = wr2.wr_order_number) > select > ws_order_number > from > web_sales ws1 > where > ws1.ws_order_number in (select wr_order_number > from web_returns,ws_wh > where wr_order_number = ws_wh.ws_order_number) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23206) Project not defined correctly after reordering a join
[ https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23206: -- Attachment: HIVE-23206.3.patch > Project not defined correctly after reordering a join > - > > Key: HIVE-23206 > URL: https://issues.apache.org/jira/browse/HIVE-23206 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Steve Carlin >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch, > HIVE-23206.3.patch > > > The following highlighted line seems to be incorrect in the test suite: > [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89] > Note that the project takes all the columns from the table scan, yet it only > needs a couple of them. > I did some very small debugging on this. When I removed the > applyJoinOrderingTransform here: > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897] > ... the problem goes away. So presumably one of the rules in there is > causing the problem. > Here is a slightly simplified version of the query which has the same problem > (using the same tpc-ds database): > explain cbo with ws_wh as > (select ws1.ws_order_number > from web_sales ws1,web_returns wr2 > where ws1.ws_order_number = wr2.wr_order_number) > select > ws_order_number > from > web_sales ws1 > where > ws1.ws_order_number in (select wr_order_number > from web_returns,ws_wh > where wr_order_number = ws_wh.ws_order_number) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23206) Project not defined correctly after reordering a join
[ https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-23206: -- Status: Open (was: Patch Available) > Project not defined correctly after reordering a join > - > > Key: HIVE-23206 > URL: https://issues.apache.org/jira/browse/HIVE-23206 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Steve Carlin >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch, > HIVE-23206.3.patch > > > The following highlighted line seems to be incorrect in the test suite: > [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89] > Note that the project takes all the columns from the table scan, yet it only > needs a couple of them. > I did some very small debugging on this. When I removed the > applyJoinOrderingTransform here: > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897] > ... the problem goes away. So presumably one of the rules in there is > causing the problem. > Here is a slightly simplified version of the query which has the same problem > (using the same tpc-ds database): > explain cbo with ws_wh as > (select ws1.ws_order_number > from web_sales ws1,web_returns wr2 > where ws1.ws_order_number = wr2.wr_order_number) > select > ws_order_number > from > web_sales ws1 > where > ws1.ws_order_number in (select wr_order_number > from web_returns,ws_wh > where wr_order_number = ws_wh.ws_order_number) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093033#comment-17093033
]
Hive QA commented on HIVE-22737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 19 new + 27 unchanged - 0
fixed = 46 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21962/dev-support/hive-personality.sh
|
| git revision | master / b523426 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21962/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21962/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-23089:
--
Status: Patch Available (was: Open)
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
> Reduce Operator Tree:
> ...
> Filter Operator
> predicate: enforce_constraint((null is not null and _col3 is not
> null)) (type: boolean)
> {code}
> In CBO path the enforce_constraint function is added to the plan when CBO
> plan is already generated and optimized.
> {code}
> HiveSortExchange(distribution=[any], collation=[[0]])
> HiveProject(row__id=[$5], i=[CAST(1):INTEGER], _o__c2=[null:NULL], vc=[$2])
> HiveFilter(condition=[=($0, 1)])
> HiveTableScan(table=[[default, acid_uami]], table:alias=[acid_uami])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-23089:
--
Status: Open (was: Patch Available)
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
> Reduce Operator Tree:
> ...
> Filter Operator
> predicate: enforce_constraint((null is not null and _col3 is not
> null)) (type: boolean)
> {code}
> In CBO path the enforce_constraint function is added to the plan when CBO
> plan is already generated and optimized.
> {code}
> HiveSortExchange(distribution=[any], collation=[[0]])
> HiveProject(row__id=[$5], i=[CAST(1):INTEGER], _o__c2=[null:NULL], vc=[$2])
> HiveFilter(condition=[=($0, 1)])
> HiveTableScan(table=[[default, acid_uami]], table:alias=[acid_uami])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23089) Add constraint checks to CBO plan
[
https://issues.apache.org/jira/browse/HIVE-23089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-23089:
--
Attachment: HIVE-23089.4.patch
> Add constraint checks to CBO plan
> -
>
> Key: HIVE-23089
> URL: https://issues.apache.org/jira/browse/HIVE-23089
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23089.1.patch, HIVE-23089.2.patch,
> HIVE-23089.3.patch, HIVE-23089.4.patch
>
>
> {code}
> create table acid_uami(i int,
> de decimal(5,2) constraint nn1 not null enforced,
> vc varchar(128) constraint nn2 not null enforced) clustered
> by (i) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> explain
> update acid_uami set de=null where i=1;
> {code}
> Non-CBO path:
> {code:java}
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: ((i = 1) and enforce_constraint(vc is not null))
> (type: boolean)
> {code}
> CBO path:
> {code:java}
> Map Reduce
> Map Operator Tree:
> TableScan
> alias: acid_uami
> filterExpr: (i = 1) (type: boolean)
> Statistics: Num rows: 1 Data size: 216 Basic stats: COMPLETE
> Column stats: NONE
> Filter Operator
> predicate: (i = 1) (type: boolean)
> ...
> Reduce Operator Tree:
> ...
> Filter Operator
> predicate: enforce_constraint((null is not null and _col3 is not
> null)) (type: boolean)
> {code}
> In CBO path the enforce_constraint function is added to the plan when CBO
> plan is already generated and optimized.
> {code}
> HiveSortExchange(distribution=[any], collation=[[0]])
> HiveProject(row__id=[$5], i=[CAST(1):INTEGER], _o__c2=[null:NULL], vc=[$2])
> HiveFilter(condition=[=($0, 1)])
> HiveTableScan(table=[[default, acid_uami]], table:alias=[acid_uami])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23284) Remove dependency on mariadb-java-client
[ https://issues.apache.org/jira/browse/HIVE-23284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23284: - Attachment: HIVE-23284.01.patch Status: Patch Available (was: Open) > Remove dependency on mariadb-java-client > > > Key: HIVE-23284 > URL: https://issues.apache.org/jira/browse/HIVE-23284 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23284.01.patch, HIVE-23284.01.patch, > HIVE-23284.01.patch > > > It has GNU Lesser General Public License which is [Category > X|https://www.apache.org/legal/resolved.html#category-x]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23280) Trigger compaction with old aborted txns
[ https://issues.apache.org/jira/browse/HIVE-23280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23280: - Status: Open (was: Patch Available) > Trigger compaction with old aborted txns > > > Key: HIVE-23280 > URL: https://issues.apache.org/jira/browse/HIVE-23280 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23280.01.patch, HIVE-23280.01.patch, > HIVE-23280.01.patch, HIVE-23280.01.patch > > > When a txn is aborted and the compaction threshold for number of aborted txns > is not reached then the aborted transaction can remain forever in the RDBMS > database. This could result in several serious performance degradations: > - getOpenTxns has to list this aborted txn forever > - TXN_TO_WRITE_ID table is not cleaned > We should add a threshold, so after a given time the compaction is started > anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23280) Trigger compaction with old aborted txns
[ https://issues.apache.org/jira/browse/HIVE-23280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23280: - Attachment: HIVE-23280.01.patch Status: Patch Available (was: Open) > Trigger compaction with old aborted txns > > > Key: HIVE-23280 > URL: https://issues.apache.org/jira/browse/HIVE-23280 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23280.01.patch, HIVE-23280.01.patch, > HIVE-23280.01.patch, HIVE-23280.01.patch > > > When a txn is aborted and the compaction threshold for number of aborted txns > is not reached then the aborted transaction can remain forever in the RDBMS > database. This could result in several serious performance degradations: > - getOpenTxns has to list this aborted txn forever > - TXN_TO_WRITE_ID table is not cleaned > We should add a threshold, so after a given time the compaction is started > anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23284) Remove dependency on mariadb-java-client
[ https://issues.apache.org/jira/browse/HIVE-23284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-23284: - Status: Open (was: Patch Available) > Remove dependency on mariadb-java-client > > > Key: HIVE-23284 > URL: https://issues.apache.org/jira/browse/HIVE-23284 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23284.01.patch, HIVE-23284.01.patch > > > It has GNU Lesser General Public License which is [Category > X|https://www.apache.org/legal/resolved.html#category-x]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23040) Checkpointing for repl dump incremental phase
[ https://issues.apache.org/jira/browse/HIVE-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anishek Agarwal updated HIVE-23040: --- Resolution: Fixed Status: Resolved (was: Patch Available) +1 Patch committed to master, Thanks for the patch [~pkumarsinha] and review [~aasha] > Checkpointing for repl dump incremental phase > - > > Key: HIVE-23040 > URL: https://issues.apache.org/jira/browse/HIVE-23040 > Project: Hive > Issue Type: Improvement >Reporter: Aasha Medhi >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23040.01.patch, HIVE-23040.02.patch, > HIVE-23040.03.patch, HIVE-23040.04.patch, HIVE-23040.05.patch, > HIVE-23040.06.patch, HIVE-23040.06.patch, HIVE-23040.07.patch > > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-19064) Add mode to support delimited identifiers enclosed within double quotation
[
https://issues.apache.org/jira/browse/HIVE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-19064:
--
Status: Patch Available (was: Open)
> Add mode to support delimited identifiers enclosed within double quotation
> --
>
> Key: HIVE-19064
> URL: https://issues.apache.org/jira/browse/HIVE-19064
> Project: Hive
> Issue Type: Improvement
> Components: Parser, SQL
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-19064.01.patch, HIVE-19064.02.patch,
> HIVE-19064.03.patch, HIVE-19064.4.patch, HIVE-19064.5.patch,
> HIVE-19064.6.patch, HIVE-19064.7.patch, HIVE-19064.7.patch, HIVE-19064.8.patch
>
>
> As per SQL standard. Hive currently uses `` (backticks). Default will
> continue being backticks, but we will support identifiers within double
> quotation via configuration parameter.
> This issue will also extends support for arbitrary char sequences, e.g.,
> containing {{~ ! @ # $ % ^ & * () , < >}}, in database and table names.
> Currently, special characters are only supported for column names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-19064) Add mode to support delimited identifiers enclosed within double quotation
[
https://issues.apache.org/jira/browse/HIVE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-19064:
--
Status: Open (was: Patch Available)
> Add mode to support delimited identifiers enclosed within double quotation
> --
>
> Key: HIVE-19064
> URL: https://issues.apache.org/jira/browse/HIVE-19064
> Project: Hive
> Issue Type: Improvement
> Components: Parser, SQL
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-19064.01.patch, HIVE-19064.02.patch,
> HIVE-19064.03.patch, HIVE-19064.4.patch, HIVE-19064.5.patch,
> HIVE-19064.6.patch, HIVE-19064.7.patch, HIVE-19064.7.patch, HIVE-19064.8.patch
>
>
> As per SQL standard. Hive currently uses `` (backticks). Default will
> continue being backticks, but we will support identifiers within double
> quotation via configuration parameter.
> This issue will also extends support for arbitrary char sequences, e.g.,
> containing {{~ ! @ # $ % ^ & * () , < >}}, in database and table names.
> Currently, special characters are only supported for column names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-19064) Add mode to support delimited identifiers enclosed within double quotation
[
https://issues.apache.org/jira/browse/HIVE-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-19064:
--
Attachment: HIVE-19064.8.patch
> Add mode to support delimited identifiers enclosed within double quotation
> --
>
> Key: HIVE-19064
> URL: https://issues.apache.org/jira/browse/HIVE-19064
> Project: Hive
> Issue Type: Improvement
> Components: Parser, SQL
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-19064.01.patch, HIVE-19064.02.patch,
> HIVE-19064.03.patch, HIVE-19064.4.patch, HIVE-19064.5.patch,
> HIVE-19064.6.patch, HIVE-19064.7.patch, HIVE-19064.7.patch, HIVE-19064.8.patch
>
>
> As per SQL standard. Hive currently uses `` (backticks). Default will
> continue being backticks, but we will support identifiers within double
> quotation via configuration parameter.
> This issue will also extends support for arbitrary char sequences, e.g.,
> containing {{~ ! @ # $ % ^ & * () , < >}}, in database and table names.
> Currently, special characters are only supported for column names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Status: Patch Available (was: Open)
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Status: Open (was: Patch Available)
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Attachment: HIVE-22737.3.patch
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23133) Numeric operations can have different result across hardware archs
[ https://issues.apache.org/jira/browse/HIVE-23133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092970#comment-17092970 ] Zhenyu Zheng commented on HIVE-23133: - For those that might have concerns on the performance degration of changing from Math to StrictMath, here is some testing data: [https://bugs.openjdk.java.net/browse/JDK-8210416] from the data, we can see that the difference between Math and StrictMath function is actually very very small and StrictMath provides correct answers. > Numeric operations can have different result across hardware archs > -- > > Key: HIVE-23133 > URL: https://issues.apache.org/jira/browse/HIVE-23133 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Yikun Jiang >Priority: Major > Attachments: HIVE-23133.1.patch, HIVE-23133.2.patch, > HIVE-23133.3.patch > > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/] > Among the failures, we have observed that some numeric operations can have > different result across hardware archs, such as: > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_vector_decimal_udf2_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_subquery_select_/] > [https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.cli/TestSparkCliDriver/testCliDriver_vectorized_math_funcs_/] > we can see that the calculation results of log, exp, cos, toRadians etc is > slitly different than the .out file results that we are > comparing(they are tested and wrote on X86 machines), this is because of we > use [Math > Library|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html] for > these kind of calculations. > and according to the > [illustration|https://docs.oracle.com/javase/6/docs/api/java/lang/Math.html]: > _Unlike some of the numeric methods of class StrictMath, all implementations > of the equivalent functions of class Math are not_ > _defined to return the bit-for-bit same results. This relaxation permits > better-performing implementations where strict reproducibility_ > _is not required._ > _By default many of the Math methods simply call the equivalent method in > StrictMath for their implementation._ > _Code generators are encouraged to use platform-specific native libraries or > microprocessor instructions, where available,_ > _to provide higher-performance implementations of Math methods._ > so the result will have difference across hardware archs. > On the other hand, JAVA provided another library > [StrictMath|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html] > that will not have this kind of problem as according to its' > [reference|https://docs.oracle.com/javase/6/docs/api/java/lang/StrictMath.html]: > _To help ensure portability of Java programs, the definitions of some of the > numeric functions in this package require that they produce_ > _the same results as certain published algorithms._ > So in order to fix the above mentioned problem, we have to consider switch to > use StrictMath instead of Math. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092895#comment-17092895
]
Hive QA commented on HIVE-22737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001252/HIVE-22737.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17141 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_insert_partitioned]
(batchId=98)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21961/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21961/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21961/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001252 - PreCommit-HIVE-Build
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092888#comment-17092888
]
Hive QA commented on HIVE-22737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 19 new + 27 unchanged - 0
fixed = 46 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21961/dev-support/hive-personality.sh
|
| git revision | master / 19314f5 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21961/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21961/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Status: Patch Available (was: Open)
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Attachment: HIVE-22737.2.patch
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22737:
Status: Open (was: Patch Available)
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23268) Eliminate beanutils transitive dependency
[
https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092876#comment-17092876
]
Hive QA commented on HIVE-23268:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
38s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 91m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21959/dev-support/hive-personality.sh
|
| git revision | master / 2a84989 |
| Default Java | 1.8.0_111 |
| modules | C: storage-api shims/common shims/0.23 shims/scheduler
standalone-metastore standalone-metastore/metastore-common common serde
metastore llap-common llap-client llap-tez
standalone-metastore/metastore-server ql llap-server service accumulo-handler
jdbc beeline cli contrib druid-handler hbase-handler hcatalog hcatalog/core
hcatalog/hcatalog-pig-adapter hcatalog/server-extensions
hcatalog/webhcat/java-client hcatalog/webhcat/svr hcatalog/streaming hplsql
streaming llap-ext-client kryo-registrator kudu-handler kafka-handler
standalone-metastore/metastore-tools/metastore-benchmarks
upgrade-acid/pre-upgrade U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21959/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Eliminate beanutils transitive dependency
> -
>
> Key: HIVE-23268
> URL: https://issues.apache.org/jira/browse/HIVE-23268
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23268.2.patch, HIVE-23268.3.patch, HIVE-23268.patch
>
>
> Transitively retrieved from hadoop-commons
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23268) Eliminate beanutils transitive dependency
[ https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23268: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Eliminate beanutils transitive dependency > - > > Key: HIVE-23268 > URL: https://issues.apache.org/jira/browse/HIVE-23268 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23268.2.patch, HIVE-23268.3.patch, HIVE-23268.patch > > > Transitively retrieved from hadoop-commons -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23268) Eliminate beanutils transitive dependency
[
https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092871#comment-17092871
]
Hive QA commented on HIVE-23268:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001245/HIVE-23268.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21959/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21959/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21959/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001245 - PreCommit-HIVE-Build
> Eliminate beanutils transitive dependency
> -
>
> Key: HIVE-23268
> URL: https://issues.apache.org/jira/browse/HIVE-23268
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23268.2.patch, HIVE-23268.3.patch, HIVE-23268.patch
>
>
> Transitively retrieved from hadoop-commons
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23262) Remove dependency on activemq
[ https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23262: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Remove dependency on activemq > - > > Key: HIVE-23262 > URL: https://issues.apache.org/jira/browse/HIVE-23262 > Project: Hive > Issue Type: Improvement > Components: HCatalog >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23262.2.patch, HIVE-23262.patch > > > Activemq is a test dependency introduced to test message bus feature in > hcatalog. Even when it was introduced very first, there were concerns of > taking activemq as a dependency. > https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113 > AFAIK no one uses message bus feature of HCatalog. We can remove it > altogether. > As a first step removing tests for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23262) Remove dependency on activemq
[
https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092862#comment-17092862
]
Hive QA commented on HIVE-23262:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
48s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
55s{color} | {color:red} branch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} hcatalog: The patch generated 0 new + 0 unchanged -
9 fixed = 0 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch server-extensions passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} root: The patch generated 0 new + 0 unchanged - 9
fixed = 0 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
56s{color} | {color:red} patch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red} patch/hcatalog/server-extensions cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 17m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21958/dev-support/hive-personality.sh
|
| git revision | master / 2a84989 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21958/yetus/branch-findbugs-hcatalog.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21958/yetus/patch-findbugs-hcatalog.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21958/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| modules | C: hcatalog hcatalog/server-extensions . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21958/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove dependency on activemq
[jira] [Commented] (HIVE-23262) Remove dependency on activemq
[
https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092859#comment-17092859
]
Hive QA commented on HIVE-23262:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001246/HIVE-23262.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17141 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21958/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21958/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21958/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001246 - PreCommit-HIVE-Build
> Remove dependency on activemq
> -
>
> Key: HIVE-23262
> URL: https://issues.apache.org/jira/browse/HIVE-23262
> Project: Hive
> Issue Type: Improvement
> Components: HCatalog
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23262.2.patch, HIVE-23262.patch
>
>
> Activemq is a test dependency introduced to test message bus feature in
> hcatalog. Even when it was introduced very first, there were concerns of
> taking activemq as a dependency.
> https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113
> AFAIK no one uses message bus feature of HCatalog. We can remove it
> altogether.
> As a first step removing tests for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23301) Optimising privilege synchroniser: UDF for updating privileges
[
https://issues.apache.org/jira/browse/HIVE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092838#comment-17092838
]
Hive QA commented on HIVE-23301:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001248/UDFSplitMapPrivs.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17144 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[show_functions]
(batchId=109)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21957/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21957/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21957/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001248 - PreCommit-HIVE-Build
> Optimising privilege synchroniser: UDF for updating privileges
> --
>
> Key: HIVE-23301
> URL: https://issues.apache.org/jira/browse/HIVE-23301
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, UDF
>Affects Versions: 3.1.1
>Reporter: Simhadri G
>Assignee: Simhadri G
>Priority: Minor
> Attachments: UDFSplitMapPrivs.patch, UDFSplitMapPrivs.patch
>
>
> Privilege synchronizer pulls the list of databases, tables and columns from
> the Hive Metastore. For each of these objects it fetches the privilege
> information and invokes HMS API to refresh the privilege information in HMS.
> The current UDF Maps a bit string to a privilege based on if the privilege
> is granted or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[
https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092835#comment-17092835
]
Hive QA commented on HIVE-23207:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
23s{color} | {color:blue} standalone-metastore/metastore-server in master has
189 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} contrib in master has 11 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} itests/qtest-druid in master has 7 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
55s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 505 unchanged - 12 fixed = 505 total (was 517) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch contrib passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
33s{color} | {color:green} root: The patch generated 0 new + 650 unchanged - 12
fixed = 650 total (was 662) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch qtest-accumulo passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} The patch qtest-druid passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} The patch qtest-kudu passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
13s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
59s{color} | {color:red} patch/standalone-metastore/metastore-server cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 6m
26s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
41s{color} | {col
[jira] [Commented] (HIVE-23301) Optimising privilege synchroniser: UDF for updating privileges
[
https://issues.apache.org/jira/browse/HIVE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092831#comment-17092831
]
Hive QA commented on HIVE-23301:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m
18s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
19s{color} | {color:red} ql: The patch generated 213 new + 82 unchanged - 0
fixed = 295 total (was 82) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
26s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21957/dev-support/hive-personality.sh
|
| git revision | master / 62ed1aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21957/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21957/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21957/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21957/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimising privilege synchroniser: UDF for updating privileges
> --
>
> Key: HIVE-23301
> URL: https://issues.apache.org/jira/browse/HIVE-23301
> Project: Hive
> Issue Type: Improvement
> Components: Metastore, UDF
>Affects Versions: 3.1.1
>Reporter: Simhadri G
>Assignee: Simhadri G
>Priority: Minor
> Attachments: UDFSplitMapPrivs.patch, UDFSplitMapPrivs.patch
>
>
> Privilege synchronizer pulls the list of databases, tables and columns from
> the Hive Metastore. For each of these objects it fetches the privilege
> information and invokes HMS API to refresh the privilege information in HMS.
> The current UDF Maps a bit string to a privilege based on if the privilege
> is granted or not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23227) Refactor RexConverter and move some of its functionality into HiveFunctionHelper
[
https://issues.apache.org/jira/browse/HIVE-23227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23227:
---
Target Version/s: 4.0.0
> Refactor RexConverter and move some of its functionality into
> HiveFunctionHelper
>
>
> Key: HIVE-23227
> URL: https://issues.apache.org/jira/browse/HIVE-23227
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100, {{HiveFunctionHelper}} makes a few calls to methods that
> are in {{RexConverter}}. Those methods do not need to be there anymore but
> were not moved as part of that patch to avoid further changes in it. This
> issue is to tackle that refactoring.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23229) CAST to string on column instead of simplification over literal column
[
https://issues.apache.org/jira/browse/HIVE-23229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23229:
---
Target Version/s: 4.0.0
> CAST to string on column instead of simplification over literal column
> --
>
> Key: HIVE-23229
> URL: https://issues.apache.org/jira/browse/HIVE-23229
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, we end up for one of the queries with CAST over a
> column instead of applying CAST on literal and comparing in CHAR, which can
> be seen in ql/src/test/results/clientpositive/in_typecheck_char.q.out .
> {code}
> filterExpr: (((s = 'a') and (t = 'a ')) or (null and (t
> = 'bb'))) is null (type: boolean)
> {code}
> was replaced by:
> {code}
> filterExpr: (((CAST( s AS STRING) = 'a') and (CAST( t AS STRING)
> = 'a')) or (null and (CAST( t AS STRING) = 'bb'))) is null (type: boolean)
> {code}
> Probably this is as a result of the changes introduced in HIVE-23100 wrt IN
> handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23228) Missed optimization opportunity with equals and not equals
[
https://issues.apache.org/jira/browse/HIVE-23228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23228:
---
Target Version/s: 4.0.0
> Missed optimization opportunity with equals and not equals
> --
>
> Key: HIVE-23228
> URL: https://issues.apache.org/jira/browse/HIVE-23228
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity on the
> simplification of an AND predicate containing equals and not equals clause,
> which can be seen in ql/src/test/results/clientpositive/pcs.q.out .
> {code}
> filterExpr: ((key = 3) or (ds = '2000-04-08') or key is not
> null) and (key = 2)) or ((ds <> '2000-04-08') and (key = 3))) and ((key + 5)
> > 0))) (type: boolean)
> {code}
> was replaced by:
> {code}
> filterExpr: ((key = 3) or (ds = '2000-04-08') or key is not
> null) and (key = 2)) or ((ds <> '2000-04-08') and (key <> 2) and (key = 3)))
> and ((key + 5) > 0))) (type: boolean)
> {code}
> Note the additional {{key <> 2}} in predicate below.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23222) Missed opportunity in IN merge
[
https://issues.apache.org/jira/browse/HIVE-23222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23222:
---
Target Version/s: 4.0.0
> Missed opportunity in IN merge
> --
>
> Key: HIVE-23222
> URL: https://issues.apache.org/jira/browse/HIVE-23222
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity merging IN clauses,
> which can be seen in
> ql/src/test/results/clientpositive/llap/vector_between_in.q.out .
> {code}
>filterExpr: (cdecimal1) IN (2365.8945945946,
> 881.0135135135, -3367.6517567568) (type: boolean)
> {code}
> was replaced by:
> {code}
>filterExpr: ((cdecimal1) IN (2365.8945945946,
> -3367.6517567568) or (cdecimal1) IN (881.0135135135)) (type: boolean)
> {code}
> The problem seems to be that with decimal type, we are considering values
> with different precision/scale as a different type, thus we do not merge them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23225) Simplify ExprFactory, ExprNodeDescExprFactory and RexNodeExprFactory
[
https://issues.apache.org/jira/browse/HIVE-23225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23225:
---
Target Version/s: 4.0.0
> Simplify ExprFactory, ExprNodeDescExprFactory and RexNodeExprFactory
>
>
> Key: HIVE-23225
> URL: https://issues.apache.org/jira/browse/HIVE-23225
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> The new {{ExprFactory}} was created based on existing calls from
> {{TypeCheckProcFactory}}. Now that we have the {{ExprNodeDesc}} and
> {{RexNode}} implementations, it seems we could do some work consolidating
> those methods, simplifying the super/subclasses, etc. For instance, the
> handling of literal values seems quite convoluted (handled by many different
> method) and could possibly be abstracted in a different way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23223) Unnecessary CAST to decimal around CASE statement
[
https://issues.apache.org/jira/browse/HIVE-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23223:
---
Target Version/s: 4.0.0
> Unnecessary CAST to decimal around CASE statement
> -
>
> Key: HIVE-23223
> URL: https://issues.apache.org/jira/browse/HIVE-23223
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity on the
> simplification of a CAST statement on top of a CASE clause, which can be seen
> in ql/src/test/results/clientpositive/vector_case_when_2.q.out .
> {code}
>expressions: q548284 (type: int), CASE WHEN ((q548284 = 4))
> THEN (0.8) WHEN ((q548284 = 5)) THEN (1) ELSE (8) END (type: decimal(2,1))
> {code}
> was replaced by:
> {code}
>expressions: q548284 (type: int), CAST( CASE WHEN ((q548284 =
> 4)) THEN (0.8) WHEN ((q548284 = 5)) THEN (1) ELSE (8) END AS decimal(11,1))
> (type: decimal(11,1))
> {code}
> The type of the CASE expression could be inferred and enforce without the
> CAST.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23226) Implement Calcite rule to transform CASE into COALESCE when possible
[
https://issues.apache.org/jira/browse/HIVE-23226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23226:
---
Target Version/s: 4.0.0
> Implement Calcite rule to transform CASE into COALESCE when possible
>
>
> Key: HIVE-23226
> URL: https://issues.apache.org/jira/browse/HIVE-23226
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently, it is done in {{TypeCheckProcFactory}} when we create a Hive
> expression after Calcite optimization.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23224) Literals in CBO plan could show less information
[
https://issues.apache.org/jira/browse/HIVE-23224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23224:
---
Target Version/s: 4.0.0
> Literals in CBO plan could show less information
>
>
> Key: HIVE-23224
> URL: https://issues.apache.org/jira/browse/HIVE-23224
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently, they are very verbose. For all char and varchar literals it will
> show the encoding, thought it is always the same. For varchar literals, it
> prints type and length, which seems unnecessary.
> For instance:
> {code}
>HiveFilter(condition=[AND(IN($10,
> _UTF-16LE'wallpaper':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'parenting':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'musical':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'womens':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'birdal':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'pants':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($12,
> _UTF-16LE'Home':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Books':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Electronics':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Shoes':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Jewelry':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Men':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), OR(AND(IN($12,
> _UTF-16LE'Home':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Books':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Electronics':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($10,
> _UTF-16LE'wallpaper':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'parenting':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'musical':VARCHAR(2147483647) CHARACTER SET "UTF-16LE")),
> AND(IN($12, _UTF-16LE'Shoes':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Jewelry':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Men':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($10,
> _UTF-16LE'womens':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'birdal':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'pants':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23302) Create HiveJdbcDatabaseAccessor for JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-23302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23302:
---
Target Version/s: 4.0.0
> Create HiveJdbcDatabaseAccessor for JDBC storage handler
>
>
> Key: HIVE-23302
> URL: https://issues.apache.org/jira/browse/HIVE-23302
> Project: Hive
> Issue Type: Bug
> Components: StorageHandler
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> The {{JdbcDatabaseAccessor}} associated with the storage handler makes some
> SQL calls to the RDBMS through the JDBC connection. There is a
> {{GenericJdbcDatabaseAccessor}} with a generic implementation that the
> storage handler uses if there is no specific implementation for a certain
> RDBMS.
> Currently, Hive uses the {{GenericJdbcDatabaseAccessor}}. Afaik the only
> generic query that will not work is splitting the query based on offset and
> limit, since the syntax for that query is different than the one accepted by
> Hive. We should create a {{HiveJdbcDatabaseAccessor}} to override that query
> and possibly fix any other existing incompatibilities.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23291: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~vgarg], I have created HIVE-23302. Pushed to master. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092822#comment-17092822 ] Jesus Camacho Rodriguez edited comment on HIVE-23291 at 4/26/20, 7:03 PM: -- AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split \-for complex queries, this will just not work\-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you have mentioned. was (Author: jcamachorodriguez): AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split -for complex queries, this will just not work-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you have mentioned. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092822#comment-17092822 ] Jesus Camacho Rodriguez edited comment on HIVE-23291 at 4/26/20, 7:02 PM: -- AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split -for complex queries, this will just not work-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you have mentioned. was (Author: jcamachorodriguez): AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split \-for complex queries, this will just not work\-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you are mentioned. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092827#comment-17092827 ] Vineet Garg commented on HIVE-23291: [~jcamachorodriguez] Makes sense. +1. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?focusedWorklogId=427389&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427389 ] ASF GitHub Bot logged work on HIVE-23216: - Author: ASF GitHub Bot Created on: 26/Apr/20 18:46 Start Date: 26/Apr/20 18:46 Worklog Time Spent: 10m Work Description: jcamachor commented on a change in pull request #990: URL: https://github.com/apache/hive/pull/990#discussion_r415374316 ## File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java ## @@ -66,9 +66,7 @@ import org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy; import org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge; import org.apache.hadoop.hive.metastore.txn.TxnCommonUtils; -import org.apache.hadoop.hive.metastore.utils.JavaUtils; -import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils; -import org.apache.hadoop.hive.metastore.utils.SecurityUtils; +import org.apache.hadoop.hive.metastore.utils.*; Review comment: No worries. Fwiw, it can be disabled in the preferences 👍 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 427389) Time Spent: 1h 40m (was: 1.5h) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch, HIVE-23216.9.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092822#comment-17092822 ] Jesus Camacho Rodriguez edited comment on HIVE-23291 at 4/26/20, 6:45 PM: -- AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split \-for complex queries, this will just not work\-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you are mentioned. was (Author: jcamachorodriguez): AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split -for complex queries, this will just not work-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you are mentioned. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[ https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092822#comment-17092822 ] Jesus Camacho Rodriguez commented on HIVE-23291: AFAIK the only generic query that will not work is splitting the query based on offset and limit, since the syntax is different in the generic accessor. This is enabled only when quite tight conditions are met: user specifies number of partitions per read in their table, Calcite does not disable the split -for complex queries, this will just not work-, and user does not specify a partition column for a JDBC table. Fwiw that mode is bogus in any case because we cannot guarantee that records will come in some order for multiple executions of same query, e.g., reading from other distributed RDBMSs. Nevertheless, since the mode is there, probably we should include a new Hive accessor like you are mentioned. > Add Hive to DatabaseType in JDBC storage handler > > > Key: HIVE-23291 > URL: https://issues.apache.org/jira/browse/HIVE-23291 > Project: Hive > Issue Type: Improvement > Components: StorageHandler >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23291.patch > > > Inception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23216: --- Status: Open (was: Patch Available) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch, HIVE-23216.9.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23216: --- Attachment: HIVE-23216.9.patch > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch, HIVE-23216.9.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092817#comment-17092817 ] Jesus Camacho Rodriguez commented on HIVE-23216: Left a minor comment in latest patch concerning the imports. +1 (pending tests) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?focusedWorklogId=427388&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427388 ] ASF GitHub Bot logged work on HIVE-23216: - Author: ASF GitHub Bot Created on: 26/Apr/20 18:26 Start Date: 26/Apr/20 18:26 Worklog Time Spent: 10m Work Description: vineetgarg02 commented on a change in pull request #990: URL: https://github.com/apache/hive/pull/990#discussion_r415370305 ## File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java ## @@ -66,9 +66,7 @@ import org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy; import org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge; import org.apache.hadoop.hive.metastore.txn.TxnCommonUtils; -import org.apache.hadoop.hive.metastore.utils.JavaUtils; -import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils; -import org.apache.hadoop.hive.metastore.utils.SecurityUtils; +import org.apache.hadoop.hive.metastore.utils.*; Review comment: Yes let me fix this. This is an annoying side-effect of intellij :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 427388) Time Spent: 1.5h (was: 1h 20m) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?focusedWorklogId=427387&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427387 ] ASF GitHub Bot logged work on HIVE-23216: - Author: ASF GitHub Bot Created on: 26/Apr/20 18:26 Start Date: 26/Apr/20 18:26 Worklog Time Spent: 10m Work Description: vineetgarg02 commented on a change in pull request #990: URL: https://github.com/apache/hive/pull/990#discussion_r415370305 ## File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java ## @@ -66,9 +66,7 @@ import org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy; import org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge; import org.apache.hadoop.hive.metastore.txn.TxnCommonUtils; -import org.apache.hadoop.hive.metastore.utils.JavaUtils; -import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils; -import org.apache.hadoop.hive.metastore.utils.SecurityUtils; +import org.apache.hadoop.hive.metastore.utils.*; Review comment: Yes let me fix this. This is an annoying side-affect of intellij :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 427387) Time Spent: 1h 20m (was: 1h 10m) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?focusedWorklogId=427386&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427386 ] ASF GitHub Bot logged work on HIVE-23216: - Author: ASF GitHub Bot Created on: 26/Apr/20 18:24 Start Date: 26/Apr/20 18:24 Worklog Time Spent: 10m Work Description: jcamachor commented on a change in pull request #990: URL: https://github.com/apache/hive/pull/990#discussion_r415369776 ## File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java ## @@ -66,9 +66,7 @@ import org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy; import org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge; import org.apache.hadoop.hive.metastore.txn.TxnCommonUtils; -import org.apache.hadoop.hive.metastore.utils.JavaUtils; -import org.apache.hadoop.hive.metastore.utils.MetaStoreUtils; -import org.apache.hadoop.hive.metastore.utils.SecurityUtils; +import org.apache.hadoop.hive.metastore.utils.*; Review comment: nit. Can we expand this again? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 427386) Time Spent: 1h 10m (was: 1h) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[
https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092816#comment-17092816
]
Hive QA commented on HIVE-23207:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001244/HIVE-23207.9.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21956/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21956/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21956/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001244 - PreCommit-HIVE-Build
> Create integration tests for TxnManager for different rdbms metastores
> --
>
> Key: HIVE-23207
> URL: https://issues.apache.org/jira/browse/HIVE-23207
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Varga
>Assignee: Peter Varga
>Priority: Minor
> Attachments: HIVE-23207.1.patch, HIVE-23207.2.patch,
> HIVE-23207.3.patch, HIVE-23207.4.patch, HIVE-23207.5.patch,
> HIVE-23207.6.patch, HIVE-23207.7.patch, HIVE-23207.8.patch, HIVE-23207.9.patch
>
>
> Create an integration test suite that runs tests for TxnManager with the
> metastore configured to use different kind of RDBMS-s. Use the different
> DatabaseRule-s defined in the standalone-metastore for docker environments,
> and use the real init schema for every database type instead of the hardwired
> TxnDbUtil.prepDb.
> This test will be useful for easy manual validation of schema changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[
https://issues.apache.org/jira/browse/HIVE-23216?focusedWorklogId=427383&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427383
]
ASF GitHub Bot logged work on HIVE-23216:
-
Author: ASF GitHub Bot
Created on: 26/Apr/20 18:17
Start Date: 26/Apr/20 18:17
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on a change in pull request #990:
URL: https://github.com/apache/hive/pull/990#discussion_r415368466
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
##
@@ -1941,6 +1941,45 @@ public boolean listPartitionsByExpr(String catName,
String db_name, String tbl_n
return !r.isSetHasUnknownPartitions() || r.isHasUnknownPartitions(); //
Assume the worst.
}
+ @Override
+ public boolean listPartitionsSpecByExpr(String dbName, String tblName,
+ byte[] expr, String defaultPartName, short maxParts, List
result)
+ throws TException {
+return listPartitionsSpecByExpr(getDefaultCatalog(conf), dbName, tblName,
expr, defaultPartName,
+maxParts, result);
+ }
+
+ @Override
+ public boolean listPartitionsSpecByExpr(String catName, String dbName,
String tblName, byte[] expr,
+ String defaultPartitionName, short maxParts, List result)
+ throws TException {
+assert result != null;
+PartitionsByExprRequest req = new PartitionsByExprRequest(
+dbName, tblName, ByteBuffer.wrap(expr));
+if (defaultPartitionName != null) {
+ req.setDefaultPartitionName(defaultPartitionName);
+}
+if (maxParts >= 0) {
+ req.setMaxParts(maxParts);
+}
+PartitionsSpecByExprResult r;
+try {
+ r = client.get_partitions_spec_by_expr(req);
+} catch (TApplicationException te) {
+ if (te.getType() != TApplicationException.UNKNOWN_METHOD
+ && te.getType() != TApplicationException.WRONG_METHOD_NAME) {
+throw te;
+ }
+ throw new IncompatibleMetastoreException(
+ "Metastore doesn't support listPartitionsByExpr: " +
te.getMessage());
+}
+
+//TODO: filtering if client side filtering isClientFilterEnabled on
Review comment:
I looked into it and actually API already existed for filtering
PartitionSpec. I have update the code to do filtering now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 427383)
Time Spent: 1h (was: 50m)
> Add new api as replacement of get_partitions_by_expr to return PartitionSpec
> instead of Partitions
> --
>
> Key: HIVE-23216
> URL: https://issues.apache.org/jira/browse/HIVE-23216
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch,
> HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch,
> HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092813#comment-17092813 ] Vineet Garg commented on HIVE-23216: Latest patch (#8) address review comment to enable client side partition spec filtering. > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23216: --- Attachment: HIVE-23216.8.patch > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23216: --- Status: Open (was: Patch Available) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23216) Add new api as replacement of get_partitions_by_expr to return PartitionSpec instead of Partitions
[ https://issues.apache.org/jira/browse/HIVE-23216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-23216: --- Status: Patch Available (was: Open) > Add new api as replacement of get_partitions_by_expr to return PartitionSpec > instead of Partitions > -- > > Key: HIVE-23216 > URL: https://issues.apache.org/jira/browse/HIVE-23216 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 4.0.0 >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-23216.1.patch, HIVE-23216.2.patch, > HIVE-23216.3.patch, HIVE-23216.4.patch, HIVE-23216.5.patch, > HIVE-23216.6.patch, HIVE-23216.7.patch, HIVE-23216.8.patch > > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23291) Add Hive to DatabaseType in JDBC storage handler
[
https://issues.apache.org/jira/browse/HIVE-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092812#comment-17092812
]
Vineet Garg commented on HIVE-23291:
One question regarding this change - with this change Hive will use
{{GenericJdbcDatabaseAccessor}} instead of specialized database accessor. Is
{{GenericJdbcDatabaseAccessor}} capable of handling Hive jdbc?
> Add Hive to DatabaseType in JDBC storage handler
>
>
> Key: HIVE-23291
> URL: https://issues.apache.org/jira/browse/HIVE-23291
> Project: Hive
> Issue Type: Improvement
> Components: StorageHandler
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23291.patch
>
>
> Inception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23301) Optimising privilege synchroniser: UDF for updating privileges
[ https://issues.apache.org/jira/browse/HIVE-23301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri G updated HIVE-23301: -- Attachment: UDFSplitMapPrivs.patch Assignee: Simhadri G Status: Patch Available (was: Open) > Optimising privilege synchroniser: UDF for updating privileges > -- > > Key: HIVE-23301 > URL: https://issues.apache.org/jira/browse/HIVE-23301 > Project: Hive > Issue Type: Improvement > Components: Metastore, UDF >Affects Versions: 3.1.1 >Reporter: Simhadri G >Assignee: Simhadri G >Priority: Minor > Attachments: UDFSplitMapPrivs.patch, UDFSplitMapPrivs.patch > > > Privilege synchronizer pulls the list of databases, tables and columns from > the Hive Metastore. For each of these objects it fetches the privilege > information and invokes HMS API to refresh the privilege information in HMS. > The current UDF Maps a bit string to a privilege based on if the privilege > is granted or not. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23275) Represent UNBOUNDED in window functions in CBO correctly
[
https://issues.apache.org/jira/browse/HIVE-23275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23275:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Represent UNBOUNDED in window functions in CBO correctly
>
>
> Key: HIVE-23275
> URL: https://issues.apache.org/jira/browse/HIVE-23275
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23275.01.patch, HIVE-23275.01.patch,
> HIVE-23275.01.patch, HIVE-23275.patch, HIVE-23275.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently we use a bounded representation with bound set to
> Integer.MAX_VALUE, which works correctly since that is the Hive
> implementation. However, Calcite has a specific boundary class
> {{RexWindowBoundUnbounded}} that we should be using instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23262) Remove dependency on activemq
[ https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23262: Attachment: HIVE-23262.2.patch > Remove dependency on activemq > - > > Key: HIVE-23262 > URL: https://issues.apache.org/jira/browse/HIVE-23262 > Project: Hive > Issue Type: Improvement > Components: HCatalog >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23262.2.patch, HIVE-23262.patch > > > Activemq is a test dependency introduced to test message bus feature in > hcatalog. Even when it was introduced very first, there were concerns of > taking activemq as a dependency. > https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113 > AFAIK no one uses message bus feature of HCatalog. We can remove it > altogether. > As a first step removing tests for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23262) Remove dependency on activemq
[ https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23262: Status: Patch Available (was: Open) > Remove dependency on activemq > - > > Key: HIVE-23262 > URL: https://issues.apache.org/jira/browse/HIVE-23262 > Project: Hive > Issue Type: Improvement > Components: HCatalog >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23262.2.patch, HIVE-23262.patch > > > Activemq is a test dependency introduced to test message bus feature in > hcatalog. Even when it was introduced very first, there were concerns of > taking activemq as a dependency. > https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113 > AFAIK no one uses message bus feature of HCatalog. We can remove it > altogether. > As a first step removing tests for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23268) Eliminate beanutils transitive dependency
[ https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23268: Attachment: HIVE-23268.3.patch > Eliminate beanutils transitive dependency > - > > Key: HIVE-23268 > URL: https://issues.apache.org/jira/browse/HIVE-23268 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23268.2.patch, HIVE-23268.3.patch, HIVE-23268.patch > > > Transitively retrieved from hadoop-commons -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23262) Remove dependency on activemq
[ https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23262: Status: Open (was: Patch Available) > Remove dependency on activemq > - > > Key: HIVE-23262 > URL: https://issues.apache.org/jira/browse/HIVE-23262 > Project: Hive > Issue Type: Improvement > Components: HCatalog >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23262.patch > > > Activemq is a test dependency introduced to test message bus feature in > hcatalog. Even when it was introduced very first, there were concerns of > taking activemq as a dependency. > https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113 > AFAIK no one uses message bus feature of HCatalog. We can remove it > altogether. > As a first step removing tests for it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23268) Eliminate beanutils transitive dependency
[ https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23268: Status: Patch Available (was: Open) > Eliminate beanutils transitive dependency > - > > Key: HIVE-23268 > URL: https://issues.apache.org/jira/browse/HIVE-23268 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23268.2.patch, HIVE-23268.3.patch, HIVE-23268.patch > > > Transitively retrieved from hadoop-commons -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23268) Eliminate beanutils transitive dependency
[ https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23268: Status: Open (was: Patch Available) > Eliminate beanutils transitive dependency > - > > Key: HIVE-23268 > URL: https://issues.apache.org/jira/browse/HIVE-23268 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-23268.2.patch, HIVE-23268.patch > > > Transitively retrieved from hadoop-commons -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23287) Reduce dependency on icu4j
[ https://issues.apache.org/jira/browse/HIVE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23287: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Reduce dependency on icu4j > -- > > Key: HIVE-23287 > URL: https://issues.apache.org/jira/browse/HIVE-23287 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23287.2.patch, HIVE-23287.patch > > > Brought in transitively via druid. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[ https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23207: --- Status: In Progress (was: Patch Available) > Create integration tests for TxnManager for different rdbms metastores > -- > > Key: HIVE-23207 > URL: https://issues.apache.org/jira/browse/HIVE-23207 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23207.1.patch, HIVE-23207.2.patch, > HIVE-23207.3.patch, HIVE-23207.4.patch, HIVE-23207.5.patch, > HIVE-23207.6.patch, HIVE-23207.7.patch, HIVE-23207.8.patch, HIVE-23207.9.patch > > > Create an integration test suite that runs tests for TxnManager with the > metastore configured to use different kind of RDBMS-s. Use the different > DatabaseRule-s defined in the standalone-metastore for docker environments, > and use the real init schema for every database type instead of the hardwired > TxnDbUtil.prepDb. > This test will be useful for easy manual validation of schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[ https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23207: --- Status: Patch Available (was: In Progress) > Create integration tests for TxnManager for different rdbms metastores > -- > > Key: HIVE-23207 > URL: https://issues.apache.org/jira/browse/HIVE-23207 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23207.1.patch, HIVE-23207.2.patch, > HIVE-23207.3.patch, HIVE-23207.4.patch, HIVE-23207.5.patch, > HIVE-23207.6.patch, HIVE-23207.7.patch, HIVE-23207.8.patch, HIVE-23207.9.patch > > > Create an integration test suite that runs tests for TxnManager with the > metastore configured to use different kind of RDBMS-s. Use the different > DatabaseRule-s defined in the standalone-metastore for docker environments, > and use the real init schema for every database type instead of the hardwired > TxnDbUtil.prepDb. > This test will be useful for easy manual validation of schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23267) Reduce dependency on groovy
[ https://issues.apache.org/jira/browse/HIVE-23267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23267: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Reduce dependency on groovy > --- > > Key: HIVE-23267 > URL: https://issues.apache.org/jira/browse/HIVE-23267 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23267.2.patch, HIVE-23267.patch > > > Transitively pulled where its unneeded. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[ https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23207: --- Attachment: (was: HIVE-23207.9.patch) > Create integration tests for TxnManager for different rdbms metastores > -- > > Key: HIVE-23207 > URL: https://issues.apache.org/jira/browse/HIVE-23207 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23207.1.patch, HIVE-23207.2.patch, > HIVE-23207.3.patch, HIVE-23207.4.patch, HIVE-23207.5.patch, > HIVE-23207.6.patch, HIVE-23207.7.patch, HIVE-23207.8.patch, HIVE-23207.9.patch > > > Create an integration test suite that runs tests for TxnManager with the > metastore configured to use different kind of RDBMS-s. Use the different > DatabaseRule-s defined in the standalone-metastore for docker environments, > and use the real init schema for every database type instead of the hardwired > TxnDbUtil.prepDb. > This test will be useful for easy manual validation of schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[ https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Varga updated HIVE-23207: --- Attachment: HIVE-23207.9.patch > Create integration tests for TxnManager for different rdbms metastores > -- > > Key: HIVE-23207 > URL: https://issues.apache.org/jira/browse/HIVE-23207 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Minor > Attachments: HIVE-23207.1.patch, HIVE-23207.2.patch, > HIVE-23207.3.patch, HIVE-23207.4.patch, HIVE-23207.5.patch, > HIVE-23207.6.patch, HIVE-23207.7.patch, HIVE-23207.8.patch, HIVE-23207.9.patch > > > Create an integration test suite that runs tests for TxnManager with the > metastore configured to use different kind of RDBMS-s. Use the different > DatabaseRule-s defined in the standalone-metastore for docker environments, > and use the real init schema for every database type instead of the hardwired > TxnDbUtil.prepDb. > This test will be useful for easy manual validation of schema changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23278) Remove dependency on bouncycastle
[ https://issues.apache.org/jira/browse/HIVE-23278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23278: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. > Remove dependency on bouncycastle > - > > Key: HIVE-23278 > URL: https://issues.apache.org/jira/browse/HIVE-23278 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23278.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[
https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092784#comment-17092784
]
Hive QA commented on HIVE-23282:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001222/HIVE-23282.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 691 failed/errored test(s), 17143 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10]
(batchId=231)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11]
(batchId=231)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_12]
(batchId=231)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_part]
(batchId=237)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_nonstd_partitions_loc]
(batchId=237)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[parquet_format_part]
(batchId=237)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[parquet_nonstd_partitions_loc]
(batchId=237)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_format_part]
(batchId=237)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_nonstd_partitions_loc]
(batchId=237)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats3] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats4] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[archive_excludeHadoop20]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[archive_multi]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_8]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join14] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19] (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19_inclause]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join9] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_schema_evolution_native]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin10]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin11]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin12]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin13]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin8]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin9]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_1]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_3]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_4]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_5]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_8]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_const] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_input26] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_ppd_non_deterministic]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_outer_join_ppr]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_simple_select]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_union_view]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp]
(batchId=25)
org.apache.
[jira] [Commented] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[
https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092777#comment-17092777
]
Hive QA commented on HIVE-23282:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
22s{color} | {color:blue} standalone-metastore/metastore-server in master has
189 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 4 new + 601 unchanged - 3 fixed = 605 total (was 604) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21955/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21955/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21955/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
> -
>
> Key: HIVE-23282
> URL: https://issues.apache.org/jira/browse/HIVE-23282
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23282.1.patch, image-2020-04-23-14-07-06-077.png
>
>
> ObjectStore::getPartitionsByExprInternal internally uses Table information
> for getting partitionKeys, table, catalog name.
>
> For this, it ends up populating entire table data from DB (including skew
> column, parameters, sort, bucket cols etc). This makes it a lot more
> expensive call. It would be good to check if MTable itself can be used
> instead of Table.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327]
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669]
>
> !image-2020-04-23-14-07-06-077.png|width=665,height=592!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23244) Extract Create View analyzer from SemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092764#comment-17092764
]
Hive QA commented on HIVE-23244:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001220/HIVE-23244.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21954/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21954/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21954/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001220 - PreCommit-HIVE-Build
> Extract Create View analyzer from SemanticAnalyzer
> --
>
> Key: HIVE-23244
> URL: https://issues.apache.org/jira/browse/HIVE-23244
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23244.01.patch, HIVE-23244.02.patch,
> HIVE-23244.03.patch
>
>
> Create View commands are not queries, but commands which have queries as a
> part of them. Therefore a separate CreateViewAnalyzer is needed which uses
> SemanticAnalyer to analyze it's query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23244) Extract Create View analyzer from SemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092760#comment-17092760
]
Hive QA commented on HIVE-23244:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
11s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
55s{color} | {color:blue} parser in master has 3 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 2 new + 590 unchanged - 8
fixed = 592 total (was 598) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
2s{color} | {color:red} ql generated 1 new + 1529 unchanged - 1 fixed = 1530
total (was 1530) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | The field org.apache.hadoop.hive.ql.plan.LoadFileDesc.createViewDesc is
transient but isn't set by deserialization In LoadFileDesc.java:but isn't set
by deserialization In LoadFileDesc.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21954/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21954/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21954/yetus/new-findbugs-ql.html
|
| modules | C: common parser ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21954/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract Create View analyzer from SemanticAnalyzer
> --
>
> Key: HIVE-23244
> URL: https://issues.apache.org/jira/browse/HIVE-23244
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23244.01.patch, HIVE-23244.02.patch,
> HIVE-23244.03.patch
>
>
> Create View commands ar
[jira] [Commented] (HIVE-23294) Remove sync bottleneck in TezConfigurationFactory
[
https://issues.apache.org/jira/browse/HIVE-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092744#comment-17092744
]
Hive QA commented on HIVE-23294:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001197/HIVE-23294.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21953/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21953/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21953/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001197 - PreCommit-HIVE-Build
> Remove sync bottleneck in TezConfigurationFactory
> -
>
> Key: HIVE-23294
> URL: https://issues.apache.org/jira/browse/HIVE-23294
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23294.1.patch, HIVE-23294.2.patch, Screenshot
> 2020-04-24 at 1.53.20 PM.png
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java#L53]
> [https://github.com/apache/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L1628]
> It ends up locking for property names in the config. For short running
> queries with concurrency, this is an issue.
>
> !Screenshot 2020-04-24 at 1.53.20 PM.png|width=1086,height=459!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23294) Remove sync bottleneck in TezConfigurationFactory
[
https://issues.apache.org/jira/browse/HIVE-23294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092735#comment-17092735
]
Hive QA commented on HIVE-23294:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 4 unchanged - 0 fixed
= 5 total (was 4) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21953/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21953/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21953/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove sync bottleneck in TezConfigurationFactory
> -
>
> Key: HIVE-23294
> URL: https://issues.apache.org/jira/browse/HIVE-23294
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23294.1.patch, HIVE-23294.2.patch, Screenshot
> 2020-04-24 at 1.53.20 PM.png
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java#L53]
> [https://github.com/apache/hadoop/blob/master/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L1628]
> It ends up locking for property names in the config. For short running
> queries with concurrency, this is an issue.
>
> !Screenshot 2020-04-24 at 1.53.20 PM.png|width=1086,height=459!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092723#comment-17092723
]
Hive QA commented on HIVE-22737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001186/HIVE-22737.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17115 tests
executed
*Failed tests:*
{noformat}
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=14)
[autogen_colalias.q,nonreserved_keywords_input37.q,cte_join.q,avro_timestamp.q,groupby_ppr.q,correlated_join_keys.q,ppr_pushdown3.q,masking_5.q,input30.q,spark_explain_groupbyshuffle.q,groupby2_map.q,show_create_table_delimited.q,rcfile_null_value.q,join_star.q,columnStatsUpdateForStatsOptimizer_2.q,udf_second.q,skewjoin_union_remove_2.q,udf_size.q,bucketmapjoin_negative3.q,union32.q,timestamp.q,flatten_and_or.q,constant_prop_2.q,parquet_vectorization_17.q,smb_mapjoin_47.q,fold_when.q,udf_mask_hash.q,join18_multi_distinct.q,udf_bitwise_shiftrightunsigned.q,sample8.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_substr]
(batchId=85)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[udf_substring]
(batchId=90)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_database_cascade]
(batchId=45)
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testPermFunc (batchId=219)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.org.apache.hive.jdbc.TestJdbcWithMiniHS2
(batchId=219)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testPermFunc (batchId=219)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21952/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21952/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21952/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001186 - PreCommit-HIVE-Build
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092714#comment-17092714
]
Hive QA commented on HIVE-22737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 15 new + 27 unchanged - 0
fixed = 42 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21952/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21952/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21952/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23287) Reduce dependency on icu4j
[
https://issues.apache.org/jira/browse/HIVE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092704#comment-17092704
]
Hive QA commented on HIVE-23287:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001182/HIVE-23287.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21951/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21951/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21951/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001182 - PreCommit-HIVE-Build
> Reduce dependency on icu4j
> --
>
> Key: HIVE-23287
> URL: https://issues.apache.org/jira/browse/HIVE-23287
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23287.2.patch, HIVE-23287.patch
>
>
> Brought in transitively via druid.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23287) Reduce dependency on icu4j
[
https://issues.apache.org/jira/browse/HIVE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092686#comment-17092686
]
Hive QA commented on HIVE-23287:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21951/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21951/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce dependency on icu4j
> --
>
> Key: HIVE-23287
> URL: https://issues.apache.org/jira/browse/HIVE-23287
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23287.2.patch, HIVE-23287.patch
>
>
> Brought in transitively via druid.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23262) Remove dependency on activemq
[
https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092684#comment-17092684
]
Hive QA commented on HIVE-23262:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
52s{color} | {color:red} branch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} hcatalog: The patch generated 0 new + 0 unchanged -
9 fixed = 0 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch server-extensions passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
9s{color} | {color:green} root: The patch generated 0 new + 0 unchanged - 9
fixed = 0 total (was 9) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
15s{color} | {color:red} patch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red} patch/hcatalog/server-extensions cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 10m
43s{color} | {color:red} root in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21950/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21950/yetus/branch-findbugs-hcatalog.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21950/yetus/patch-findbugs-hcatalog.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21950/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21950/yetus/patch-javadoc-root.txt
|
| modules | C: hcatalog hcatalog/server-extensions . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21950/yetus.txt |
| Powered by | Apache Yetusht
[jira] [Commented] (HIVE-23262) Remove dependency on activemq
[
https://issues.apache.org/jira/browse/HIVE-23262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092678#comment-17092678
]
Hive QA commented on HIVE-23262:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000643/HIVE-23262.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17138 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithAllAndBinary.org.apache.hive.service.cli.thrift.TestThriftCLIServiceWithAllAndBinary
(batchId=209)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21950/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21950/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21950/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000643 - PreCommit-HIVE-Build
> Remove dependency on activemq
> -
>
> Key: HIVE-23262
> URL: https://issues.apache.org/jira/browse/HIVE-23262
> Project: Hive
> Issue Type: Improvement
> Components: HCatalog
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23262.patch
>
>
> Activemq is a test dependency introduced to test message bus feature in
> hcatalog. Even when it was introduced very first, there were concerns of
> taking activemq as a dependency.
> https://issues.apache.org/jira/browse/HCATALOG-3?focusedCommentId=13035113&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13035113
> AFAIK no one uses message bus feature of HCatalog. We can remove it
> altogether.
> As a first step removing tests for it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23267) Reduce dependency on groovy
[
https://issues.apache.org/jira/browse/HIVE-23267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092653#comment-17092653
]
Hive QA commented on HIVE-23267:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000887/HIVE-23267.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17143 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21949/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21949/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21949/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000887 - PreCommit-HIVE-Build
> Reduce dependency on groovy
> ---
>
> Key: HIVE-23267
> URL: https://issues.apache.org/jira/browse/HIVE-23267
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23267.2.patch, HIVE-23267.patch
>
>
> Transitively pulled where its unneeded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23282: Assignee: Rajesh Balamohan Status: Patch Available (was: Open) > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Major > Attachments: HIVE-23282.1.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092649#comment-17092649 ] Rajesh Balamohan edited comment on HIVE-23282 at 4/26/20, 10:38 AM: [~ashutoshc]: Yes, only "PartitionKeys" are needed in sqlfilter. Posted the patch and the RB link. was (Author: rajesh.balamohan): [~ashutoshc]: PartitionKeys are needed in sqlfilter. Posted the patch and the RB link. > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Priority: Major > Attachments: HIVE-23282.1.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23282: Attachment: HIVE-23282.1.patch > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Priority: Major > Attachments: HIVE-23282.1.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23282) Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal
[ https://issues.apache.org/jira/browse/HIVE-23282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092649#comment-17092649 ] Rajesh Balamohan commented on HIVE-23282: - [~ashutoshc]: PartitionKeys are needed in sqlfilter. Posted the patch and the RB link. > Reduce number of DB calls in ObjectStore::getPartitionsByExprInternal > - > > Key: HIVE-23282 > URL: https://issues.apache.org/jira/browse/HIVE-23282 > Project: Hive > Issue Type: Improvement > Components: Metastore >Reporter: Rajesh Balamohan >Priority: Major > Attachments: HIVE-23282.1.patch, image-2020-04-23-14-07-06-077.png > > > ObjectStore::getPartitionsByExprInternal internally uses Table information > for getting partitionKeys, table, catalog name. > > For this, it ends up populating entire table data from DB (including skew > column, parameters, sort, bucket cols etc). This makes it a lot more > expensive call. It would be good to check if MTable itself can be used > instead of Table. > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3327] > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3669] > > !image-2020-04-23-14-07-06-077.png|width=665,height=592! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23268) Eliminate beanutils transitive dependency
[
https://issues.apache.org/jira/browse/HIVE-23268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092637#comment-17092637
]
Hive QA commented on HIVE-23268:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
13s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 13s{color}
| {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 1m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 14m
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21948/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21948/yetus/patch-compile-ql.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21948/yetus/patch-compile-ql.txt
|
| modules | C: storage-api shims/common shims/0.23 shims/scheduler
standalone-metastore standalone-metastore/metastore-common common serde
metastore llap-common llap-client llap-tez
standalone-metastore/metastore-server ql llap-server service accumulo-handler
jdbc beeline cli contrib druid-handler hbase-handler hcatalog hcatalog/core
hcatalog/hcatalog-pig-adapter hcatalog/server-extensions
hcatalog/webhcat/java-client hcatalog/webhcat/svr hcatalog/streaming hplsql
streaming llap-ext-client kryo-registrator kudu-handler kafka-handler
standalone-metastore/metastore-tools/metastore-benchmarks
upgrade-acid/pre-upgrade U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21948/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Eliminate beanutils transitive dependency
> -
>
> Key: HIVE-23268
> URL: https://issues.apache.org/jira/browse/HIVE-23268
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23268.2.patch, HIVE-23268.patch
>
>
> Transitively retrieved from hadoop-commons
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23267) Reduce dependency on groovy
[
https://issues.apache.org/jira/browse/HIVE-23267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092634#comment-17092634
]
Hive QA commented on HIVE-23267:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21949/dev-support/hive-personality.sh
|
| git revision | master / abbe72e |
| Default Java | 1.8.0_111 |
| modules | C: hcatalog hcatalog/hcatalog-pig-adapter upgrade-acid/pre-upgrade
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21949/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce dependency on groovy
> ---
>
> Key: HIVE-23267
> URL: https://issues.apache.org/jira/browse/HIVE-23267
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: HIVE-23267.2.patch, HIVE-23267.patch
>
>
> Transitively pulled where its unneeded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23244) Extract Create View analyzer from SemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23244: -- Attachment: (was: HIVE-23244.03.patch) > Extract Create View analyzer from SemanticAnalyzer > -- > > Key: HIVE-23244 > URL: https://issues.apache.org/jira/browse/HIVE-23244 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23244.01.patch, HIVE-23244.02.patch, > HIVE-23244.03.patch > > > Create View commands are not queries, but commands which have queries as a > part of them. Therefore a separate CreateViewAnalyzer is needed which uses > SemanticAnalyer to analyze it's query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
