[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024928#comment-17024928
]
Hive QA commented on HIVE-22736:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991975/HIVE-22736.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17966 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestGetPartitionsUsingProjectionAndFilterSpecs.testGetPartitionsUsingValues
(batchId=231)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20374/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20374/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20374/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991975 - PreCommit-HIVE-Build
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024915#comment-17024915
]
Hive QA commented on HIVE-22736:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
2s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
19s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
21s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
34s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} shims/common: The patch generated 0 new + 93
unchanged - 1 fixed = 93 total (was 94) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
|
[jira] [Updated] (HIVE-22739) Schematool should check if upgradeFrom version is identical to current DB schema version
[ https://issues.apache.org/jira/browse/HIVE-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22739: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been pushed to master. Thank you for your contribution [~afan] > Schematool should check if upgradeFrom version is identical to current DB > schema version > > > Key: HIVE-22739 > URL: https://issues.apache.org/jira/browse/HIVE-22739 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Alice Fan >Assignee: Alice Fan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22739.1.patch > > > Schematool should check if upgradeFrom version is identical to current DB > schema version. > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/tools/schematool/SchemaToolTaskUpgrade.java#L41 > We should check if the user input fromVersion is identical to current DB > version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Attachment: HIVE-22538.8.patch
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch,
> HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Status: Patch Available (was: Open)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch,
> HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22538:
--
Status: Open (was: Patch Available)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch,
> HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[ https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024896#comment-17024896 ] Gopal Vijayaraghavan commented on HIVE-22784: - Looks like rightTrimAndTruncate has an off-by-1 error in it? > Boundary condition to check if there is nothing to truncate in StringExpr > functions > --- > > Key: HIVE-22784 > URL: https://issues.apache.org/jira/browse/HIVE-22784 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22784.1.patch > > > Add a boundary condition to check if there is nothing to truncate in > StringExpr functions > truncate() > rightTrimAndTruncate() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Attachment: HIVE-22736.patch Status: Patch Available (was: In Progress) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22736) Support replication across multiple encryption zones
[ https://issues.apache.org/jira/browse/HIVE-22736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22736: --- Status: In Progress (was: Patch Available) > Support replication across multiple encryption zones > > > Key: HIVE-22736 > URL: https://issues.apache.org/jira/browse/HIVE-22736 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, > HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378047
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 05:53
Start Date: 28/Jan/20 05:53
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371620277
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/ReplChangeManager.java
##
@@ -150,16 +152,23 @@ private ReplChangeManager(Configuration conf) throws
MetaException {
if (MetastoreConf.getBoolVar(conf, ConfVars.REPLCMENABLED)) {
ReplChangeManager.enabled = true;
ReplChangeManager.conf = conf;
-
+ cmRootDir = MetastoreConf.getVar(conf, ConfVars.REPLCMDIR);
+ encryptedCmRootDir = MetastoreConf.getVar(conf,
ConfVars.REPLCMENCRYPTEDDIR);
//Create default cm root
- Path cmroot = new Path(MetastoreConf.getVar(conf,
ConfVars.REPLCMDIR));
+ Path cmroot = new Path(cmRootDir);
+ HdfsEncryptionShim pathEncryptionShim = hadoopShims
+ .createHdfsEncryptionShim(cmroot.getFileSystem(conf), conf);
+ if (pathEncryptionShim.isPathEncrypted(cmroot)) {
+LOG.warn(ConfVars.REPLCMDIR + " should not be encrypted. To pass
cm dir for encrypted path use "
Review comment:
Yes this case needs to be handled
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378047)
Time Spent: 2h 10m (was: 2h)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378048
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 05:53
Start Date: 28/Jan/20 05:53
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371620301
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/ReplChangeManager.java
##
@@ -442,32 +451,39 @@ public static boolean isCMFileUri(Path fromPath) {
public void run() {
try {
LOG.info("CMClearer started");
-
-long now = System.currentTimeMillis();
-FileSystem fs = cmroot.getFileSystem(conf);
-FileStatus[] files = fs.listStatus(cmroot);
-
-for (FileStatus file : files) {
- long modifiedTime = file.getModificationTime();
- if (now - modifiedTime > secRetain*1000) {
-try {
- if
(fs.getXAttrs(file.getPath()).containsKey(REMAIN_IN_TRASH_TAG)) {
-boolean succ = Trash.moveToAppropriateTrash(fs,
file.getPath(), conf);
-if (succ) {
- LOG.debug("Move " + file.toString() + " to trash");
-} else {
- LOG.warn("Fail to move " + file.toString() + " to trash");
-}
- } else {
-boolean succ = fs.delete(file.getPath(), false);
-if (succ) {
- LOG.debug("Remove " + file.toString());
+for (String encryptionZone : encryptionZones) {
+ Path cmroot;
Review comment:
Yes
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378048)
Time Spent: 2h 20m (was: 2h 10m)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024864#comment-17024864
]
Hive QA commented on HIVE-22777:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991972/HIVE-22777.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20373/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20373/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20373/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-28 04:52:10.159
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20373/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-28 04:52:10.162
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 6a5c0cd HIVE-22753: Fix gradual mem leak: Operationlog related
appenders should be cleared up on errors (Rajesh Balamohan, reviewed by
Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6a5c0cd HIVE-22753: Fix gradual mem leak: Operationlog related
appenders should be cleared up on errors (Rajesh Balamohan, reviewed by
Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-28 04:52:11.235
+ rm -rf ../yetus_PreCommit-HIVE-Build-20373
+ mkdir ../yetus_PreCommit-HIVE-Build-20373
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20373
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20373/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/itests/src/test/resources/testconfiguration.properties: does not exist
in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedDynPartitionOptimizer.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:
does not exist in index
error: a/ql/src/test/queries/clientpositive/dynpart_sort_opt_bucketing.q: does
not exist in index
error: a/ql/src/test/results/clientpositive/bucket1.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/bucket2.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/bucket3.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/bucketsortoptimize_insert_3.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/dynpart_sort_opt_bucketing.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket2.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/bucket3.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/bucket4.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/bucket5.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/bucket_many.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_num_reducers.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_num_reducers2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/check_constraint.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/default_constraint.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/disable_merge_for_bucketing.q.out:
does not exist in
[jira] [Commented] (HIVE-22786) Agg with distinct can be optimised in HASH mode
[
https://issues.apache.org/jira/browse/HIVE-22786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024861#comment-17024861
]
Hive QA commented on HIVE-22786:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991971/HIVE-22786.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 38 failed/errored test(s), 17953 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join18] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join18_multi_distinct]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_stats]
(batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby8_map_skew]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby9] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_distinct_samekey]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_multi_insert_common_distinct]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_nocolumnalign]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_position]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join18] (batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join18_multi_distinct]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_distinct]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_gby3]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup4] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup4_multi_distinct]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_count]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_limit]
(batchId=41)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[count]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[metadataonly1]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_union_multiinsert]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_count]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_cube1]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_rollup1]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_sort_11]
(batchId=188)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_distinct_gby]
(batchId=185)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[count] (batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby8_map_skew]
(batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby9]
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_multi_insert_common_distinct]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_position]
(batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[multi_insert_gby3]
(batchId=151)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[nullgroup4]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[nullgroup4_multi_distinct]
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union17]
(batchId=149)
org.apache.hadoop.hive.ql.TestTxnAddPartition.addPartition (batchId=355)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20372/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20372/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20372/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 38 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991971 - PreCommit-HIVE-Build
> Agg with distinct can be optimised in HASH mode
> ---
>
> Key: HIVE-22786
> URL: https://issues.apache.org/jira/browse/HIVE-22786
> Project: Hive
>
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378034=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378034
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 04:29
Start Date: 28/Jan/20 04:29
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371604799
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -48,6 +48,8 @@ public void setUp() throws Exception {
MetastoreConf.setBoolVar(conf, ConfVars.METASTORE_CACHE_CAN_USE_EVENT,
true);
MetastoreConf.setBoolVar(conf, ConfVars.HIVE_TXN_STATS_ENABLED, true);
MetastoreConf.setBoolVar(conf, ConfVars.AGGREGATE_STATS_CACHE_ENABLED,
false);
+MetastoreConf.setBoolVar(conf, ConfVars.REPLCMENABLED, true);
+MetastoreConf.setVar(conf, ConfVars.REPLCMDIR, "cmroot");
Review comment:
REPLCMDIR default value is /user/${system:user.name}/repl/functions/
UTs are not able to resolve ${system:user.name}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378034)
Time Spent: 2h (was: 1h 50m)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Status: Patch Available (was: Open)
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Status: Open (was: Patch Available)
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22777) Sorted dynamic partition optimization doesn't work if plan require implicit cast
[
https://issues.apache.org/jira/browse/HIVE-22777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22777:
---
Attachment: HIVE-22777.2.patch
> Sorted dynamic partition optimization doesn't work if plan require implicit
> cast
>
>
> Key: HIVE-22777
> URL: https://issues.apache.org/jira/browse/HIVE-22777
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22777.1.patch, HIVE-22777.2.patch
>
>
> *Repro*
> set hive.stats.autogather=false;
> set hive.optimize.sort.dynamic.partition.threshold=1;
> set hive.optimize.bucketingsorting = true;
> {code:sql}
> drop table if exists t1_staging;
> create table t1_staging(
>a string,
>b int,
>c int,
>d string)
> partitioned by (e decimal(18,0))
> clustered by(a)
> into 256 buckets STORED AS TEXTFILE;
> load data local inpath '../../data/files/sortdp/00_0' overwrite into
> table t1_staging partition (e=100);
> drop table t1_n147;
> create table t1_n147(
> a string,
> b decimal(6,0),
> c int,
> d string)
> partitioned by (e decimal(3,0))
> clustered by(a,b)
> into 10 buckets STORED AS ORC TBLPROPERTIES ('transactional'='true');
> set hive.stats.autogather=false;
> set hive.optimize.bucketingsorting = true;
> explain insert overwrite table t1_n147 partition(e) select a,b,c,d,e from
> t1_staging;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22786) Agg with distinct can be optimised in HASH mode
[
https://issues.apache.org/jira/browse/HIVE-22786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024850#comment-17024850
]
Hive QA commented on HIVE-22786:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
0s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20372/dev-support/hive-personality.sh
|
| git revision | master / 6a5c0cd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20372/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Agg with distinct can be optimised in HASH mode
> ---
>
> Key: HIVE-22786
> URL: https://issues.apache.org/jira/browse/HIVE-22786
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22786.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378029=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378029
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 03:15
Start Date: 28/Jan/20 03:15
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371589321
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/ReplChangeManager.java
##
@@ -442,32 +451,39 @@ public static boolean isCMFileUri(Path fromPath) {
public void run() {
try {
LOG.info("CMClearer started");
-
-long now = System.currentTimeMillis();
-FileSystem fs = cmroot.getFileSystem(conf);
-FileStatus[] files = fs.listStatus(cmroot);
-
-for (FileStatus file : files) {
- long modifiedTime = file.getModificationTime();
- if (now - modifiedTime > secRetain*1000) {
-try {
- if
(fs.getXAttrs(file.getPath()).containsKey(REMAIN_IN_TRASH_TAG)) {
-boolean succ = Trash.moveToAppropriateTrash(fs,
file.getPath(), conf);
-if (succ) {
- LOG.debug("Move " + file.toString() + " to trash");
-} else {
- LOG.warn("Fail to move " + file.toString() + " to trash");
-}
- } else {
-boolean succ = fs.delete(file.getPath(), false);
-if (succ) {
- LOG.debug("Remove " + file.toString());
+for (String encryptionZone : encryptionZones) {
+ Path cmroot;
Review comment:
Do we have any existing test case for CMCleaner ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378029)
Time Spent: 1h 50m (was: 1h 40m)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378026=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378026
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 03:15
Start Date: 28/Jan/20 03:15
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371589875
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -2836,9 +2836,14 @@ private boolean checkTableDataShouldBeDeleted(Table
tbl, boolean deleteData) {
*data from warehouse
* @param shouldEnableCm If cm should be enabled
*/
-private void deleteTableData(Path tablePath, boolean ifPurge, boolean
shouldEnableCm) throws MetaException {
+private void deleteTableData(Path tablePath, boolean ifPurge, boolean
shouldEnableCm) {
if (tablePath != null) {
-wh.deleteDir(tablePath, true, ifPurge, shouldEnableCm);
+try {
+ wh.deleteDir(tablePath, true, ifPurge, shouldEnableCm);
+} catch (MetaException e) {
Review comment:
shall we change it to any exception ..including run time exception ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378026)
Time Spent: 1h 40m (was: 1.5h)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378025
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 03:15
Start Date: 28/Jan/20 03:15
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371588458
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -48,6 +48,8 @@ public void setUp() throws Exception {
MetastoreConf.setBoolVar(conf, ConfVars.METASTORE_CACHE_CAN_USE_EVENT,
true);
MetastoreConf.setBoolVar(conf, ConfVars.HIVE_TXN_STATS_ENABLED, true);
MetastoreConf.setBoolVar(conf, ConfVars.AGGREGATE_STATS_CACHE_ENABLED,
false);
+MetastoreConf.setBoolVar(conf, ConfVars.REPLCMENABLED, true);
+MetastoreConf.setVar(conf, ConfVars.REPLCMDIR, "cmroot");
Review comment:
why we need to set this ? is it not working with default value ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378025)
Time Spent: 1.5h (was: 1h 20m)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378028=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378028
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 03:15
Start Date: 28/Jan/20 03:15
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371589038
##
File path:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/ReplChangeManager.java
##
@@ -150,16 +152,23 @@ private ReplChangeManager(Configuration conf) throws
MetaException {
if (MetastoreConf.getBoolVar(conf, ConfVars.REPLCMENABLED)) {
ReplChangeManager.enabled = true;
ReplChangeManager.conf = conf;
-
+ cmRootDir = MetastoreConf.getVar(conf, ConfVars.REPLCMDIR);
+ encryptedCmRootDir = MetastoreConf.getVar(conf,
ConfVars.REPLCMENCRYPTEDDIR);
//Create default cm root
- Path cmroot = new Path(MetastoreConf.getVar(conf,
ConfVars.REPLCMDIR));
+ Path cmroot = new Path(cmRootDir);
+ HdfsEncryptionShim pathEncryptionShim = hadoopShims
+ .createHdfsEncryptionShim(cmroot.getFileSystem(conf), conf);
+ if (pathEncryptionShim.isPathEncrypted(cmroot)) {
+LOG.warn(ConfVars.REPLCMDIR + " should not be encrypted. To pass
cm dir for encrypted path use "
Review comment:
It will create issue if user has a single encryption zone setup and he has
created a directory with permission to hive as cmroot. And user has not given
the permission to create directory on encryption zone root to hive. In that
case we will try to create .cmroot (the default value) and will fail. Should we
consider this case ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378028)
Time Spent: 1h 50m (was: 1h 40m)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22736) Support replication across multiple encryption zones
[
https://issues.apache.org/jira/browse/HIVE-22736?focusedWorklogId=378027=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-378027
]
ASF GitHub Bot logged work on HIVE-22736:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 03:15
Start Date: 28/Jan/20 03:15
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #883: HIVE-22736
Support multiple encryption zones in replication
URL: https://github.com/apache/hive/pull/883#discussion_r371588545
##
File path:
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniHS2.java
##
@@ -104,8 +104,11 @@
public static void setupBeforeClass() throws Exception {
MiniHS2.cleanupLocalDir();
HiveConf conf = new HiveConf();
+ // conf.set(HiveConf.ConfVars.REPLCMDIR.varname, "hdfs://cmroot");
+ // conf.set(ConfVars.REPLCMENABLED.varname, "true");
Review comment:
remove the lines if not required.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 378027)
Time Spent: 1h 40m (was: 1.5h)
> Support replication across multiple encryption zones
>
>
> Key: HIVE-22736
> URL: https://issues.apache.org/jira/browse/HIVE-22736
> Project: Hive
> Issue Type: Task
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch, HIVE-22736.patch,
> HIVE-22736.patch, HIVE-22736.patch
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22753: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~ashutoshc], [~zchovan]. Committed to master. > Fix gradual mem leak: Operationlog related appenders should be cleared up on > errors > > > Key: HIVE-22753 > URL: https://issues.apache.org/jira/browse/HIVE-22753 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch, > HIVE-22753.3.patch, HIVE-22753.4.patch, image-2020-01-21-11-14-37-911.png, > image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png > > > In case of exception in SQLOperation, operational log does not get cleared > up. This causes gradual build up of HushableRandomAccessFileAppender causing > HS2 to OOM after some time. > !image-2020-01-21-11-14-37-911.png|width=431,height=267! > > Allocation tree > !image-2020-01-21-11-18-37-294.png|width=425,height=178! > > Prod instance mem > !image-2020-01-21-11-17-59-279.png|width=698,height=209! > > Each HushableRandomAccessFileAppender holds internal ref to > RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem > leak. > Related ticket: HIVE-18820 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22786) Agg with distinct can be optimised in HASH mode
[ https://issues.apache.org/jira/browse/HIVE-22786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22786: Attachment: HIVE-22786.1.patch > Agg with distinct can be optimised in HASH mode > --- > > Key: HIVE-22786 > URL: https://issues.apache.org/jira/browse/HIVE-22786 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22786.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22786) Agg with distinct can be optimised in HASH mode
[ https://issues.apache.org/jira/browse/HIVE-22786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22786: Status: Patch Available (was: Open) > Agg with distinct can be optimised in HASH mode > --- > > Key: HIVE-22786 > URL: https://issues.apache.org/jira/browse/HIVE-22786 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22786.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22779) Hive build fails with Spark 3
[
https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024816#comment-17024816
]
Hive QA commented on HIVE-22779:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991962/HIVE-22779.001.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20371/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20371/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20371/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-28 01:59:12.049
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20371/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-28 01:59:12.052
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at b7650eb HIVE-22366: Multiple metastore calls for same table and
constraints during planning (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing ${project.basedir}/
Removing itests/${project.basedir}/
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at b7650eb HIVE-22366: Multiple metastore calls for same table and
constraints during planning (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-28 01:59:12.923
+ rm -rf ../yetus_PreCommit-HIVE-Build-20371
+ mkdir ../yetus_PreCommit-HIVE-Build-20371
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20371
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20371/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java: does
not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java:
does not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/counter/SparkCounter.java:
does not exist in index
Trying to apply the patch with -p1
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java:26
Falling back to three-way merge...
Applied patch to
'ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java' with
conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java:26
Falling back to three-way merge...
Applied patch to
'ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java' with
conflicts.
U ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20371
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991962 - PreCommit-HIVE-Build
> Hive build fails with Spark 3
> -
>
> Key: HIVE-22779
> URL: https://issues.apache.org/jira/browse/HIVE-22779
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 3.1.2
>Reporter: Igor Dvorzhak
>Priority: Major
> Attachments: HIVE-22779.001.patch
>
>
> Hive build with Spark 3 fails because of deletion of few deprecated APIs in
> Spark 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[
https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024814#comment-17024814
]
Hive QA commented on HIVE-22784:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991958/HIVE-22784.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17953 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part_llap_io]
(batchId=189)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_table]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_table_llap_io]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part_llap_io]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_table]
(batchId=190)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_table_llap_io]
(batchId=181)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorCastStatement.testString
(batchId=346)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20370/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20370/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20370/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991958 - PreCommit-HIVE-Build
> Boundary condition to check if there is nothing to truncate in StringExpr
> functions
> ---
>
> Key: HIVE-22784
> URL: https://issues.apache.org/jira/browse/HIVE-22784
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22784.1.patch
>
>
> Add a boundary condition to check if there is nothing to truncate in
> StringExpr functions
> truncate()
> rightTrimAndTruncate()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377977=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377977 ] ASF GitHub Bot logged work on HIVE-22515: - Author: ASF GitHub Bot Created on: 28/Jan/20 01:28 Start Date: 28/Jan/20 01:28 Worklog Time Spent: 10m Work Description: ramesh0201 commented on issue #880: HIVE-22515 Support cast to decimal64 in Vectorization URL: https://github.com/apache/hive/pull/880#issuecomment-579036627 Whenever we have an expression like `long column decimal column`, we always cast long to decimal even if the decimal column can be represented as decimal64. This prevents us from using Deicmal64 class of operations that will be faster than the equivalent Decimal class of operations. This Jira will implements cast from long to decimal64 that can help in more optimizations in the decimal64 operations. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377977) Time Spent: 1h 10m (was: 1h) > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024796#comment-17024796 ] Ramesh Kumar Thangarajan commented on HIVE-22515: - Whenever we have an expression like `long column decimal column`, we always cast long to decimal even if the decimal column can be represented as decimal64. This prevents us from using Deicmal64 class of operations that will be faster than the equivalent Decimal class of operations. This Jira will implements cast from long to decimal64 that can help in more optimizations in the decimal64 operations. > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377961=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377961
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 01:14
Start Date: 28/Jan/20 01:14
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371568274
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FuncLongToDecimal64.java
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import java.util.Arrays;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.VectorExpressionDescriptor;
+import
org.apache.hadoop.hive.ql.exec.vector.VectorizedExpressionsSupportDecimal64;
+import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo;
+
+/**
+ * This is a superclass for unary long functions and expressions returning
decimals that
+ * operate directly on the input and set the output.
+ */
+@VectorizedExpressionsSupportDecimal64()
+public abstract class FuncLongToDecimal64 extends VectorExpression {
+ private static final long serialVersionUID = 1L;
+ private final int inputColumn;
+
+ private static final long[] powerOfTenTable = {
+1L, // 0
+10L,
+100L,
+1_000L,
+10_000L,
+100_000L,
+1_000_000L,
+10_000_000L,
+100_000_000L, // 8
+1_000_000_000L,
+10_000_000_000L,
+100_000_000_000L,
+1_000_000_000_000L,
+10_000_000_000_000L,
+100_000_000_000_000L,
+1_000_000_000_000_000L,
+10_000_000_000_000_000L, // 16
+100_000_000_000_000_000L,
+1_000_000_000_000_000_000L, // 18
+ };
+
+ public FuncLongToDecimal64(int inputColumn, int outputColumnNum) {
+super(outputColumnNum);
+this.inputColumn = inputColumn;
+ }
+
+ public FuncLongToDecimal64() {
+super();
+
+// Dummy final assignments.
+inputColumn = -1;
+ }
+
+ abstract protected void func(Decimal64ColumnVector outputColVector,
LongColumnVector inputColVector, int i,
+ long scaleFactor);
+
+ @Override
+ public void evaluate(VectorizedRowBatch batch) throws HiveException {
+
+if (childExpressions != null) {
+ super.evaluateChildren(batch);
+}
+
+LongColumnVector inputColVector = (LongColumnVector)
batch.cols[inputColumn];
+int[] sel = batch.selected;
+int n = batch.size;
+Decimal64ColumnVector outputColVector = (Decimal64ColumnVector)
batch.cols[outputColumnNum];
+int outputScale = ((DecimalTypeInfo) outputTypeInfo).scale();
+final long scaleFactor = powerOfTenTable[outputScale];
Review comment:
Yes, before we create cast expression, we verify if the output.precision +
output.scale <= 18(to avoid intermediate overflow) and hence implicitly
outputscale will be less than or equal to 18.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377961)
Time Spent: 40m (was: 0.5h)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels:
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377962=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377962
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 01:14
Start Date: 28/Jan/20 01:14
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371568368
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastLongToDecimal64.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+
+/**
+ * To be used to cast long and boolean to decimal.
+ * This works for boolean too because boolean is encoded as 0
+ * for false and 1 for true.
+ */
+public class CastLongToDecimal64 extends FuncLongToDecimal64 {
Review comment:
Thank you, I have addressed this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377962)
Time Spent: 50m (was: 40m)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377963=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377963
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 28/Jan/20 01:14
Start Date: 28/Jan/20 01:14
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371568394
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FuncLongToDecimal64.java
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import java.util.Arrays;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.VectorExpressionDescriptor;
+import
org.apache.hadoop.hive.ql.exec.vector.VectorizedExpressionsSupportDecimal64;
+import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo;
+
+/**
+ * This is a superclass for unary long functions and expressions returning
decimals that
+ * operate directly on the input and set the output.
+ */
+@VectorizedExpressionsSupportDecimal64()
+public abstract class FuncLongToDecimal64 extends VectorExpression {
+ private static final long serialVersionUID = 1L;
+ private final int inputColumn;
+
+ private static final long[] powerOfTenTable = {
+1L, // 0
+10L,
+100L,
+1_000L,
+10_000L,
+100_000L,
+1_000_000L,
+10_000_000L,
+100_000_000L, // 8
+1_000_000_000L,
+10_000_000_000L,
+100_000_000_000L,
+1_000_000_000_000L,
+10_000_000_000_000L,
+100_000_000_000_000L,
+1_000_000_000_000_000L,
+10_000_000_000_000_000L, // 16
+100_000_000_000_000_000L,
+1_000_000_000_000_000_000L, // 18
+ };
+
+ public FuncLongToDecimal64(int inputColumn, int outputColumnNum) {
+super(outputColumnNum);
+this.inputColumn = inputColumn;
+ }
+
+ public FuncLongToDecimal64() {
+super();
+
+// Dummy final assignments.
+inputColumn = -1;
+ }
+
+ abstract protected void func(Decimal64ColumnVector outputColVector,
LongColumnVector inputColVector, int i,
Review comment:
Thank you, I have addressed this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377963)
Time Spent: 1h (was: 50m)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377960=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377960 ] ASF GitHub Bot logged work on HIVE-22515: - Author: ASF GitHub Bot Created on: 28/Jan/20 01:12 Start Date: 28/Jan/20 01:12 Worklog Time Spent: 10m Work Description: ramesh0201 commented on pull request #880: HIVE-22515 Support cast to decimal64 in Vectorization URL: https://github.com/apache/hive/pull/880#discussion_r371567891 ## File path: ql/src/test/results/clientpositive/llap/vector_reuse_scratchcols.q.out ## @@ -369,7 +369,7 @@ STAGE PLANS: includeColumns: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11] dataColumns: ctinyint:tinyint, csmallint:smallint, cint:int, cbigint:bigint, cfloat:float, cdouble:double, cstring1:string, cstring2:string, ctimestamp1:timestamp, ctimestamp2:timestamp, cboolean1:boolean, cboolean2:boolean partitionColumnCount: 0 -scratchColumnTypeNames: [double, double, double, decimal(22,3), decimal(13,3), double, double, double, double, double, double, double, double] +scratchColumnTypeNames: [double, double, double, decimal(22,3), decimal(13,3)/DECIMAL_64, double, double, double, double, double, double, double, double] Review comment: Yes, because we can safely store a decimal with (13,3) by scaling them up and storing in long vectors and tracking the scale. We do this until precision+scale is <=18. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377960) Time Spent: 0.5h (was: 20m) > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: HIVE-22779.001.patch Status: Patch Available (was: Open) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.001.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Status: Open (was: Patch Available) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.001.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: (was: HIVE-22779.001.patch) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.001.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[
https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024789#comment-17024789
]
Hive QA commented on HIVE-22784:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 58 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} storage-api: The patch generated 0 new + 9 unchanged
- 1 fixed = 9 total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20370/dev-support/hive-personality.sh
|
| git revision | master / b7650eb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api U: storage-api |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20370/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Boundary condition to check if there is nothing to truncate in StringExpr
> functions
> ---
>
> Key: HIVE-22784
> URL: https://issues.apache.org/jira/browse/HIVE-22784
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22784.1.patch
>
>
> Add a boundary condition to check if there is nothing to truncate in
> StringExpr functions
> truncate()
> rightTrimAndTruncate()
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivam updated HIVE-22771:
--
Status: In Progress (was: Patch Available)
Incorporating comments to add testcase
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[
https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024784#comment-17024784
]
Hive QA commented on HIVE-22753:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991952/HIVE-22753.4.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17953 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20369/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20369/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20369/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991952 - PreCommit-HIVE-Build
> Fix gradual mem leak: Operationlog related appenders should be cleared up on
> errors
>
>
> Key: HIVE-22753
> URL: https://issues.apache.org/jira/browse/HIVE-22753
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch,
> HIVE-22753.3.patch, HIVE-22753.4.patch, image-2020-01-21-11-14-37-911.png,
> image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png
>
>
> In case of exception in SQLOperation, operational log does not get cleared
> up. This causes gradual build up of HushableRandomAccessFileAppender causing
> HS2 to OOM after some time.
> !image-2020-01-21-11-14-37-911.png|width=431,height=267!
>
> Allocation tree
> !image-2020-01-21-11-18-37-294.png|width=425,height=178!
>
> Prod instance mem
> !image-2020-01-21-11-17-59-279.png|width=698,height=209!
>
> Each HushableRandomAccessFileAppender holds internal ref to
> RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem
> leak.
> Related ticket: HIVE-18820
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024768#comment-17024768
]
Mithun Radhakrishnan commented on HIVE-22771:
-
FWIW, +1 from me as well. This is a good catch.
{quote}bq. Can we add a test for it (e.g., in TestHCatStorer)?
{quote}
This bug probably got through because while there are tests in
{{AbstractHCatStorerTest}} for the dynamic-partitioning case, and the hybrid
(i.e. partially dynamic) case, the static case (i.e. when the partition-ids are
fully known _a priori_) isn't covered. There will be value in adding a test for
Pig scripts writing to static partitions.
Thank you for working on this, [~shivam-mohan]. :]
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[
https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024765#comment-17024765
]
Hive QA commented on HIVE-22753:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
32s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20369/dev-support/hive-personality.sh
|
| git revision | master / b7650eb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20369/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix gradual mem leak: Operationlog related appenders should be cleared up on
> errors
>
>
> Key: HIVE-22753
> URL: https://issues.apache.org/jira/browse/HIVE-22753
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch,
> HIVE-22753.3.patch, HIVE-22753.4.patch, image-2020-01-21-11-14-37-911.png,
> image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png
>
>
> In case of exception in SQLOperation, operational log does not get cleared
> up. This causes gradual build up of HushableRandomAccessFileAppender causing
> HS2 to OOM after some time.
> !image-2020-01-21-11-14-37-911.png|width=431,height=267!
>
> Allocation tree
> !image-2020-01-21-11-18-37-294.png|width=425,height=178!
>
> Prod instance mem
> !image-2020-01-21-11-17-59-279.png|width=698,height=209!
>
> Each HushableRandomAccessFileAppender holds internal ref to
> RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem
> leak.
> Related ticket: HIVE-18820
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024759#comment-17024759
]
Jesus Camacho Rodriguez commented on HIVE-22771:
Thanks [~shivam-mohan]. Fix makes sense. Can we add a test for it (e.g., in
TestHCatStorer)?
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377910=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377910
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 23:27
Start Date: 27/Jan/20 23:27
Worklog Time Spent: 10m
Work Description: jdere commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371533452
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FuncLongToDecimal64.java
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import java.util.Arrays;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.VectorExpressionDescriptor;
+import
org.apache.hadoop.hive.ql.exec.vector.VectorizedExpressionsSupportDecimal64;
+import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo;
+
+/**
+ * This is a superclass for unary long functions and expressions returning
decimals that
+ * operate directly on the input and set the output.
+ */
+@VectorizedExpressionsSupportDecimal64()
+public abstract class FuncLongToDecimal64 extends VectorExpression {
+ private static final long serialVersionUID = 1L;
+ private final int inputColumn;
+
+ private static final long[] powerOfTenTable = {
+1L, // 0
+10L,
+100L,
+1_000L,
+10_000L,
+100_000L,
+1_000_000L,
+10_000_000L,
+100_000_000L, // 8
+1_000_000_000L,
+10_000_000_000L,
+100_000_000_000L,
+1_000_000_000_000L,
+10_000_000_000_000L,
+100_000_000_000_000L,
+1_000_000_000_000_000L,
+10_000_000_000_000_000L, // 16
+100_000_000_000_000_000L,
+1_000_000_000_000_000_000L, // 18
+ };
+
+ public FuncLongToDecimal64(int inputColumn, int outputColumnNum) {
+super(outputColumnNum);
+this.inputColumn = inputColumn;
+ }
+
+ public FuncLongToDecimal64() {
+super();
+
+// Dummy final assignments.
+inputColumn = -1;
+ }
+
+ abstract protected void func(Decimal64ColumnVector outputColVector,
LongColumnVector inputColVector, int i,
Review comment:
Can you use a slightly more descriptive method name here and describe what
it's for?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377910)
Time Spent: 20m (was: 10m)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[ https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377911=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377911 ] ASF GitHub Bot logged work on HIVE-22515: - Author: ASF GitHub Bot Created on: 27/Jan/20 23:27 Start Date: 27/Jan/20 23:27 Worklog Time Spent: 10m Work Description: jdere commented on pull request #880: HIVE-22515 Support cast to decimal64 in Vectorization URL: https://github.com/apache/hive/pull/880#discussion_r371537293 ## File path: ql/src/test/results/clientpositive/llap/vector_reuse_scratchcols.q.out ## @@ -369,7 +369,7 @@ STAGE PLANS: includeColumns: [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11] dataColumns: ctinyint:tinyint, csmallint:smallint, cint:int, cbigint:bigint, cfloat:float, cdouble:double, cstring1:string, cstring2:string, ctimestamp1:timestamp, ctimestamp2:timestamp, cboolean1:boolean, cboolean2:boolean partitionColumnCount: 0 -scratchColumnTypeNames: [double, double, double, decimal(22,3), decimal(13,3), double, double, double, double, double, double, double, double] +scratchColumnTypeNames: [double, double, double, decimal(22,3), decimal(13,3)/DECIMAL_64, double, double, double, double, double, double, double, double] Review comment: any idea how this shows up as decimal(13,3)/DECIMAL_64 - would the decimal(13,3) part also be converted to decimal64? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377911) Time Spent: 20m (was: 10m) > Support cast to decimal64 in Vectorization > -- > > Key: HIVE-22515 > URL: https://issues.apache.org/jira/browse/HIVE-22515 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch, > HIVE-22515.9.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Support cast to decimal64 in Vectorization -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377909
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 23:27
Start Date: 27/Jan/20 23:27
Worklog Time Spent: 10m
Work Description: jdere commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371534428
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastLongToDecimal64.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+
+/**
+ * To be used to cast long and boolean to decimal.
+ * This works for boolean too because boolean is encoded as 0
+ * for false and 1 for true.
+ */
+public class CastLongToDecimal64 extends FuncLongToDecimal64 {
Review comment:
This is currently the only concrete class for FuncLongToDecimal64 .. is it
expected that there are going to be more in the future (which would be the
point of making FuncLongToDecimal64 abstract?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377909)
Time Spent: 20m (was: 10m)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Support cast to decimal64 in Vectorization
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22515) Support cast to decimal64 in Vectorization
[
https://issues.apache.org/jira/browse/HIVE-22515?focusedWorklogId=377908=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377908
]
ASF GitHub Bot logged work on HIVE-22515:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 23:27
Start Date: 27/Jan/20 23:27
Worklog Time Spent: 10m
Work Description: jdere commented on pull request #880: HIVE-22515
Support cast to decimal64 in Vectorization
URL: https://github.com/apache/hive/pull/880#discussion_r371539542
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FuncLongToDecimal64.java
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.vector.expressions;
+
+import java.util.Arrays;
+
+import org.apache.hadoop.hive.ql.exec.vector.Decimal64ColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.LongColumnVector;
+import org.apache.hadoop.hive.ql.exec.vector.VectorExpressionDescriptor;
+import
org.apache.hadoop.hive.ql.exec.vector.VectorizedExpressionsSupportDecimal64;
+import org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo;
+
+/**
+ * This is a superclass for unary long functions and expressions returning
decimals that
+ * operate directly on the input and set the output.
+ */
+@VectorizedExpressionsSupportDecimal64()
+public abstract class FuncLongToDecimal64 extends VectorExpression {
+ private static final long serialVersionUID = 1L;
+ private final int inputColumn;
+
+ private static final long[] powerOfTenTable = {
+1L, // 0
+10L,
+100L,
+1_000L,
+10_000L,
+100_000L,
+1_000_000L,
+10_000_000L,
+100_000_000L, // 8
+1_000_000_000L,
+10_000_000_000L,
+100_000_000_000L,
+1_000_000_000_000L,
+10_000_000_000_000L,
+100_000_000_000_000L,
+1_000_000_000_000_000L,
+10_000_000_000_000_000L, // 16
+100_000_000_000_000_000L,
+1_000_000_000_000_000_000L, // 18
+ };
+
+ public FuncLongToDecimal64(int inputColumn, int outputColumnNum) {
+super(outputColumnNum);
+this.inputColumn = inputColumn;
+ }
+
+ public FuncLongToDecimal64() {
+super();
+
+// Dummy final assignments.
+inputColumn = -1;
+ }
+
+ abstract protected void func(Decimal64ColumnVector outputColVector,
LongColumnVector inputColVector, int i,
+ long scaleFactor);
+
+ @Override
+ public void evaluate(VectorizedRowBatch batch) throws HiveException {
+
+if (childExpressions != null) {
+ super.evaluateChildren(batch);
+}
+
+LongColumnVector inputColVector = (LongColumnVector)
batch.cols[inputColumn];
+int[] sel = batch.selected;
+int n = batch.size;
+Decimal64ColumnVector outputColVector = (Decimal64ColumnVector)
batch.cols[outputColumnNum];
+int outputScale = ((DecimalTypeInfo) outputTypeInfo).scale();
+final long scaleFactor = powerOfTenTable[outputScale];
Review comment:
Is outputScale always guaranteed to be between [0, 18]?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377908)
Time Spent: 20m (was: 10m)
> Support cast to decimal64 in Vectorization
> --
>
> Key: HIVE-22515
> URL: https://issues.apache.org/jira/browse/HIVE-22515
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22515.5.patch, HIVE-22515.8.patch,
> HIVE-22515.9.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
[jira] [Updated] (HIVE-22785) Update/delete/merge statements not optimized through CBO
[
https://issues.apache.org/jira/browse/HIVE-22785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22785:
---
Description:
Currently, CBO is bypassed for update/delete/merge statements.
To support optimizing these statements through CBO, we need to complete three
main tasks: 1) support for sort in Calcite planner, 2) support for SORT in AST
converter, and 3) {{RewriteSemanticAnalyzer}} should extend {{CalcitePlanner}}
instead of {{SemanticAnalyzer}}.
was:
Currently, CBO is bypassed for update/delete/merge statements.
To support optimizing these statements through CBO, we need to complete three
main tasks: 1) support for sort in CBO, 2) support for SORT in AST converter,
and 3) {{RewriteSemanticAnalyzer}} should extend {{CalcitePlanner}} instead of
{{SemanticAnalyzer}}.
> Update/delete/merge statements not optimized through CBO
>
>
> Key: HIVE-22785
> URL: https://issues.apache.org/jira/browse/HIVE-22785
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Priority: Critical
>
> Currently, CBO is bypassed for update/delete/merge statements.
> To support optimizing these statements through CBO, we need to complete three
> main tasks: 1) support for sort in Calcite planner, 2) support for SORT in
> AST converter, and 3) {{RewriteSemanticAnalyzer}} should extend
> {{CalcitePlanner}} instead of {{SemanticAnalyzer}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[ https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22784: Attachment: HIVE-22784.1.patch Status: Patch Available (was: Open) > Boundary condition to check if there is nothing to truncate in StringExpr > functions > --- > > Key: HIVE-22784 > URL: https://issues.apache.org/jira/browse/HIVE-22784 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22784.1.patch > > > Add a boundary condition to check if there is nothing to truncate in > StringExpr functions > truncate() > rightTrimAndTruncate() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024716#comment-17024716 ] Rajesh Balamohan commented on HIVE-22753: - Fixed the comment. Config for this would be an overkill. > Fix gradual mem leak: Operationlog related appenders should be cleared up on > errors > > > Key: HIVE-22753 > URL: https://issues.apache.org/jira/browse/HIVE-22753 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch, > HIVE-22753.3.patch, HIVE-22753.4.patch, image-2020-01-21-11-14-37-911.png, > image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png > > > In case of exception in SQLOperation, operational log does not get cleared > up. This causes gradual build up of HushableRandomAccessFileAppender causing > HS2 to OOM after some time. > !image-2020-01-21-11-14-37-911.png|width=431,height=267! > > Allocation tree > !image-2020-01-21-11-18-37-294.png|width=425,height=178! > > Prod instance mem > !image-2020-01-21-11-17-59-279.png|width=698,height=209! > > Each HushableRandomAccessFileAppender holds internal ref to > RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem > leak. > Related ticket: HIVE-18820 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22753: Attachment: HIVE-22753.4.patch > Fix gradual mem leak: Operationlog related appenders should be cleared up on > errors > > > Key: HIVE-22753 > URL: https://issues.apache.org/jira/browse/HIVE-22753 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch, > HIVE-22753.3.patch, HIVE-22753.4.patch, image-2020-01-21-11-14-37-911.png, > image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png > > > In case of exception in SQLOperation, operational log does not get cleared > up. This causes gradual build up of HushableRandomAccessFileAppender causing > HS2 to OOM after some time. > !image-2020-01-21-11-14-37-911.png|width=431,height=267! > > Allocation tree > !image-2020-01-21-11-18-37-294.png|width=425,height=178! > > Prod instance mem > !image-2020-01-21-11-17-59-279.png|width=698,height=209! > > Each HushableRandomAccessFileAppender holds internal ref to > RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem > leak. > Related ticket: HIVE-18820 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22779) Hive build fails with Spark 3
[
https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024708#comment-17024708
]
Hive QA commented on HIVE-22779:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991935/HIVE-22779.001.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20368/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20368/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20368/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-27 22:10:44.783
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20368/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-27 22:10:44.787
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at b7650eb HIVE-22366: Multiple metastore calls for same table and
constraints during planning (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at b7650eb HIVE-22366: Multiple metastore calls for same table and
constraints during planning (Vineet Garg, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-27 22:10:46.008
+ rm -rf ../yetus_PreCommit-HIVE-Build-20368
+ mkdir ../yetus_PreCommit-HIVE-Build-20368
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20368
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20368/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java: does
not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/client/metrics/ShuffleWriteMetrics.java:
does not exist in index
error:
a/spark-client/src/main/java/org/apache/hive/spark/counter/SparkCounter.java:
does not exist in index
Trying to apply the patch with -p1
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java:26
Falling back to three-way merge...
Applied patch to
'ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java' with
conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java:26
Falling back to three-way merge...
Applied patch to
'ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java' with
conflicts.
U ql/src/test/org/apache/hadoop/hive/ql/stats/TestStatsUtils.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20368
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991935 - PreCommit-HIVE-Build
> Hive build fails with Spark 3
> -
>
> Key: HIVE-22779
> URL: https://issues.apache.org/jira/browse/HIVE-22779
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 3.1.2
>Reporter: Igor Dvorzhak
>Priority: Major
> Attachments: HIVE-22779.001.patch
>
>
> Hive build with Spark 3 fails because of deletion of few deprecated APIs in
> Spark 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22770) Skip interning of MapWork fields for tez during deserialization
[
https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024707#comment-17024707
]
Hive QA commented on HIVE-22770:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991934/HIVE-22770.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20367/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20367/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20367/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991934/HIVE-22770.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991934 - PreCommit-HIVE-Build
> Skip interning of MapWork fields for tez during deserialization
> ---
>
> Key: HIVE-22770
> URL: https://issues.apache.org/jira/browse/HIVE-22770
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch,
> HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch
>
>
> HIVE-19937 introduced some interning logic into mapwork deserialization
> process, but it's only related to spark, maybe we should skip this for tez,
> reducing the cpu pressure in tez tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22780) Upgrade slf4j version to 1.7.30
[
https://issues.apache.org/jira/browse/HIVE-22780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024706#comment-17024706
]
Hive QA commented on HIVE-22780:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991920/HIVE-22780.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17953 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20366/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20366/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20366/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991920 - PreCommit-HIVE-Build
> Upgrade slf4j version to 1.7.30
> ---
>
> Key: HIVE-22780
> URL: https://issues.apache.org/jira/browse/HIVE-22780
> Project: Hive
> Issue Type: Improvement
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22780.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22780) Upgrade slf4j version to 1.7.30
[
https://issues.apache.org/jira/browse/HIVE-22780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024683#comment-17024683
]
Hive QA commented on HIVE-22780:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20366/dev-support/hive-personality.sh
|
| git revision | master / b7650eb |
| Default Java | 1.8.0_111 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20366/yetus/whitespace-tabs.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20366/yetus/patch-asflicense-problems.txt
|
| modules | C: storage-api kafka-handler standalone-metastore/metastore-tools .
itests/qtest itests/qtest-druid testutils/ptest2 U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20366/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade slf4j version to 1.7.30
> ---
>
> Key: HIVE-22780
> URL: https://issues.apache.org/jira/browse/HIVE-22780
> Project: Hive
> Issue Type: Improvement
>Reporter: David Lavati
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22780.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22784) Boundary condition to check if there is nothing to truncate in StringExpr functions
[ https://issues.apache.org/jira/browse/HIVE-22784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan reassigned HIVE-22784: --- Assignee: Ramesh Kumar Thangarajan > Boundary condition to check if there is nothing to truncate in StringExpr > functions > --- > > Key: HIVE-22784 > URL: https://issues.apache.org/jira/browse/HIVE-22784 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > > Add a boundary condition to check if there is nothing to truncate in > StringExpr functions > truncate() > rightTrimAndTruncate() -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024648#comment-17024648
]
Hive QA commented on HIVE-22538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991918/HIVE-22538.8.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20365/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20365/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20365/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991918/HIVE-22538.8.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991918 - PreCommit-HIVE-Build
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22770) Skip interning of MapWork fields for tez during deserialization
[
https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024625#comment-17024625
]
Hive QA commented on HIVE-22770:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1538 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 162 unchanged - 0
fixed = 163 total (was 162) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
7s{color} | {color:red} ql generated 1 new + 1538 unchanged - 0 fixed = 1539
total (was 1538) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unchecked/unconfirmed cast from com.esotericsoftware.kryo.Kryo to
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks in
org.apache.hadoop.hive.ql.exec.SerializationUtilities.releaseKryo(Kryo) At
SerializationUtilities.java:org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks
in org.apache.hadoop.hive.ql.exec.SerializationUtilities.releaseKryo(Kryo) At
SerializationUtilities.java:[line 307] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20364/dev-support/hive-personality.sh
|
| git revision | master / b7650eb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20364/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20364/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20364/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Skip interning of MapWork fields for tez during deserialization
> ---
>
> Key: HIVE-22770
> URL: https://issues.apache.org/jira/browse/HIVE-22770
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch,
> HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch
>
>
> HIVE-19937 introduced some interning logic into mapwork deserialization
> process, but it's only related to spark, maybe we should skip this for tez,
> reducing the cpu pressure in tez tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024605#comment-17024605
]
Hive QA commented on HIVE-22538:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991918/HIVE-22538.8.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17954 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query25]
(batchId=303)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20362/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20362/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20362/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991918 - PreCommit-HIVE-Build
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22783) Add test for HIVE-22366
[ https://issues.apache.org/jira/browse/HIVE-22783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-22783: -- > Add test for HIVE-22366 > --- > > Key: HIVE-22783 > URL: https://issues.apache.org/jira/browse/HIVE-22783 > Project: Hive > Issue Type: Sub-task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Fix Version/s: 4.0.0
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch, HIVE-22366.5.patch,
> HIVE-22366.6.patch, HIVE-22366.7.patch, HIVE-22366.8.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~jcamachorodriguez]
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch, HIVE-22366.5.patch,
> HIVE-22366.6.patch, HIVE-22366.7.patch, HIVE-22366.8.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024564#comment-17024564
]
Hive QA commented on HIVE-22538:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
47s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} ql: The patch generated 0 new + 499 unchanged - 4
fixed = 499 total (was 503) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 170
unchanged - 2 fixed = 170 total (was 172) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20362/dev-support/hive-personality.sh
|
| git revision | master / f7eb8b6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20362/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
>
[jira] [Updated] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22366:
---
Attachment: HIVE-22366.8.patch
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch, HIVE-22366.5.patch,
> HIVE-22366.6.patch, HIVE-22366.7.patch, HIVE-22366.8.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_aggr_stats_for: table=hive.default.test2
> 2019-10-17T22:44:50,014 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:50,023 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:50,024 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:50,025 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:50,026 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22782) Consolidate metastore call to fetch constraints
[ https://issues.apache.org/jira/browse/HIVE-22782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-22782: -- > Consolidate metastore call to fetch constraints > --- > > Key: HIVE-22782 > URL: https://issues.apache.org/jira/browse/HIVE-22782 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > Currently separate calls are made to metastore to fetch constraints like Pk, > fk, not null etc. Since planner always retrieve these constraints we should > retrieve all of them in one call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22366) Multiple metastore calls for same table and constraints during planning
[
https://issues.apache.org/jira/browse/HIVE-22366?focusedWorklogId=377798=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377798
]
ASF GitHub Bot logged work on HIVE-22366:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 18:38
Start Date: 27/Jan/20 18:38
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #888:
HIVE-22366: Multiple metastore calls for same table and constraints d…
URL: https://github.com/apache/hive/pull/888#discussion_r371412106
##
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
##
@@ -1126,71 +1149,167 @@ public Boolean isOutdatedForRewriting() {
return outdatedForRewritingMaterializedView;
}
- /* These are only populated during optimization and describing */
+ public ColumnStatistics getColStats() {
+return tTable.isSetColStats() ? tTable.getColStats() : null;
+ }
+
+ /**
+ * Setup the table level stats as if the table is new. Used when setting up
Table for a new
+ * table or during replication.
+ */
+ public void setStatsStateLikeNewTable() {
+if (isPartitioned()) {
+ StatsSetupConst.setStatsStateForCreateTable(getParameters(), null,
+ StatsSetupConst.FALSE);
+} else {
+ StatsSetupConst.setStatsStateForCreateTable(getParameters(),
+ MetaStoreUtils.getColumnNames(getCols()), StatsSetupConst.TRUE);
+}
+ }
+
+ /** Constraints related methods
+ * Note that set apis are used by DESCRIBE only, although get apis return
RELY or ENABLE
+ * constraints DESCRIBE could set all type of constraints
+ * */
+
+ /* This only return PK with RELY */
public PrimaryKeyInfo getPrimaryKeyInfo() {
+if(!this.isPKFetched) {
+ try {
+pki = Hive.get().getReliablePrimaryKeys(this.getDbName(),
this.getTableName());
+this.isPKFetched = true;
+ } catch (HiveException e) {
+LOG.warn("Cannot retrieve PK info for table : " + this.getTableName()
++ " ignoring exception: " + e);
+ }
+}
return pki;
}
public void setPrimaryKeyInfo(PrimaryKeyInfo pki) {
this.pki = pki;
+this.isPKFetched = true;
}
+ /* This only return FK with RELY */
public ForeignKeyInfo getForeignKeyInfo() {
Review comment:
Created https://issues.apache.org/jira/browse/HIVE-22782 for follow-up
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377798)
Time Spent: 0.5h (was: 20m)
> Multiple metastore calls for same table and constraints during planning
> ---
>
> Key: HIVE-22366
> URL: https://issues.apache.org/jira/browse/HIVE-22366
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22366.1.patch, HIVE-22366.2.patch,
> HIVE-22366.3.patch, HIVE-22366.4.patch, HIVE-22366.5.patch,
> HIVE-22366.6.patch, HIVE-22366.7.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> e.g. for a simple query like
> {code:sql}
> explain select count(i) from test2 group by j limit 3;
> {code}
> planner makes the following calls:
> {code}
> 2019-10-17T22:44:49,892 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr cmd=get_table :
> tbl=hive.default.test2
> 2019-10-17T22:44:49,908 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_primary_keys : tbl=hive.default.test2
> 2019-10-17T22:44:49,910 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_foreign_keys : parentdb=null parenttbl=null foreigndb=default
> foreigntbl=test2
> 2019-10-17T22:44:49,911 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_unique_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,913 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_not_null_constraints : tbl=hive.default.test2
> 2019-10-17T22:44:49,979 INFO [812910d4-bc6a-450c-aa8b-3487b4edceab main]
> HiveMetaStore.audit: ugi=vgarg ip=unknown-ip-addr
> cmd=get_partitions : tbl=hive.default.test2
> 2019-10-17T22:44:49,997 INFO
[jira] [Commented] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024536#comment-17024536
]
Hive QA commented on HIVE-22765:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991910/HIVE-22765.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17953 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20361/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20361/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20361/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991910 - PreCommit-HIVE-Build
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.2.patch, HIVE-22765.3.patch, HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:311) at
> org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:297) at
> org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:274) at
> org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:225) at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findNextToCompact(CompactionTxnHandler.java:157)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.find_next_compact(HiveMetaStore.java:8061)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy28.find_next_compact(Unknown Source) at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.findNextCompact(HiveMetaStoreClient.java:4046)
> at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:208)
> at com.sun.proxy.$Proxy29.findNextCompact(Unknown Source) at
> org.apache.hadoop.hive.ql.txn.compactor.Worker.run(Worker.java:102){code}
> This is due to missing quotes around table names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: HIVE-22779.001.patch Target Version/s: (was: 3.1.3) Status: Patch Available (was: Open) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > Attachments: HIVE-22779.001.patch > > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Status: Open (was: Patch Available) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22779) Hive build fails with Spark 3
[ https://issues.apache.org/jira/browse/HIVE-22779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-22779: - Attachment: (was: HIVE-22779.patch) > Hive build fails with Spark 3 > - > > Key: HIVE-22779 > URL: https://issues.apache.org/jira/browse/HIVE-22779 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 3.1.2 >Reporter: Igor Dvorzhak >Priority: Major > > Hive build with Spark 3 fails because of deletion of few deprecated APIs in > Spark 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22770) Skip interning of MapWork fields for tez during deserialization
[ https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-22770: Attachment: HIVE-22770.03.patch > Skip interning of MapWork fields for tez during deserialization > --- > > Key: HIVE-22770 > URL: https://issues.apache.org/jira/browse/HIVE-22770 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch, > HIVE-22770.02.patch, HIVE-22770.02.patch, HIVE-22770.03.patch > > > HIVE-19937 introduced some interning logic into mapwork deserialization > process, but it's only related to spark, maybe we should skip this for tez, > reducing the cpu pressure in tez tasks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22746) Make TypeCheckProcFactory generic
[
https://issues.apache.org/jira/browse/HIVE-22746?focusedWorklogId=377740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377740
]
ASF GitHub Bot logged work on HIVE-22746:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 17:04
Start Date: 27/Jan/20 17:04
Worklog Time Spent: 10m
Work Description: asfgit commented on pull request #887: HIVE-22746
URL: https://github.com/apache/hive/pull/887
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377740)
Time Spent: 1h (was: 50m)
> Make TypeCheckProcFactory generic
> -
>
> Key: HIVE-22746
> URL: https://issues.apache.org/jira/browse/HIVE-22746
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22746.01.patch, HIVE-22746.01.patch,
> HIVE-22746.02.patch, HIVE-22746.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{TypeCheckProcFactory}} is responsible for processing AST nodes and
> generating ExprNode objects from them. When we generate the expressions for
> Calcite planning, we go through a {{AST node -> ExprNode -> RexNode}}
> transformation. We would like to avoid the overhead of going through the
> ExprNode, and thus generate directly the RexNode from the AST.
> To do that, the first step is to make {{TypeCheckProcFactory}} generic, so it
> can receive an expression factory and create expressions in different realms.
> For the time being, the only factory implementation is the ExprNode factory.
> Thus, this patch focuses mainly on refactoring {{TypeCheckProcFactory}}
> without breaking anything that is already working.
> In a follow-up patch, we will create a {{RexNode}} factory and use it when we
> parse the query in CalcitePlanner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22746) Make TypeCheckProcFactory generic
[
https://issues.apache.org/jira/browse/HIVE-22746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22746:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~mgergely], [~vgarg].
> Make TypeCheckProcFactory generic
> -
>
> Key: HIVE-22746
> URL: https://issues.apache.org/jira/browse/HIVE-22746
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22746.01.patch, HIVE-22746.01.patch,
> HIVE-22746.02.patch, HIVE-22746.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{TypeCheckProcFactory}} is responsible for processing AST nodes and
> generating ExprNode objects from them. When we generate the expressions for
> Calcite planning, we go through a {{AST node -> ExprNode -> RexNode}}
> transformation. We would like to avoid the overhead of going through the
> ExprNode, and thus generate directly the RexNode from the AST.
> To do that, the first step is to make {{TypeCheckProcFactory}} generic, so it
> can receive an expression factory and create expressions in different realms.
> For the time being, the only factory implementation is the ExprNode factory.
> Thus, this patch focuses mainly on refactoring {{TypeCheckProcFactory}}
> without breaking anything that is already working.
> In a follow-up patch, we will create a {{RexNode}} factory and use it when we
> parse the query in CalcitePlanner.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024504#comment-17024504
]
Jesus Camacho Rodriguez commented on HIVE-22538:
+1 (pending tests)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22765) Quote table names in CompactionTxnHandler
[
https://issues.apache.org/jira/browse/HIVE-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024503#comment-17024503
]
Hive QA commented on HIVE-22765:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 24 new + 55 unchanged - 25 fixed = 79 total (was 80) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20361/dev-support/hive-personality.sh
|
| git revision | master / 8dec57c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20361/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20361/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Quote table names in CompactionTxnHandler
> -
>
> Key: HIVE-22765
> URL: https://issues.apache.org/jira/browse/HIVE-22765
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22765.2.patch, HIVE-22765.3.patch, HIVE-22765.patch
>
>
> When I run a metastore backed up by a postgres db, I get the following
> exception:
> {code:java}
> MetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not existMetaException(message:Unable to connect to transaction database
> org.postgresql.util.PSQLException: ERROR: relation "compaction_queue" does
> not exist Position: 83 at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2468)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2211)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:309)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) at
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) at
>
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024486#comment-17024486 ] Zoltan Chovan commented on HIVE-22753: -- [~rajesh.balamohan] the code comments state that the elements will have 5s ttl, but this line shows only 1s: .expireAfterWrite(1, TimeUnit.SECONDS) Also do you think having a config value for this would be overkill? Would more time be needed potentially, in case there's a heavy IO load for some reason on the host of HS2? > Fix gradual mem leak: Operationlog related appenders should be cleared up on > errors > > > Key: HIVE-22753 > URL: https://issues.apache.org/jira/browse/HIVE-22753 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch, > HIVE-22753.3.patch, image-2020-01-21-11-14-37-911.png, > image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png > > > In case of exception in SQLOperation, operational log does not get cleared > up. This causes gradual build up of HushableRandomAccessFileAppender causing > HS2 to OOM after some time. > !image-2020-01-21-11-14-37-911.png|width=431,height=267! > > Allocation tree > !image-2020-01-21-11-18-37-294.png|width=425,height=178! > > Prod instance mem > !image-2020-01-21-11-17-59-279.png|width=698,height=209! > > Each HushableRandomAccessFileAppender holds internal ref to > RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem > leak. > Related ticket: HIVE-18820 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024485#comment-17024485
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991909/HIVE-10362.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20360/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20360/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20360/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991909/HIVE-10362.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991909 - PreCommit-HIVE-Build
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic partition insert. Currently any data can be inserted as a partition
> column value as long as it is quoted as a string. For example,
> create table dynparttypechecknum (key int, value string) partitioned by (part
> int);
> insert into dynparttypechecknum partition (part) select key, value, '1'
> from src limit 1;
> show partitions dynparttypechecknum;
> --
> part=1
> The partition column value is non-normalized int 1. It causes some
> unnecessary problems such as integer partition column JDO filter pushdown
> (see HIVE-6052) and others like HIVE-10210.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024484#comment-17024484
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991909/HIVE-10362.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20359/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20359/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20359/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12991909/HIVE-10362.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991909 - PreCommit-HIVE-Build
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic partition insert. Currently any data can be inserted as a partition
> column value as long as it is quoted as a string. For example,
> create table dynparttypechecknum (key int, value string) partitioned by (part
> int);
> insert into dynparttypechecknum partition (part) select key, value, '1'
> from src limit 1;
> show partitions dynparttypechecknum;
> --
> part=1
> The partition column value is non-normalized int 1. It causes some
> unnecessary problems such as integer partition column JDO filter pushdown
> (see HIVE-6052) and others like HIVE-10210.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22775) Use the qt:authorizer option in qtests
[
https://issues.apache.org/jira/browse/HIVE-22775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024483#comment-17024483
]
Hive QA commented on HIVE-22775:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991904/HIVE-22775.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20358/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20358/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20358/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-27 16:34:55.003
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20358/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-27 16:34:55.005
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
7bb1d1e..8dec57c master -> origin/master
+ git reset --hard HEAD
HEAD is now at 7bb1d1e HIVE-22518: SQLStdHiveAuthorizerFactoryForTest doesn't
work correctly for llap tests (Zoltan Haindrich reviewed by Miklos Gergely)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 3 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 8dec57c HIVE-22679: Replace Base64 in metastore-common Package
(David Mollitor via Naveen Gangam)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-27 16:34:57.416
+ rm -rf ../yetus_PreCommit-HIVE-Build-20358
+ mkdir ../yetus_PreCommit-HIVE-Build-20358
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20358
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20358/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:25: trailing whitespace.
warning: 1 line adds whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc1330264086802028845.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc1330264086802028845.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(process-resource-bundles) on project hive-shims-0.23: Execution
process-resource-bundles of goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed.
ConcurrentModificationException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024478#comment-17024478
]
Hive QA commented on HIVE-10362:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12991909/HIVE-10362.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 17947 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_6]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge10] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge1] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge2] (batchId=103)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_join_partition_key]
(batchId=15)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge10]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge1]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge2]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_merge2]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_dml]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge1]
(batchId=197)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge2]
(batchId=200)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_diff_fs]
(batchId=197)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=141)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20357/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20357/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20357/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12991909 - PreCommit-HIVE-Build
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for partition column. Hive has many ways to introduce
> such problematic data. HIVE-10307 mainly provides the support to type
> check/convert/normalize the partition column value in static partition
> specification. This JIRA tries to deal with the partition column type in
> dynamic partition insert. Currently any data can be inserted as a partition
> column value as long as it is quoted as a string. For example,
> create table dynparttypechecknum (key int, value string) partitioned by (part
> int);
> insert into dynparttypechecknum partition (part) select key, value, '1'
> from src limit 1;
> show partitions dynparttypechecknum;
> --
> part=1
> The partition column value is non-normalized int 1. It causes some
> unnecessary problems such as integer
[jira] [Comment Edited] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023549#comment-17023549
]
Shivam edited comment on HIVE-22771 at 1/27/20 4:32 PM:
[~mithun] [~jcamachorodriguez] [~kgyrtkirk] Can anyone please review this?
Thanks.
was (Author: shivam-mohan):
[~mithun] [~jcamachorodriguez] Can you please review this?
Thanks.
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22753) Fix gradual mem leak: Operationlog related appenders should be cleared up on errors
[ https://issues.apache.org/jira/browse/HIVE-22753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024471#comment-17024471 ] Ashutosh Chauhan commented on HIVE-22753: - +1 > Fix gradual mem leak: Operationlog related appenders should be cleared up on > errors > > > Key: HIVE-22753 > URL: https://issues.apache.org/jira/browse/HIVE-22753 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22753.1.patch, HIVE-22753.2.patch, > HIVE-22753.3.patch, image-2020-01-21-11-14-37-911.png, > image-2020-01-21-11-17-59-279.png, image-2020-01-21-11-18-37-294.png > > > In case of exception in SQLOperation, operational log does not get cleared > up. This causes gradual build up of HushableRandomAccessFileAppender causing > HS2 to OOM after some time. > !image-2020-01-21-11-14-37-911.png|width=431,height=267! > > Allocation tree > !image-2020-01-21-11-18-37-294.png|width=425,height=178! > > Prod instance mem > !image-2020-01-21-11-17-59-279.png|width=698,height=209! > > Each HushableRandomAccessFileAppender holds internal ref to > RandomAccessFileAppender which holds a 256 KB bytebuffer, causing the mem > leak. > Related ticket: HIVE-18820 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22770) Skip interning of MapWork fields for tez during deserialization
[
https://issues.apache.org/jira/browse/HIVE-22770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024468#comment-17024468
]
Ashutosh Chauhan commented on HIVE-22770:
-
Configuration conf = ((KryoWithHooks) kryo).getConf();
+ if (conf != null &&
conf.get(HiveConf.ConfVars.HIVE_EXECUTION_ENGINE.varname,
"").equalsIgnoreCase("spark")) {
+mapWork.internFields();
+ }
+
Lets remove above code block. With that removed we won't need {{new
Configuration}} which is an expensive call. That means we will never intern
fields.
> Skip interning of MapWork fields for tez during deserialization
> ---
>
> Key: HIVE-22770
> URL: https://issues.apache.org/jira/browse/HIVE-22770
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22770.01.patch, HIVE-22770.01.patch,
> HIVE-22770.02.patch, HIVE-22770.02.patch
>
>
> HIVE-19937 introduced some interning logic into mapwork deserialization
> process, but it's only related to spark, maybe we should skip this for tez,
> reducing the cpu pressure in tez tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024455#comment-17024455
]
Chiran Ravani commented on HIVE-22769:
--
[~kuczoram] Can we skip get split operation if the files are text and
compressed?
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Marta Kuczora
>Priority: Critical
> Attachments: testcase1.csv.bz2, testcase2.csv.bz2
>
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 3|
> +--+
> 1 row selected (4.335 seconds)
> {code}
> Case 2: Select count/aggregate query fails with java.lang.ClassCastException:
> java.io.PushbackInputStream cannot be cast to org.apache.hadoop.fs.Seekable
> The issue is only seen when there is a space in a field (eg:- "3,2019-12-31
> 01" second column has a space)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase2`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase2'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase2.csv.bz2 file to /user/hive/testcase2
> 3. Run count(*) on table.
> {code}
> 0: > select * from testcase2;
> INFO : Compiling
> command(queryId=hive_20200124053159_5d8ce56a-183d-4359-a147-bd470d82e134):
> select * from testcase2
> INFO : Semantic Analysis Completed
[jira] [Updated] (HIVE-22780) Upgrade slf4j version to 1.7.30
[ https://issues.apache.org/jira/browse/HIVE-22780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lavati updated HIVE-22780: Attachment: HIVE-22780.01.patch Status: Patch Available (was: Open) > Upgrade slf4j version to 1.7.30 > --- > > Key: HIVE-22780 > URL: https://issues.apache.org/jira/browse/HIVE-22780 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22780.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22780) Upgrade slf4j version to 1.7.30
[ https://issues.apache.org/jira/browse/HIVE-22780?focusedWorklogId=377708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377708 ] ASF GitHub Bot logged work on HIVE-22780: - Author: ASF GitHub Bot Created on: 27/Jan/20 15:39 Start Date: 27/Jan/20 15:39 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #890: HIVE-22780: Upgrade slf4j to 1.7.30 URL: https://github.com/apache/hive/pull/890 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377708) Remaining Estimate: 0h Time Spent: 10m > Upgrade slf4j version to 1.7.30 > --- > > Key: HIVE-22780 > URL: https://issues.apache.org/jira/browse/HIVE-22780 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22780) Upgrade slf4j version to 1.7.30
[ https://issues.apache.org/jira/browse/HIVE-22780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22780: -- Labels: pull-request-available (was: ) > Upgrade slf4j version to 1.7.30 > --- > > Key: HIVE-22780 > URL: https://issues.apache.org/jira/browse/HIVE-22780 > Project: Hive > Issue Type: Improvement >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-10362) Support Type check/conversion in dynamic partition column
[
https://issues.apache.org/jira/browse/HIVE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024436#comment-17024436
]
Hive QA commented on HIVE-10362:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
53s{color} | {color:red} ql generated 2 new + 1533 unchanged - 0 fixed = 1535
total (was 1533) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Redundant nullcheck of destinationTable which is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:is known to be null in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) Redundant null check at SemanticAnalyzer.java:[line 7680] |
| | Nullcheck of destinationTable at line 7335 of value previously
dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) At SemanticAnalyzer.java:7335 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(String, QB,
Operator) At SemanticAnalyzer.java:[line 7335] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20357/dev-support/hive-personality.sh
|
| git revision | master / 7bb1d1e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20357/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20357/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Type check/conversion in dynamic partition column
> -
>
> Key: HIVE-10362
> URL: https://issues.apache.org/jira/browse/HIVE-10362
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor, Types
>Affects Versions: 1.0.0, 3.0.0, 4.0.0
>Reporter: Chaoyu Tang
>Assignee: Karen Coppage
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-10362.01.patch, HIVE-10362.02.patch,
> HIVE-10362.03.patch, HIVE-10362.04.patch, HIVE-10362.04.patch,
> HIVE-10362.06.patch
>
>
> There are quite a lot of issues associated with the non-noramlized or
> type-mismatched values for
[jira] [Comment Edited] (HIVE-22767) Beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024433#comment-17024433
]
Zoltan Haindrich edited comment on HIVE-22767 at 1/27/20 3:36 PM:
--
because there were no real code changes...waiting for another ptest will not
change anything...I've run the new test manually.
pushed to master. Thank you [~zmatyus]!
was (Author: kgyrtkirk):
pushed to master. Thank you [~zmatyus]
> Beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch,
> HIVE-22767.03.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
> INFO : Executing command(queryId=...): select 1
> INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> 1 row selected (2.007 seconds)
> INFO : Compiling command(queryId=...): select /*
> ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
> '' '' in select clause
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near '' '' '' in select clause
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at ...
> {noformat}
>
> Similarly, the following query also fails:
> {code:sql}select /* ' */ 1; select /* ' */ 2;{code}
> I suspect line comments are also not handled properly but I cannot reproduce
> this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22767) Beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22767:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
pushed to master. Thank you [~zmatyus]
> Beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch,
> HIVE-22767.03.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
> INFO : Executing command(queryId=...): select 1
> INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> 1 row selected (2.007 seconds)
> INFO : Compiling command(queryId=...): select /*
> ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
> '' '' in select clause
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near '' '' '' in select clause
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at ...
> {noformat}
>
> Similarly, the following query also fails:
> {code:sql}select /* ' */ 1; select /* ' */ 2;{code}
> I suspect line comments are also not handled properly but I cannot reproduce
> this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22774) Usability improvements of scheduled queries
[ https://issues.apache.org/jira/browse/HIVE-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22774: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master thank you Jesus for reviewing the changes! > Usability improvements of scheduled queries > --- > > Key: HIVE-22774 > URL: https://issues.apache.org/jira/browse/HIVE-22774 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22774.01.patch, HIVE-22774.02.patch, > HIVE-22774.03.patch > > > * add feature toggle switch to HMS side rpc call sites - make it possible to > disable it > * rename query state ERRORED state to FAILED > * ProgressReporter thread should not enter a busy wait loop in case the > thread is interrupted -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22679) Replace Base64 in metastore-common Package
[ https://issues.apache.org/jira/browse/HIVE-22679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22679: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you David! > Replace Base64 in metastore-common Package > -- > > Key: HIVE-22679 > URL: https://issues.apache.org/jira/browse/HIVE-22679 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22679.1.patch, HIVE-22679.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22767) Beeline doesn't parse semicolons in comments properly
[
https://issues.apache.org/jira/browse/HIVE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22767:
Summary: Beeline doesn't parse semicolons in comments properly (was:
beeline doesn't parse semicolons in comments properly)
> Beeline doesn't parse semicolons in comments properly
> -
>
> Key: HIVE-22767
> URL: https://issues.apache.org/jira/browse/HIVE-22767
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Major
> Attachments: HIVE-22767.01.patch, HIVE-22767.02.patch,
> HIVE-22767.03.patch
>
>
> HIVE-12646 fixed the handling of semicolons in quoted strings, but leaves the
> problem of semicolons in comments. E.g. with beeline connected to any
> database...
> this works: {code:sql}select 1; select /* */ 2; select /* */ 3;{code}
> this doesn't work: {code:sql}select 1; select /* ; */ 2; select /* ; */
> 3;{code}
> This has been fixed and reintroduced before (possibly multiple times).
> Ideally, there should be a single utility method somewhere to separate
> comments, strings and commands -- with the proper testing in place (q files).
> However, I'm trying to make this fix back-portable, so a light touch is
> needed. I'm focusing on beeline for now, and only writing (very thorough)
> unit tests, as I cannot exclude any new q files from TestCliDriver (which
> would break, since it's using a different parsing method).
>
> P.S. excerpt of the error message:
> {noformat}
> 0: jdbc:hive2://...> select 1; select /* ; */ 2; select /* ; */ 3;
> INFO : Compiling command(queryId=...): select 1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=...); Time taken: 0.38 seconds
> INFO : Executing command(queryId=...): select 1
> INFO : Completed executing command(queryId=...); Time taken: 0.004 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 1|
> +--+
> 1 row selected (2.007 seconds)
> INFO : Compiling command(queryId=...): select /*
> ERROR : FAILED: ParseException line 1:9 cannot recognize input near ''
> '' '' in select clause
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near '' '' '' in select clause
> at
> org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:233)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:79)
> at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:72)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:598)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1505)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1452)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1447)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at ...
> {noformat}
>
> Similarly, the following query also fails:
> {code:sql}select /* ' */ 1; select /* ' */ 2;{code}
> I suspect line comments are also not handled properly but I cannot reproduce
> this in interactive beeline...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?focusedWorklogId=377703=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377703
]
ASF GitHub Bot logged work on HIVE-22538:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 15:31
Start Date: 27/Jan/20 15:31
Worklog Time Spent: 10m
Work Description: kasakrisz commented on pull request #877: HIVE-22538:
RS deduplication does not always enforce
hive.optimize.reducededuplication.min.reducer
URL: https://github.com/apache/hive/pull/877#discussion_r371308488
##
File path:
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_3.q.out
##
@@ -814,30 +791,20 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE
Column stats: NONE
Group By Operator
aggregations: compute_stats(a, 'hll'), compute_stats(b,
'hll')
-minReductionHashAggr: 0.99
-mode: hash
+mode: complete
Review comment:
Enabling parallelism when inserting eliminates this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377703)
Time Spent: 1.5h (was: 1h 20m)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?focusedWorklogId=377702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377702
]
ASF GitHub Bot logged work on HIVE-22538:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 15:30
Start Date: 27/Jan/20 15:30
Worklog Time Spent: 10m
Work Description: kasakrisz commented on pull request #877: HIVE-22538:
RS deduplication does not always enforce
hive.optimize.reducededuplication.min.reducer
URL: https://github.com/apache/hive/pull/877#discussion_r371308303
##
File path:
ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_3.q.out
##
@@ -261,31 +260,21 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE
Column stats: NONE
Group By Operator
aggregations: compute_stats(a, 'hll'), compute_stats(b,
'hll')
-minReductionHashAggr: 0.99
-mode: hash
+mode: complete
Review comment:
Enabling parallelism when inserting eliminates this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377702)
Time Spent: 1h 20m (was: 1h 10m)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?focusedWorklogId=377700=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377700
]
ASF GitHub Bot logged work on HIVE-22538:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 15:29
Start Date: 27/Jan/20 15:29
Worklog Time Spent: 10m
Work Description: kasakrisz commented on pull request #877: HIVE-22538:
RS deduplication does not always enforce
hive.optimize.reducededuplication.min.reducer
URL: https://github.com/apache/hive/pull/877#discussion_r371307699
##
File path: ql/src/test/results/clientpositive/autoColumnStats_4.q.out
##
@@ -128,16 +127,20 @@ STAGE PLANS:
Statistics: Num rows: 10 Data size: 1728 Basic stats: COMPLETE
Column stats: COMPLETE
Group By Operator
aggregations: compute_stats(a, 'hll'), compute_stats(b, 'hll')
- minReductionHashAggr: 0.99
- mode: hash
+ mode: complete
Review comment:
Enabling parallelism when inserting eliminates this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377700)
Time Spent: 1h 10m (was: 1h)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?focusedWorklogId=377698=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377698
]
ASF GitHub Bot logged work on HIVE-22538:
-
Author: ASF GitHub Bot
Created on: 27/Jan/20 15:28
Start Date: 27/Jan/20 15:28
Worklog Time Spent: 10m
Work Description: kasakrisz commented on pull request #877: HIVE-22538:
RS deduplication does not always enforce
hive.optimize.reducededuplication.min.reducer
URL: https://github.com/apache/hive/pull/877#discussion_r371306847
##
File path: ql/src/test/results/clientpositive/acid_table_directories_test.q.out
##
@@ -154,6 +154,7 @@ POSTHOOK: Input: default@acidparttbl@p=200
### ACID BASE DIR ###
### ACID BASE DIR ###
### ACID BASE DIR ###
+### ACID BASE DIR ###
Review comment:
Enabling parallelism when inserting eliminates this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 377698)
Time Spent: 1h (was: 50m)
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.2.patch, HIVE-22538.3.patch,
> HIVE-22538.4.patch, HIVE-22538.5.patch, HIVE-22538.6.patch,
> HIVE-22538.6.patch, HIVE-22538.7.patch, HIVE-22538.8.patch, HIVE-22538.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22680) Replace Base64 in druid-handler Package
[ https://issues.apache.org/jira/browse/HIVE-22680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17024428#comment-17024428 ] Zoltan Haindrich commented on HIVE-22680: - +1 > Replace Base64 in druid-handler Package > --- > > Key: HIVE-22680 > URL: https://issues.apache.org/jira/browse/HIVE-22680 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22680.1.patch, HIVE-22680.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
