[jira] [Work logged] (HIVE-23583) Upgrade to ant 1.10.9 due to CVEs
[ https://issues.apache.org/jira/browse/HIVE-23583?focusedWorklogId=529336=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529336 ] ASF GitHub Bot logged work on HIVE-23583: - Author: ASF GitHub Bot Created on: 30/Dec/20 01:05 Start Date: 30/Dec/20 01:05 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1599: URL: https://github.com/apache/hive/pull/1599 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 529336) Time Spent: 0.5h (was: 20m) > Upgrade to ant 1.10.9 due to CVEs > - > > Key: HIVE-23583 > URL: https://issues.apache.org/jira/browse/HIVE-23583 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: Renukaprasad C >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23583.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update ANT to fix: > CVE-2020-1945: Apache Ant insecure temporary file vulnerability > Severity: Medium > Vendor: > The Apache Software Foundation > Versions Affected: > Apache Ant 1.1 to 1.9.14 and 1.10.0 to 1.10.7 > Description: > Apache Ant uses the default temporary directory identified by the Java > system property java.io.tmpdir for several tasks and may thus leak > sensitive information. The fixcrlf and replaceregexp tasks also copy > files from the temporary directory back into the build tree allowing an > attacker to inject modified source files into the build process. > Mitigation: > Ant users of versions 1.1 to 1.9.14 and 1.10.0 to 1.10.7 should set the > java.io.tmpdir system property to point to a directory only readable and > writable by the current user prior to running Ant. > Users of versions 1.9.15 and 1.10.8 can use the Ant property ant.tmpfile > instead. Users of Ant 1.10.8 can rely on Ant protecting the temporary > files if the underlying filesystem allows it, but we still recommend > using a private temporary directory instead. > References: > https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-1945 > https://nvd.nist.gov/vuln/detail/CVE-2020-1945 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22415) Upgrade to Java 11
[ https://issues.apache.org/jira/browse/HIVE-22415?focusedWorklogId=529335=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529335 ] ASF GitHub Bot logged work on HIVE-22415: - Author: ASF GitHub Bot Created on: 30/Dec/20 01:05 Start Date: 30/Dec/20 01:05 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1624: URL: https://github.com/apache/hive/pull/1624#issuecomment-752292164 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 529335) Time Spent: 4h 10m (was: 4h) > Upgrade to Java 11 > -- > > Key: HIVE-22415 > URL: https://issues.apache.org/jira/browse/HIVE-22415 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Critical > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > Upgrade Hive to Java JDK 11 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=529337=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529337
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 30/Dec/20 01:05
Start Date: 30/Dec/20 01:05
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1588:
URL: https://github.com/apache/hive/pull/1588
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529337)
Time Spent: 1h 10m (was: 1h)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24242) Relax safety checks in SharedWorkOptimizer
[ https://issues.apache.org/jira/browse/HIVE-24242?focusedWorklogId=529338=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529338 ] ASF GitHub Bot logged work on HIVE-24242: - Author: ASF GitHub Bot Created on: 30/Dec/20 01:05 Start Date: 30/Dec/20 01:05 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1564: URL: https://github.com/apache/hive/pull/1564#issuecomment-752292179 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 529338) Time Spent: 0.5h (was: 20m) > Relax safety checks in SharedWorkOptimizer > -- > > Key: HIVE-24242 > URL: https://issues.apache.org/jira/browse/HIVE-24242 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > there are some checks to lock out problematic cases > For UnionOperator > [here|https://github.com/apache/hive/blob/1507d80fd47aad38b87bba4fd58c1427ba89dbbf/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SharedWorkOptimizer.java#L1571] > This check could prevent the optimization even if the Union is only visible > from only 1 of the TS ops. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24309) Simplify ConvertJoinMapJoin logic
[ https://issues.apache.org/jira/browse/HIVE-24309?focusedWorklogId=529334=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529334 ] ASF GitHub Bot logged work on HIVE-24309: - Author: ASF GitHub Bot Created on: 30/Dec/20 01:05 Start Date: 30/Dec/20 01:05 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1606: URL: https://github.com/apache/hive/pull/1606 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 529334) Time Spent: 0.5h (was: 20m) > Simplify ConvertJoinMapJoin logic > -- > > Key: HIVE-24309 > URL: https://issues.apache.org/jira/browse/HIVE-24309 > Project: Hive > Issue Type: Improvement >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > ConvertMapJoin logic can be further simplified: > [https://github.com/pgaref/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java#L92] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24334) SyntheticJoinPredicate creation may be missed when ReduceSink has Join input
[
https://issues.apache.org/jira/browse/HIVE-24334?focusedWorklogId=529332=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529332
]
ASF GitHub Bot logged work on HIVE-24334:
-
Author: ASF GitHub Bot
Created on: 30/Dec/20 01:04
Start Date: 30/Dec/20 01:04
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1631:
URL: https://github.com/apache/hive/pull/1631#issuecomment-752292163
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529332)
Time Spent: 20m (was: 10m)
> SyntheticJoinPredicate creation may be missed when ReduceSink has Join input
>
>
> Key: HIVE-24334
> URL: https://issues.apache.org/jira/browse/HIVE-24334
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Let assume we have a plan where some Reduce Sink operators has a Join
> operator as input:
> {code}
> TS[33]-FIL[34]-SEL[35]-RS[42]-JOIN[44]-RS[45]-JOIN[47]
> TS[36]-FIL[37]-SEL[38]-RS[43]-JOIN[44]
> TS[39]-FIL[40]-SEL[41]-RS[46]-JOIN[47]
> {code}
> RS[45] inputs is JOIN[44].
> When searching for additional opportunities to create additional
> SyntheticJoinPredicates _ExprNodeDescUtils.backtrack_ does not return the
> input expression of the expression in the join operator but the expression
> itself.
> This is caused by
> - if the operator is a join operator where we create the join predicate
> derivatives the expression is not resolved
> https://github.com/apache/hive/blob/375433510b73c5a22bde4e13485dfc16eaa24706/ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java#L400
> - later the backtrack algorithm does the resolution of the expression doesn't
> do any iterations since it is already in a terminal state when it is called
> https://github.com/apache/hive/blob/375433510b73c5a22bde4e13485dfc16eaa24706/ql/src/java/org/apache/hadoop/hive/ql/ppd/SyntheticJoinPredicate.java#L414
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24307) Beeline with property-file and -e parameter is failing
[
https://issues.apache.org/jira/browse/HIVE-24307?focusedWorklogId=529333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529333
]
ASF GitHub Bot logged work on HIVE-24307:
-
Author: ASF GitHub Bot
Created on: 30/Dec/20 01:04
Start Date: 30/Dec/20 01:04
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1603:
URL: https://github.com/apache/hive/pull/1603
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529333)
Time Spent: 0.5h (was: 20m)
> Beeline with property-file and -e parameter is failing
> --
>
> Key: HIVE-24307
> URL: https://issues.apache.org/jira/browse/HIVE-24307
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-24307-01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Beeline query with property file specified with -e parameter fails with :
> {noformat}
> Cannot run commands specified using -e. No current connection
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-18284) NPE when inserting data with 'distribute by' clause with dynpart sort optimization
[
https://issues.apache.org/jira/browse/HIVE-18284?focusedWorklogId=529331=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529331
]
ASF GitHub Bot logged work on HIVE-18284:
-
Author: ASF GitHub Bot
Created on: 30/Dec/20 01:04
Start Date: 30/Dec/20 01:04
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1400:
URL: https://github.com/apache/hive/pull/1400#issuecomment-752292183
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529331)
Time Spent: 2h 40m (was: 2.5h)
> NPE when inserting data with 'distribute by' clause with dynpart sort
> optimization
> --
>
> Key: HIVE-18284
> URL: https://issues.apache.org/jira/browse/HIVE-18284
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 2.3.1, 2.3.2, 3.0.0, 3.1.1, 3.1.2, 4.0.0
>Reporter: Aki Tanaka
>Assignee: Syed Shameerur Rahman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> A Null Pointer Exception occurs when inserting data with 'distribute by'
> clause. The following snippet query reproduces this issue:
> *(non-vectorized , non-llap mode)*
> {code:java}
> create table table1 (col1 string, datekey int);
> insert into table1 values ('ROW1', 1), ('ROW2', 2), ('ROW3', 1);
> create table table2 (col1 string) partitioned by (datekey int);
> set hive.vectorized.execution.enabled=false;
> set hive.optimize.sort.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> insert into table table2
> PARTITION(datekey)
> select col1,
> datekey
> from table1
> distribute by datekey ;
> {code}
> I could run the insert query without the error if I remove Distribute By or
> use Cluster By clause.
> It seems that the issue happens because Distribute By does not guarantee
> clustering or sorting properties on the distributed keys.
> FileSinkOperator removes the previous fsp. FileSinkOperator will remove the
> previous fsp which might be re-used when we use Distribute By.
> https://github.com/apache/hive/blob/branch-2.3/ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java#L972
> The following stack trace is logged.
> {code:java}
> Vertex failed, vertexName=Reducer 2, vertexId=vertex_1513111717879_0056_1_01,
> diagnostics=[Task failed, taskId=task_1513111717879_0056_1_01_00,
> diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task (
> failure ) :
> attempt_1513111717879_0056_1_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0) {"key":{},"value":{"_col0":"ROW3","_col1":1}}
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
>
[jira] [Work logged] (HIVE-24571) Redundant code in SemanticAnalyzer.java
[ https://issues.apache.org/jira/browse/HIVE-24571?focusedWorklogId=529117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529117 ] ASF GitHub Bot logged work on HIVE-24571: - Author: ASF GitHub Bot Created on: 29/Dec/20 11:31 Start Date: 29/Dec/20 11:31 Worklog Time Spent: 10m Work Description: zzccctv opened a new pull request #1819: URL: https://github.com/apache/hive/pull/1819 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 529117) Time Spent: 40m (was: 0.5h) > Redundant code in SemanticAnalyzer.java > --- > > Key: HIVE-24571 > URL: https://issues.apache.org/jira/browse/HIVE-24571 > Project: Hive > Issue Type: Improvement >Affects Versions: 2.3.7 >Reporter: zhangzhanchang >Assignee: Miklos Gergely >Priority: Minor > Labels: pull-request-available > Attachments: image-2020-12-27-16-42-59-875.png > > Time Spent: 40m > Remaining Estimate: 0h > > In the newly released release-2.3.7, the file SemanticAnalyzer.java There are > redundant codes，Other branche-2 also have this problem > !image-2020-12-27-16-42-59-875.png|width=781,height=411! -- This message was sent by Atlassian Jira (v8.3.4#803005)
