[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=797517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797517 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 03/Aug/22 06:46 Start Date: 03/Aug/22 06:46 Worklog Time Spent: 10m Work Description: abstractdog commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1203550667 also, let me grab the opportunity to thank @belugabehr who put enormous efforts into the hadoop upgrade in the early days! Issue Time Tracking --- Worklog Id: (was: 797517) Time Spent: 15.05h (was: 14h 53m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Time Spent: 15.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=797506=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797506 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 03/Aug/22 05:32 Start Date: 03/Aug/22 05:32 Worklog Time Spent: 10m Work Description: ayushtkn merged PR #3279: URL: https://github.com/apache/hive/pull/3279 Issue Time Tracking --- Worklog Id: (was: 797506) Time Spent: 14h 43m (was: 14.55h) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=797507=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797507 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 03/Aug/22 05:32 Start Date: 03/Aug/22 05:32 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1203503443 Merged. Thanx @abstractdog, @kgyrtkirk and @steveloughran for helping with reviews. :-) Issue Time Tracking --- Worklog Id: (was: 797507) Time Spent: 14h 53m (was: 14h 43m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=797152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-797152 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 02/Aug/22 08:44 Start Date: 02/Aug/22 08:44 Worklog Time Spent: 10m Work Description: abstractdog commented on code in PR #3279: URL: https://github.com/apache/hive/pull/3279#discussion_r935282547 ## ql/src/test/queries/clientpositive/acid_table_directories_test.q: ## @@ -1,3 +1,5 @@ +--! qt:disabled:disabled Tests the output of LS and that changes, Not a functional test, just adds some masking logic Review Comment: please include hadoop upgrade as the cause in this comment, otherwise looks good to me Issue Time Tracking --- Worklog Id: (was: 797152) Time Spent: 14.55h (was: 14h 23m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=796285=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-796285 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 29/Jul/22 08:01 Start Date: 29/Jul/22 08:01 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1198993149 Got a new Test failure due to jetty. TestSSL, Fixed in the latest commit. Issue Time Tracking --- Worklog Id: (was: 796285) Time Spent: 14h 23m (was: 14h 13m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793735=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793735 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 21/Jul/22 13:32 Start Date: 21/Jul/22 13:32 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1191490566 >But Hive 3.1.x version is not very old and 4.x still looks like in alpha so we may not able to upgrade. so with this PR still we have compatibility issues with Hive 3.x version. any suggestions? thanks @sujith71955 Unfortunately, I don't have a use case for 3.x line, but that should be doable but would requires changes across Hadoop, Hive & Tez. We did the same here as well It is certainly doable, if you folks have a use case, feel free to create a Jira for 3.1.x line. Running busy so couldn't spare time to check the problem you folks stated above.. Issue Time Tracking --- Worklog Id: (was: 793735) Time Spent: 14h 13m (was: 14.05h) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793236=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793236 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 13:21 Start Date: 20/Jul/22 13:21 Worklog Time Spent: 10m Work Description: sujith71955 commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1190280390 > @ayushtkn your base branch is does not seems to be based on 3.1.x release branch. It does not have `FairSchedulerShim.java` https://github.com/ayushtkn/hive/tree/HIVE-24484/shims/scheduler. It fails if base branch is cut from 3.1.x... > > Check https://github.com/apache/hive/blob/release-3.1.3-rc3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java#L31 makes reference to `QueuePlacementPolicy` which is package protected final class https://github.com/apache/hadoop/blame/release-3.3.3-RC1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java#L54 > > Is there any plan to add support in 3.1.x version of hive? But Hive 3.1.x version is not very old and 4.x still looks like in alpha so we may not able to upgrade. so with this PR still we have compatibility issues with Hive 3.x version. any suggestions? thanks cc @sankarh Issue Time Tracking --- Worklog Id: (was: 793236) Time Spent: 14.05h (was: 13h 53m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 14.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793209=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793209 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 12:14 Start Date: 20/Jul/22 12:14 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1190200900 it's targeting 3.3.3+; there is actually a 3.3.4 RC coming out today with specific changes to assist tez (HADOOP-18332). Issue Time Tracking --- Worklog Id: (was: 793209) Time Spent: 13h 53m (was: 13h 43m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793208=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793208 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 12:14 Start Date: 20/Jul/22 12:14 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1190200835 Nopes, not chasing that branch Issue Time Tracking --- Worklog Id: (was: 793208) Time Spent: 13h 43m (was: 13.55h) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793186=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793186 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 11:54 Start Date: 20/Jul/22 11:54 Worklog Time Spent: 10m Work Description: suryanshagnihotri commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1190181001 @ayushtkn your base branch is does not seems to be based on 3.1.x release branch. It does not have `FairSchedulerShim.java` https://github.com/ayushtkn/hive/tree/HIVE-24484/shims/scheduler. It fails if base branch is cut from 3.1.x... Is there any plan to add support in 3.1.x version of hive? Issue Time Tracking --- Worklog Id: (was: 793186) Time Spent: 13.55h (was: 13h 23m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793123=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793123 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 09:27 Start Date: 20/Jul/22 09:27 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1190044149 @suryanshagnihotri If this was the case, the build would have failed, which it didn't. So, we are cool here. Nothing pending here apart from awaiting an official Tez release here.. Issue Time Tracking --- Worklog Id: (was: 793123) Time Spent: 13h 23m (was: 13h 13m) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1 And Tez to 0.10.2
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=793017=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-793017 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 20/Jul/22 04:44 Start Date: 20/Jul/22 04:44 Worklog Time Spent: 10m Work Description: suryanshagnihotri commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1189823384 @ayushtkn Did you not face this compilation error. I do not see any change in `FairSchedulerShim.java`. I compiled hive 3.1.2 with hadoop 3.3.1. `Compilation failure [ERROR] /Users/suryansh/Documents/BDS/apache_hive/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[31,68] org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy is not public in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair; cannot be accessed from outside package` Issue Time Tracking --- Worklog Id: (was: 793017) Time Spent: 13h 13m (was: 13.05h) > Upgrade Hadoop to 3.3.1 And Tez to 0.10.2 > -- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=788033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788033 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 05/Jul/22 19:43 Start Date: 05/Jul/22 19:43 Worklog Time Spent: 10m Work Description: abstractdog commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1175431162 > Thanx @abstractdog We have a green build here, so I have updated the PR title, while merging they will get squashed automatically :-) makes sense, thanks! I wish we could merge this now, hopefully we can release tez in 2 weeks I can see the PTF boolean patch in the commits, it's not intentional I guess Issue Time Tracking --- Worklog Id: (was: 788033) Time Spent: 13.05h (was: 12h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 13.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=788002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-788002 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 05/Jul/22 18:33 Start Date: 05/Jul/22 18:33 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1175369204 Thanx @abstractdog We have a green build here, so I have updated the PR title, while merging they will get squashed automatically :-) Issue Time Tracking --- Worklog Id: (was: 788002) Time Spent: 12h 53m (was: 12h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 12h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=787883=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-787883 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 05/Jul/22 14:53 Start Date: 05/Jul/22 14:53 Worklog Time Spent: 10m Work Description: abstractdog commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1175156597 > These bunch of failures looks like due to Hadoop only, Since they are passing with my Hadoop Upgrade PR. We can merge that first, then rebase this and merge post that? if precommit tests cannot pass cleanly without both upgrades (hadoop+tez), we should commit those together also (because we cannot even revert them later one by one if needed), in this case, jira title and commit message might want to contain hadoop and tez upgrade too Issue Time Tracking --- Worklog Id: (was: 787883) Time Spent: 12h 43m (was: 12.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 12h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=787757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-787757 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 05/Jul/22 08:26 Start Date: 05/Jul/22 08:26 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1174768913 These bunch of failures looks like due to Hadoop only, Since they are passing with my Hadoop Upgrade PR. We can merge that first, then rebase this and merge post that? Issue Time Tracking --- Worklog Id: (was: 787757) Time Spent: 12.55h (was: 12h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 12.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776830 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 01/Jun/22 12:02 Start Date: 01/Jun/22 12:02 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1143517119 jetty upgrade came in https://issues.apache.org/jira/browse/HADOOP-17796 & https://github.com/apache/hadoop/pull/3208 some security advisories there so it is probably better to deal with the change than try and stick to the older version. sorry Issue Time Tracking --- Worklog Id: (was: 776830) Time Spent: 11h 53m (was: 11h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 11h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776203
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 31/May/22 10:17
Start Date: 31/May/22 10:17
Worklog Time Spent: 10m
Work Description: abstractdog commented on PR #3279:
URL: https://github.com/apache/hive/pull/3279#issuecomment-1141946273
> I've deployed
>
> * hadoop-3.3.3
> * tez 0.10.1
> * hive from the PR
> running a simple insert failed with:
>
> ```
> Caused by: java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
>at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:569)
>at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:550)
>at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:117)
>at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:425)
>at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:341)
>at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:432)
>at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:428)
>at
org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:94)
>at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
>at
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1800)
>at
org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1821)
> 2022-05-31 09:17:19,422 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
> ```
>
> maybe I've missed something - but it seems like the tez dagappmaster has
issues running with the jetty because of hadoop 3.3.3
>
> I've these settings:
>
> ```
> tez/tez-site/tez.lib.uris ${fs.defaultFS}/apps/tez/tez.tar.gz
> tez/tez-site/tez.use.cluster.hadoop-libs true
> ```
>
> @abstractdog is hadoop-3.3.3 supported with tez-0.10.1?
thanks @kgyrtkirk for trying this out, I've just created TEZ-4420, as tez is
on hadoop 3.3.1, and I'm not sure about compatibility
checked tez.tar.gz contents before and after bump and I got:
```
hadoop 3.3.1
tar tf tez-dist/target/tez-0.10.2-SNAPSHOT.tar.gz | grep jetty
lib/jetty-server-9.4.40.v20210413.jar
lib/jetty-http-9.4.40.v20210413.jar
lib/jetty-io-9.4.40.v20210413.jar
lib/jetty-util-9.4.40.v20210413.jar
lib/jetty-servlet-9.4.40.v20210413.jar
lib/jetty-security-9.4.40.v20210413.jar
lib/jetty-util-ajax-9.4.40.v20210413.jar
lib/jetty-webapp-9.4.40.v20210413.jar
lib/jetty-xml-9.4.40.v20210413.jar
lib/jetty-client-9.4.40.v20210413.jar
hadoop 3.3.3
tar tf tez-dist/target/tez-0.10.2-SNAPSHOT.tar.gz | grep jetty
lib/jetty-server-9.4.43.v20210629.jar
lib/jetty-http-9.4.43.v20210629.jar
lib/jetty-io-9.4.43.v20210629.jar
lib/jetty-util-9.4.43.v20210629.jar
lib/jetty-servlet-9.4.43.v20210629.jar
lib/jetty-security-9.4.43.v20210629.jar
lib/jetty-util-ajax-9.4.43.v20210629.jar
lib/jetty-webapp-9.4.43.v20210629.jar
lib/jetty-xml-9.4.43.v20210629.jar
lib/jetty-client-9.4.43.v20210629.jar
```
tez packs jetty from hadoop, so hadoop upgrade means jetty upgrade too, so
there is a chance that an old tez.tar.gz can clash with new hadoop
I've just uplodaded the new tez.tar.gz, is there a chance you can give it a
try?
https://drive.google.com/file/d/18RMfh40s6kKdFt77E7HpS-j4EJhd2DKi/view?usp=sharing
Issue Time Tracking
---
Worklog Id: (was: 776203)
Time Spent: 11h 43m (was: 11.55h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 11h 43m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776171=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776171 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 31/May/22 09:26 Start Date: 31/May/22 09:26 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on code in PR #3279: URL: https://github.com/apache/hive/pull/3279#discussion_r885363228 ## ql/src/test/queries/clientpositive/acid_table_directories_test.q: ## @@ -1,3 +1,5 @@ +--! qt:disabled:disabled Tests the output of LS and that changes, Not a functional test, just adds some masking logic Review Comment: you may also remove this testand/or open a jira to remove the things which were added in HIVE-21650; I think `qt:replace` could do the same.. hmm...it seems like `hive.qtest.additional.partial.mask.pattern` is only used in this test and nowhere else... ## ql/src/test/results/clientpositive/llap/acid_table_directories_test.q.out: ## @@ -163,13 +170,6 @@ POSTHOOK: Input: default@acidparttbl@p=200 ### ACID DELTA DIR ### ### ACID DELTA DIR ### ### ACID DELTA DIR ### - A masked pattern was here Review Comment: I would guess the directory listing order might have changed... note: I think we should have better masking policies instead of removing the whole lines (mask only the WH part of the path)...it could be important what was the directory... Issue Time Tracking --- Worklog Id: (was: 776171) Time Spent: 11.55h (was: 11h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 11.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776170=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776170
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 31/May/22 09:25
Start Date: 31/May/22 09:25
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on PR #3279:
URL: https://github.com/apache/hive/pull/3279#issuecomment-1141893046
I've deployed
* hadoop-3.3.3
* tez 0.10.1
* hive from the PR
running a simple insert failed with:
```
Caused by: java.lang.NoSuchMethodError:
org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:569)
at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:550)
at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:117)
at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:425)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:341)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:432)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:428)
at
org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:94)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at
org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1800)
at
org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1821)
2022-05-31 09:17:19,422 [INFO] [shutdown-hook-0] |app.DAGAppMaster|:
DAGAppMasterShutdownHook invoked
```
maybe I've missed something - but it seems like the tez dagappmaster has
issues running with the jetty because of hadoop 3.3.3
I've these settings:
```
tez/tez-site/tez.lib.uris ${fs.defaultFS}/apps/tez/tez.tar.gz
tez/tez-site/tez.use.cluster.hadoop-libs true
```
@abstractdog is hadoop-3.3.3 supported with tez-0.10.1?
Issue Time Tracking
---
Worklog Id: (was: 776170)
Time Spent: 11h 23m (was: 11h 13m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 11h 23m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776152
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 31/May/22 09:10
Start Date: 31/May/22 09:10
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r885395534
##
ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java:
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
Review Comment:
Does this log line work:
```
LOG.info("Interrupted while getting the input reader for {}",
split.getInputSplit());
```
For 2nd I suppose it can get interrupted on any abort, not very sure, do you
have any suggestions
Issue Time Tracking
---
Worklog Id: (was: 776152)
Time Spent: 11h 13m (was: 11.05h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 11h 13m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776124=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776124 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 31/May/22 08:21 Start Date: 31/May/22 08:21 Worklog Time Spent: 10m Work Description: abstractdog commented on code in PR #3279: URL: https://github.com/apache/hive/pull/3279#discussion_r885344061 ## ql/src/test/results/clientpositive/llap/acid_table_directories_test.q.out: ## @@ -163,13 +170,6 @@ POSTHOOK: Input: default@acidparttbl@p=200 ### ACID DELTA DIR ### ### ACID DELTA DIR ### ### ACID DELTA DIR ### - A masked pattern was here Review Comment: in case of a hive patch, usually, I don't care about result ordering change, however, this is a hadoop upgrade, this is not expected, can we explain this change? Issue Time Tracking --- Worklog Id: (was: 776124) Time Spent: 11.05h (was: 10h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 11.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=776123=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776123
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 31/May/22 08:19
Start Date: 31/May/22 08:19
Worklog Time Spent: 10m
Work Description: abstractdog commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r885341338
##
ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java:
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
Review Comment:
the explanation on the other PR makes sense to me...considering that we only
catch InterruptedIOException here, I'm fine with ZeroRowsInputFormat, but for
future code readers, this is confusing, let's do the following:
1. put a log line here marking this branch
2. make a comment: in which scenarios does this happen?, who's typically
interrupting this codepath?
Issue Time Tracking
---
Worklog Id: (was: 776123)
Time Spent: 10h 53m (was: 10h 43m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10h 53m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=773645=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773645 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 23/May/22 18:34 Start Date: 23/May/22 18:34 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1135008475 LGTM. spark has gone up to the same version last week, incidentally Issue Time Tracking --- Worklog Id: (was: 773645) Time Spent: 10h 43m (was: 10.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 10h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=772469=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-772469 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 19/May/22 14:34 Start Date: 19/May/22 14:34 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1131786459 The build is green with ``3.3.3`` I built the distro and checked if it contains reload4j, it doesn't ``` lib % ls -l | grep reload4j lib % ``` Deployed and tried with hadoop-3.3.3, Hive on MR and ran some basic queries and they were working. @steveloughran do we need anything more or 3.3.3 or are we good Issue Time Tracking --- Worklog Id: (was: 772469) Time Spent: 10.55h (was: 10h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 10.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=771991=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-771991 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 18/May/22 15:52 Start Date: 18/May/22 15:52 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1130195121 excluding reload4j is harmless on versions without those artifacts, so safe to add and not worry too much. bad xmll can happen if the test reporter doesn't escape test names properly and you've managed to get some invalid xml in there. do you have any parameterized tests? check how the strings are created, as they get included. CI tools generally aren't paranoid enough about test method names as historically it was only a java method name Issue Time Tracking --- Worklog Id: (was: 771991) Time Spent: 10h 23m (was: 10h 13m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 10h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=771943=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-771943 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 18/May/22 15:03 Start Date: 18/May/22 15:03 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1130131300 >test not fully flushing/closing xml file before trying to read it? Looks like some maven issue only, it tries to compute diff between the generated query output file & the stored query output file, if the diff is empty, the test is said to be passing else failed. else it puts the diff in the xml, Guess diff has some character which is messing up the xml structure, I need to further investigate though > the move in 3.3.3 to reload4j might add some exclusion complications if hive is declaring its own logging classes. I just pushed a commit upgrading to 3.3.3, I haven't tested the distro, but the compilation & ran one test. Do I need to exclude reload4j from every hadoop dependency? If it creates some runtime issues, I think I am happy moving from 3.1.0 to 3.3.2 and rest wait for 3.4.0. Even if I exclude next time when I move to 3.4.x or above I need to take those changes back, right? Issue Time Tracking --- Worklog Id: (was: 771943) Time Spent: 10h 13m (was: 10.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 10h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770902=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770902 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 16/May/22 16:09 Start Date: 16/May/22 16:09 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1127862030 > Can't decode the failure reason here, it was that broken test which was causing this test not fully flushing/closing xml file before trying to read it? changes look ok to me; the move in 3.3.3 to reload4j might add some exclusion complications if hive is declaring its own logging classes. Issue Time Tracking --- Worklog Id: (was: 770902) Time Spent: 10.05h (was: 9h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 10.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770111=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770111
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 13/May/22 10:10
Start Date: 13/May/22 10:10
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r872219527
##
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java:
##
@@ -315,18 +320,19 @@ public void testOutputFormat() throws Throwable {
// Check permisssion on partition dirs and files created
for (int i = 0; i < tableNames.length; i++) {
- Path partitionFile = new Path(warehousedir + "/" + tableNames[i]
-+ "/ds=1/cluster=ag/part-m-0");
- FileSystem fs = partitionFile.getFileSystem(mrConf);
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-fs.getFileStatus(partitionFile).getPermission(),
-new FsPermission(tablePerms[i]));
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-fs.getFileStatus(partitionFile.getParent()).getPermission(),
-new FsPermission(tablePerms[i]));
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-
fs.getFileStatus(partitionFile.getParent().getParent()).getPermission(),
-new FsPermission(tablePerms[i]));
+ final Path partitionFile = new Path(warehousedir + "/" + tableNames[i] +
"/ds=1/cluster=ag/part-m-0");
+ final Path grandParentOfPartitionFile = partitionFile.getParent();
Review Comment:
Changed. I picked it as is from the previous PR, when I saw this test
failing :-)
Issue Time Tracking
---
Worklog Id: (was: 770111)
Time Spent: 9h 53m (was: 9h 43m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 53m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770110=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770110 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 13/May/22 10:09 Start Date: 13/May/22 10:09 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1125879377 The only test failure here is intermittent. Have answered/addressed all the comments. One test I have disabled, firstly it was not failing itself but was corrupting the XML, it wasn't functional, but some test infra stuff and relying on hadoop ls command, for which output was intermittently changing. Not a good test to have either. For record it is this: http://ci.hive.apache.org/job/hive-precommit/job/PR-3279/7/testReport/junit/TEST-org.apache.hadoop.hive.cli.split0.TestMiniLlapLocalCliDriver/xml/_failed_to_read_/ Can't decode the failure reason here, it was that broken test which was causing this. If everything is good here, and only this test block. I will have a followup jira and figure this test out with the original author of the test. I have tried basic stuff with Hive-On-MR. Issue Time Tracking --- Worklog Id: (was: 770110) Time Spent: 9h 43m (was: 9.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 9h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770105=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770105
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 13/May/22 10:04
Start Date: 13/May/22 10:04
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r872214761
##
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java:
##
@@ -123,57 +122,24 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws Throwable {
put(HiveConf.ConfVars.REPLDIR.varname, primary.repldDir);
}}, "test_key123");
-List dumpWithClause = Arrays.asList(
-"'hive.repl.add.raw.reserved.namespace'='true'",
-"'" + HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname +
"'='"
-+ replica.externalTableWarehouseRoot + "'",
-"'distcp.options.skipcrccheck'=''",
-"'" + HiveConf.ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname +
"'='false'",
-"'" + HiveConf.ConfVars.HIVE_DISTCP_DOAS_USER.varname + "'='"
-+ UserGroupInformation.getCurrentUser().getUserName()
+"'");
-WarehouseInstance.Tuple tuple =
-primary.run("use " + primaryDbName)
-.run("create table encrypted_table (id int, value string)")
-.run("insert into table encrypted_table values
(1,'value1')")
-.run("insert into table encrypted_table values
(2,'value2')")
-.dump(primaryDbName, dumpWithClause);
-
-replica
-.run("repl load " + primaryDbName + " into " + replicatedDbName
-+ " with('hive.repl.add.raw.reserved.namespace'='true', "
-+ "'hive.repl.replica.external.table.base.dir'='" +
replica.externalTableWarehouseRoot + "', "
-+ "'hive.exec.copyfile.maxsize'='0',
'distcp.options.skipcrccheck'='')")
-.run("use " + replicatedDbName)
-.run("repl status " + replicatedDbName)
-.verifyResult(tuple.lastReplicationId);
-
-try {
- replica
- .run("select value from encrypted_table")
- .verifyResults(new String[] { "value1", "value2" });
- Assert.fail("Src EZKey shouldn't be present on target");
-} catch (IOException e) {
- Assert.assertTrue(e.getCause().getMessage().contains("KeyVersion name
'test_key@0' does not exist"));
-}
-
//read should pass without raw-byte distcp
-dumpWithClause = Arrays.asList( "'" +
HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname + "'='"
+List dumpWithClause = Arrays.asList( "'" +
HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname + "'='"
+ replica.externalTableWarehouseRoot + "'");
-tuple = primary.run("use " + primaryDbName)
+WarehouseInstance.Tuple tuple =
+primary.run("use " + primaryDbName)
.run("create external table encrypted_table2 (id int, value
string)")
.run("insert into table encrypted_table2 values (1,'value1')")
.run("insert into table encrypted_table2 values (2,'value2')")
.dump(primaryDbName, dumpWithClause);
replica
-.run("repl load " + primaryDbName + " into " + replicatedDbName
-+ " with('hive.repl.replica.external.table.base.dir'='" +
replica.externalTableWarehouseRoot + "', "
-+ "'hive.exec.copyfile.maxsize'='0',
'distcp.options.skipcrccheck'='')")
-.run("use " + replicatedDbName)
-.run("repl status " + replicatedDbName)
-.verifyResult(tuple.lastReplicationId)
Review Comment:
DistCp itself fails, It is running with hive.repl.add.raw.reserved.namespace
and you can't copy if the key is not present on target cluster. Earlier I
converted this to a failure case test, but then the next iteration fails which
is without hive.repl.add.raw.reserved.namespace because the last load wasn't
successful, so I kept the success case
##
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java:
##
@@ -123,57 +122,24 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws Throwable {
put(HiveConf.ConfVars.REPLDIR.varname, primary.repldDir);
}}, "test_key123");
-List dumpWithClause = Arrays.asList(
Review Comment:
Same as above:
DistCp itself fails, It is running with hive.repl.add.raw.reserved.namespace
and you can't copy if the key is not present on target cluster. Earlier I
converted this to a failure case test, but then the next iteration fails which
is without hive.repl.add.raw.reserved.namespace because the
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770104
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 13/May/22 10:04
Start Date: 13/May/22 10:04
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r872214538
##
streaming/src/test/org/apache/hive/streaming/TestStreaming.java:
##
@@ -1317,6 +1318,11 @@ public void testTransactionBatchEmptyCommit() throws
Exception {
connection.close();
}
+ /**
+ * Starting with HDFS 3.3.1, the underlying system NOW SUPPORTS hflush so
this
+ * test fails.
Review Comment:
Sure, I have removed the exception assertion. Kept the reason as is.
Just for code context, why HFlush support gets rid of the exception
```
if (!out.hasCapability(StreamCapabilities.HFLUSH)) {
throw new ConnectionError(
"The backing filesystem only supports transaction batch
sizes of 1, but " + transactionBatchSize
+ " was requested.");
}
```
##
common/pom.xml:
##
@@ -195,6 +194,11 @@
tez-api
${tez.version}
+
+ org.fusesource.jansi
+ jansi
+ 2.3.4
Review Comment:
Done
Issue Time Tracking
---
Worklog Id: (was: 770104)
Time Spent: 9h 23m (was: 9h 13m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 23m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=769624=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769624
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 12/May/22 13:07
Start Date: 12/May/22 13:07
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r871357800
##
standalone-metastore/pom.xml:
##
@@ -227,6 +227,10 @@
hadoop-mapreduce-client-core
${hadoop.version}
+
+org.jline
+jline
+
Review Comment:
yeps, the best answer is to upgrade Jline, which was stuck. So, I thought to
upgrade Hadoop that shouldn't block if possible, we are already on 3.1.0 which
died long back
##
storage-api/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java:
##
@@ -18,10 +18,10 @@
package org.apache.hadoop.hive.common;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
Review Comment:
Code doesn't compile with this. It is already marked as banned import, guess
the logic has flaw.
https://github.com/apache/hive/blob/master/pom.xml#L1529
The dependency was getting pulled in from Hadoop & now it isn't there, so I
have to change it to make it compile
Issue Time Tracking
---
Worklog Id: (was: 769624)
Time Spent: 9h 13m (was: 9.05h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9h 13m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=769620=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769620
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 12/May/22 13:05
Start Date: 12/May/22 13:05
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r871355730
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##
@@ -9361,7 +9362,8 @@ public NotificationEventsCountResponse
get_notification_events_count(Notificatio
private void authorizeProxyPrivilege() throws TException {
// Skip the auth in embedded mode or if the auth is disabled
if (!HiveMetaStore.isMetaStoreRemote() ||
-!MetastoreConf.getBoolVar(conf,
ConfVars.EVENT_DB_NOTIFICATION_API_AUTH)) {
+!MetastoreConf.getBoolVar(conf,
ConfVars.EVENT_DB_NOTIFICATION_API_AUTH) ||
conf.getBoolean(HIVE_IN_TEST.getVarname(),
+false)) {
Review Comment:
It is covered via test in
TestReplicationScenarios#testAuthForNotificationAPIs
This method is also used mostly in replication context only I suppose for
getting NotificationLog entries...
Issue Time Tracking
---
Worklog Id: (was: 769620)
Time Spent: 9.05h (was: 8h 53m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 9.05h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=769619=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769619
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 12/May/22 13:04
Start Date: 12/May/22 13:04
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r871355509
##
ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java:
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
Review Comment:
Answer is here & this does fixes a couple of test so I picked it:
https://github.com/apache/hive/pull/1742/files#r674896581
##
itests/pom.xml:
##
@@ -352,6 +352,12 @@
org.apache.hadoop
hadoop-yarn-client
${hadoop.version}
+
+
+org.jline
+jline
+
Review Comment:
Just tried. Started a Hive cluster with derby, init hive db, started HS2,
then beeline.
show databases;
show tables;
create table emp(id int)
insert into emp values (1),(2),(3),(4);
select * from emp;
show create table emp;
Jline was used in Beeline, I think it should have broken that. Let me know
what else can be tested.
##
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.java:
##
@@ -178,8 +178,7 @@ public void authorize(Database db, Privilege[]
readRequiredPriv, Privilege[] wri
private static boolean userHasProxyPrivilege(String user, Configuration
conf) {
try {
- if (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(user, conf,
- HMSHandler.getIPAddress())) {
+ if (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(user, conf,
HMSHandler.getIPAddress())) {
Review Comment:
Max LineLength allowed I guess is 120?
https://github.com/apache/hive/blob/master/checkstyle/checkstyle.xml#L159-L160
Issue Time Tracking
---
Worklog Id: (was: 769619)
Time Spent: 8h 53m (was: 8h 43m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 53m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=769572=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769572
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 12/May/22 12:07
Start Date: 12/May/22 12:07
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r871272145
##
common/pom.xml:
##
@@ -195,6 +194,11 @@
tez-api
${tez.version}
+
+ org.fusesource.jansi
+ jansi
+ 2.3.4
Review Comment:
move version to root pom
##
itests/pom.xml:
##
@@ -352,6 +352,12 @@
org.apache.hadoop
hadoop-yarn-client
${hadoop.version}
+
+
+org.jline
+jline
+
Review Comment:
I'm not sure if this fix will work; it could work for the tests; but you've
just excluded the dependency; I think that will not prevent that dep from
appearing on the classpath during runtime...
have you tested a dist build as well?
##
ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java:
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
Review Comment:
why not simply propagate the `Exception` ?
This will hide away the exception
##
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.java:
##
@@ -178,8 +178,7 @@ public void authorize(Database db, Privilege[]
readRequiredPriv, Privilege[] wri
private static boolean userHasProxyPrivilege(String user, Configuration
conf) {
try {
- if (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(user, conf,
- HMSHandler.getIPAddress())) {
+ if (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(user, conf,
HMSHandler.getIPAddress())) {
Review Comment:
I think max_linelength should be <=100 ; are you using the
`dev-support/eclipse-styles.xml` ?
##
streaming/src/test/org/apache/hive/streaming/TestStreaming.java:
##
@@ -1317,6 +1318,11 @@ public void testTransactionBatchEmptyCommit() throws
Exception {
connection.close();
}
+ /**
+ * Starting with HDFS 3.3.1, the underlying system NOW SUPPORTS hflush so
this
+ * test fails.
Review Comment:
ok; then I think this test could be probably converted into a test which
checks that it works
##
hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatMultiOutputFormat.java:
##
@@ -315,18 +320,19 @@ public void testOutputFormat() throws Throwable {
// Check permisssion on partition dirs and files created
for (int i = 0; i < tableNames.length; i++) {
- Path partitionFile = new Path(warehousedir + "/" + tableNames[i]
-+ "/ds=1/cluster=ag/part-m-0");
- FileSystem fs = partitionFile.getFileSystem(mrConf);
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-fs.getFileStatus(partitionFile).getPermission(),
-new FsPermission(tablePerms[i]));
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-fs.getFileStatus(partitionFile.getParent()).getPermission(),
-new FsPermission(tablePerms[i]));
- Assert.assertEquals("File permissions of table " + tableNames[i] + " is
not correct",
-
fs.getFileStatus(partitionFile.getParent().getParent()).getPermission(),
-new FsPermission(tablePerms[i]));
+ final Path partitionFile = new Path(warehousedir + "/" + tableNames[i] +
"/ds=1/cluster=ag/part-m-0");
+ final Path grandParentOfPartitionFile = partitionFile.getParent();
Review Comment:
I would expect `grandParent` to be parent-of-parent;
I think this change could be revoked - it was more readable earlier; the
last assert now checks for the `parent` dir and not for `parent.parent`; the
second assert was also clobbered
##
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java:
##
@@ -123,57 +122,24 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=769545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-769545 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 12/May/22 11:05 Start Date: 12/May/22 11:05 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1124857444 My Last run here had 4 errors, of which I think I have fixed 3 more. The one remaining is some XML parsing error, which I think might get auto resolved or may be an after affect. @kgyrtkirk I have sorted the JLine issue here as well, which you told in the previous PR. Can you give a check once Issue Time Tracking --- Worklog Id: (was: 769545) Time Spent: 8.55h (was: 8h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 8.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=768909=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768909 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 11/May/22 08:56 Start Date: 11/May/22 08:56 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3279: URL: https://github.com/apache/hive/pull/3279#issuecomment-1123389005 i have a 3.3.3 RC1 coming out this week, if that helps Issue Time Tracking --- Worklog Id: (was: 768909) Time Spent: 8h 23m (was: 8h 13m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 8h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=768876=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768876 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 11/May/22 07:53 Start Date: 11/May/22 07:53 Worklog Time Spent: 10m Work Description: ayushtkn opened a new pull request, #3279: URL: https://github.com/apache/hive/pull/3279 Exploratory: Just to figure out what all breaks and can be fixed here or in next hadoop 3.3 release Issue Time Tracking --- Worklog Id: (was: 768876) Time Spent: 8h 13m (was: 8.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 8h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=726023=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-726023 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 14/Feb/22 02:36 Start Date: 14/Feb/22 02:36 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1742: URL: https://github.com/apache/hive/pull/1742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 726023) Time Spent: 8.05h (was: 7h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 8.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=725861=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-725861 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 14/Feb/22 00:13 Start Date: 14/Feb/22 00:13 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #1742: URL: https://github.com/apache/hive/pull/1742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 725861) Time Spent: 7h 53m (was: 7h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=721553=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-721553 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 06/Feb/22 00:52 Start Date: 06/Feb/22 00:52 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-1030718952 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 721553) Time Spent: 7h 43m (was: 7.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=691553=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-691553 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 07/Dec/21 08:30 Start Date: 07/Dec/21 08:30 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-987685455 @ayushtkn: I've done some digging into the jline3 issue in #2617 ([here](https://github.com/apache/hive/pull/2617#issuecomment-978029623)) and I'm not sure if it was deliberate move to declare jline3 as a dependency of the `hadoop-yarn-client` artifact -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 691553) Time Spent: 7.55h (was: 7h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=686492=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-686492 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 25/Nov/21 13:29 Start Date: 25/Nov/21 13:29 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-979217036 > or even... upgrade to hadoop 3.1.MAX or 3.2.ANYTHING to grab some of the changes we have to cover in upgrading to 3.3.1 Hadoop 3.1 line is EOL and 3.1.MAX & 3.2.Anything doesn't have Guava Shaded. So, guava version mismatch leads to even more issues, From 3.3 only Guava & Protobuf are shaded in hadoop. > so the issue now is that Hadoop 3.3+ is using org.jline v3 and one of the Hive dependencies of sqline has a dependency on jline v2 which is causing a clash If I decode @belugabehr's comment. The problem is only with Jline. If Jline v3 & v2 are compatible, and I see Jline is used by only Yarn-Client, So can we just not exclude the jline dependency while adding yarn-client? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 686492) Time Spent: 7h 23m (was: 7h 13m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=685740=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-685740 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 24/Nov/21 09:11 Start Date: 24/Nov/21 09:11 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-977679326 good question I don't know; but it seems like this PR have stalled! this patch have: * added a lib named `jansi` => this seems like an independent step * removed the explicit guava usage from druid => could we upgrade guava separately? probably not... * jline / jetty related things * lots of other things -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 685740) Time Spent: 7h 13m (was: 7.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=684635=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-684635 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Nov/21 11:27 Start Date: 22/Nov/21 11:27 Worklog Time Spent: 10m Work Description: abstractdog commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-975424894 what are the unresolved blockers of 3.3.1 upgrade at the moment? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 684635) Time Spent: 7.05h (was: 6h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 7.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=684617=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-684617 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Nov/21 10:52 Start Date: 22/Nov/21 10:52 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-975396279 this PR is not making much progress - I think this in its current form will not work; or will not land soon:\ I think it would make sesnse to consider: * split this thing up into some pieces which we could get in... * or even... upgrade to hadoop 3.1.MAX or 3.2.ANYTHING to grab some of the changes we have to cover in upgrading to 3.3.1 instead of waiting this thing to get in with JDK11 support and everything? - what do you guys think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 684617) Time Spent: 6h 53m (was: 6h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=684398=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-684398 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 21/Nov/21 19:36 Start Date: 21/Nov/21 19:36 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-974730579 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 684398) Time Spent: 6h 43m (was: 6.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=684282=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-684282 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 21/Nov/21 00:12 Start Date: 21/Nov/21 00:12 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-974730579 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 684282) Time Spent: 6.55h (was: 6h 23m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6.55h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=648783=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648783 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 09/Sep/21 18:06 Start Date: 09/Sep/21 18:06 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-916319837 OK, so the issue now is that Hadoop 3.3+ is using `org.jline v3` and one of the Hive dependencies of `sqline` has a dependency on `jline v2` which is causing a clash, well, not a clash per se, but it may be possible to address this by including both versions of jline since they have different namespaces. Ugh. I'll see if there is an updated `sqline` library or what that is doing exactly. Maybe it can be replaced with jline totally? I don't really know much about these libraries, just trying to rubik's cube this together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648783) Time Spent: 6h 23m (was: 6h 13m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=648782=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648782 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 09/Sep/21 18:05 Start Date: 09/Sep/21 18:05 Worklog Time Spent: 10m Work Description: belugabehr edited a comment on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-916319837 OK, so the issue now is that Hadoop 3.3+ is using `org.jline v3` and one of the Hive dependencies of `sqline` has a dependency on `jline v2` which is causing a clash, well, not a clash per se, but it may be possible to address this by including both versions of jline since they have different namespaces. Ugh. I'll see if there is an updates sqline or what that is doing exactly. Maybe it can be replaced with jline totally? I don't really know much about these libraries, just trying to rubik's cube this together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648782) Time Spent: 6h 13m (was: 6.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=648781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648781 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 09/Sep/21 18:03 Start Date: 09/Sep/21 18:03 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-916319837 OK, so the issue now is that Hadoop 3.3+ is using `org.jline v3` and one of the Hive dependencies of `sqline` has a dependency on `jline v2` which is causing a clash. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648781) Time Spent: 6.05h (was: 5h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 6.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=648596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648596 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 09/Sep/21 13:23 Start Date: 09/Sep/21 13:23 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-916087565 @ayushtkn I'm not 100% sure what's going on there. I am working on upgrading jline as a separate task HIVE-25495 (#2617) and I'm hitting a similar issue there event though I thought I synchronized the versions between Hive and Hadoop, so I need to look into it more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648596) Time Spent: 5h 53m (was: 5h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 5h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=648096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-648096 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 08/Sep/21 17:10 Start Date: 08/Sep/21 17:10 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #1742: URL: https://github.com/apache/hive/pull/1742#issuecomment-915417026 Seems the tests are failing with `` java.lang.NoSuchMethodError: org.jline.reader.impl.completer.StringsCompleter.([Lorg/jline/reader/Candidate;)V `` Should be fixable in the Hive Code itself? If there is something required in the Hadoop Code, we can get that in now, 3.3.2 release is being planned out -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 648096) Time Spent: 5h 43m (was: 5.55h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 5h 43m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=628565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628565
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 27/Jul/21 15:33
Start Date: 27/Jul/21 15:33
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r677540831
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WarehouseInstance.java
##
@@ -394,16 +393,13 @@ WarehouseInstance verifyResults(String[] data) throws
IOException {
}
WarehouseInstance verifyFailure(String[] data) throws IOException {
-List results = getOutput();
-logger.info("Expecting {}", StringUtils.join(data, ","));
-logger.info("Got {}", results);
-boolean dataMatched = (data.length == results.size());
-if (dataMatched) {
- for (int i = 0; i < data.length; i++) {
-dataMatched &=
data[i].toLowerCase().equals(results.get(i).toLowerCase());
- }
-}
-assertFalse(dataMatched);
+final List expectedResults =
+Arrays.asList(data).stream().map(r ->
r.toLowerCase()).collect(Collectors.toList());
+final List actualResults = getOutput().stream().map(r ->
r.toLowerCase()).collect(Collectors.toList());
+
+assertTrue("Data " + expectedResults + " should not be present in " +
actualResults,
+Collections.disjoint(expectedResults, actualResults));
+
Review comment:
old and new code seem to be doing different things
old is:
```
e[0] != r[0] || ... || e[n] != r[n]
```
new block is a condition which is checking a set operation between the
elements...
why is this change necessary - how this is connected to a hadoop upgrade?
##
File path: ql/src/test/results/clientpositive/llap/check_constraint.q.out
##
@@ -2415,14 +2415,14 @@ STAGE PLANS:
outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
Statistics: Num rows: 1 Data size: 409 Basic stats: COMPLETE
Column stats: NONE
Select Operator
- expressions: _col1 (type: string), _col0 (type: int), _col4
(type: string), _col5 (type: struct),
_col2 (type: string), _col3 (type: int)
+ expressions: _col0 (type: int), _col4 (type: string), _col3
(type: int), _col1 (type: string), _col5 (type:
struct), _col2 (type: string)
Review comment:
the join operand order seem to have changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 628565)
Time Spent: 5.55h (was: 5h 23m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5.55h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626809=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626809 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 17:55 Start Date: 22/Jul/21 17:55 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r675041145 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ## @@ -367,6 +368,7 @@ public static void startMetaStore(int port, HadoopThriftAuthBridge bridge, boolean tcpKeepAlive = MetastoreConf.getBoolVar(conf, ConfVars.TCP_KEEP_ALIVE); boolean useCompactProtocol = MetastoreConf.getBoolVar(conf, ConfVars.USE_THRIFT_COMPACT_PROTOCOL); boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); Review comment: So ya, this was done as a separate thing buried in the Hive code. Moving it here makes it much more explicit and less hidden. Before Hadoop 3.3, it could easily be detected if a call to `refreshSuperUserGroupsConfiguration` had already been performed because there was a corresponding getter that would return a `null` value if it had not. Well, in 3.3 that went away and instead of returning null, you get some sort of default value. So now one can't lazily refresh these configurations, if they haven't already been refreshed, so it's better to just refresh them explicitly here as part of the servers initialization and be done with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626809) Time Spent: 5h 23m (was: 5h 13m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 5h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626807=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626807 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 17:54 Start Date: 22/Jul/21 17:54 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r675041145 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ## @@ -367,6 +368,7 @@ public static void startMetaStore(int port, HadoopThriftAuthBridge bridge, boolean tcpKeepAlive = MetastoreConf.getBoolVar(conf, ConfVars.TCP_KEEP_ALIVE); boolean useCompactProtocol = MetastoreConf.getBoolVar(conf, ConfVars.USE_THRIFT_COMPACT_PROTOCOL); boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); Review comment: So ya, this was done as a separate thing buried in the Hive code. This makes it much more explicit and less hidden. Before Hadoop 3.3, it could easily be detected if a call to `refreshSuperUserGroupsConfiguration` had already been performed because there was a corresponding getter that would return a `null` value if it had not. Well, in 3.3 that went away and instead of returning null, you get some sort of default value. So now one can't lazily refresh these configurations, if they haven't already been, it's better to just refresh them explicitly here and be done with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626807) Time Spent: 5.05h (was: 4h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 5.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626808=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626808 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 17:54 Start Date: 22/Jul/21 17:54 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r675041145 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ## @@ -367,6 +368,7 @@ public static void startMetaStore(int port, HadoopThriftAuthBridge bridge, boolean tcpKeepAlive = MetastoreConf.getBoolVar(conf, ConfVars.TCP_KEEP_ALIVE); boolean useCompactProtocol = MetastoreConf.getBoolVar(conf, ConfVars.USE_THRIFT_COMPACT_PROTOCOL); boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); Review comment: So ya, this was done as a separate thing buried in the Hive code. Moving it here makes it much more explicit and less hidden. Before Hadoop 3.3, it could easily be detected if a call to `refreshSuperUserGroupsConfiguration` had already been performed because there was a corresponding getter that would return a `null` value if it had not. Well, in 3.3 that went away and instead of returning null, you get some sort of default value. So now one can't lazily refresh these configurations, if they haven't already been, it's better to just refresh them explicitly here and be done with it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626808) Time Spent: 5h 13m (was: 5.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 5h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626806=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626806 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 17:51 Start Date: 22/Jul/21 17:51 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r675037187 ## File path: standalone-metastore/pom.xml ## @@ -79,8 +79,8 @@ 0.1.2 3.1.0 -19.0 -3.1.0 +27.0-jre +3.2.1 Review comment: Wow, great catch. Nooo! Ugh. I hope it doesn't break anything. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626806) Time Spent: 4h 53m (was: 4h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 4h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626731=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626731
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 15:07
Start Date: 22/Jul/21 15:07
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674896581
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
+ return new ZeroRowsInputFormat().getRecordReader(split.getInputSplit(),
jobConf, reporter);
Review comment:
Hey.
So, in my experimentation, this is the least-bad option. I did this to
preserve the previous behavior. The Hive code is not setup to handle this
error condition. As thing currently stand in `master`, if the calling Thread
was interrupted, the thread would finish fetching the rows regardless and then
just later ignore them (throw them away). The calling code does not handle
'null' return value and it does not handle this Exception. As currently
implemented in Hive `master`, if it gets an exception it simply exits execution
with an Error message, without implementing a lot more code, there is no way to
ignore/skip this one specific error type. So, the cleanest thing to do is to
return `ZeroRows` since it's going to be thrown away later anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626731)
Time Spent: 4h 43m (was: 4.55h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 43m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626730=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626730
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 15:00
Start Date: 22/Jul/21 15:00
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674888734
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java
##
@@ -4117,28 +4118,33 @@ public void testAuthForNotificationAPIs() throws
Exception {
createDB(dbName, driver);
NotificationEventResponse rsp =
metaStoreClient.getNextNotification(firstEventId, 0, null);
assertEquals(1, rsp.getEventsSize());
+
// Test various scenarios
-// Remove the proxy privilege and the auth should fail (in reality the
proxy setting should not be changed on the fly)
-hconf.unset(proxySettingName);
-// Need to explicitly update ProxyUsers
-ProxyUsers.refreshSuperUserGroupsConfiguration(hconf);
-// Verify if the auth should fail
-Exception ex = null;
+// Remove the proxy privilege by reseting proxy configuration to default
value.
+// The auth should fail (in reality the proxy setting should not be
changed on the fly)
+// Pretty hacky: Affects both instances of HMS
+ProxyUsers.refreshSuperUserGroupsConfiguration();
+
try {
rsp = metaStoreClient.getNextNotification(firstEventId, 0, null);
+ Assert.fail("Get Next Nofitication should have failed due to no proxy
auth");
} catch (TException e) {
- ex = e;
Review comment:
The idea here is that it SHOULD throw an Exception. If it does not
throw an Exception from `getNextNofitication` then it will hit the
`Assert.fail`. I can add a comment to clarify that this is the expected
behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626730)
Time Spent: 4.55h (was: 4h 23m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4.55h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626729=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626729
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 14:58
Start Date: 22/Jul/21 14:58
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674886306
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/BaseReplicationAcrossInstances.java
##
@@ -55,14 +56,15 @@ static void internalBeforeClassSetup(Map
overrides, Class clazz)
throws Exception {
conf = new HiveConf(clazz);
conf.set("dfs.client.use.datanode.hostname", "true");
-conf.set("hadoop.proxyuser." + Utils.getUGI().getShortUserName() +
".hosts", "*");
Review comment:
Hey @abstractdog, thanks for the review.
Take a look at my notes here:
https://issues.apache.org/jira/browse/HIVE-24484?focusedCommentId=17369708=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17369708
tldr; These unit tests are launching two HMS within the same JVM (same
class-loader) and therefore they are able to modify each other's state where it
stored in static variables. This testing cannot be done any more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626729)
Time Spent: 4h 23m (was: 4h 13m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 23m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626727=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626727 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 14:54 Start Date: 22/Jul/21 14:54 Worklog Time Spent: 10m Work Description: abstractdog commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r674880301 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ## @@ -367,6 +368,7 @@ public static void startMetaStore(int port, HadoopThriftAuthBridge bridge, boolean tcpKeepAlive = MetastoreConf.getBoolVar(conf, ConfVars.TCP_KEEP_ALIVE); boolean useCompactProtocol = MetastoreConf.getBoolVar(conf, ConfVars.USE_THRIFT_COMPACT_PROTOCOL); boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); Review comment: is done somewhere else implicitly before hadoop 3.3? ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ## @@ -367,6 +368,7 @@ public static void startMetaStore(int port, HadoopThriftAuthBridge bridge, boolean tcpKeepAlive = MetastoreConf.getBoolVar(conf, ConfVars.TCP_KEEP_ALIVE); boolean useCompactProtocol = MetastoreConf.getBoolVar(conf, ConfVars.USE_THRIFT_COMPACT_PROTOCOL); boolean useSSL = MetastoreConf.getBoolVar(conf, ConfVars.USE_SSL); +ProxyUsers.refreshSuperUserGroupsConfiguration(conf); Review comment: is this done somewhere else implicitly before hadoop 3.3? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626727) Time Spent: 4.05h (was: 3h 53m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 4.05h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626728=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626728 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 14:54 Start Date: 22/Jul/21 14:54 Worklog Time Spent: 10m Work Description: abstractdog commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r674881535 ## File path: standalone-metastore/pom.xml ## @@ -79,8 +79,8 @@ 0.1.2 3.1.0 -19.0 -3.1.0 +27.0-jre +3.2.1 Review comment: I think we're targeting 3.3.1 here too, right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626728) Time Spent: 4h 13m (was: 4.05h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 4h 13m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626726 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 22/Jul/21 14:53 Start Date: 22/Jul/21 14:53 Worklog Time Spent: 10m Work Description: abstractdog commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r674879041 ## File path: spark-client/pom.xml ## @@ -159,45 +159,10 @@ - Review comment: happy to see that we can get rid of these maven magics! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626726) Time Spent: 3h 53m (was: 3h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 3h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626724=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626724
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 14:50
Start Date: 22/Jul/21 14:50
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674876607
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/security/authorization/plugin/metastore/HiveMetaStoreAuthorizer.java
##
@@ -483,7 +484,8 @@ HiveAuthorizer createHiveMetaStoreAuthorizer() throws
Exception {
boolean isSuperUser(String userName) {
Configuration conf = getConf();
StringipAddress = HMSHandler.getIPAddress();
-return (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(userName,
conf, ipAddress));
+ProxyUsers.refreshSuperUserGroupsConfiguration(conf);
+return (MetaStoreServerUtils.checkUserHasHostProxyPrivileges(userName,
ipAddress));
Review comment:
nit: extra bracket is not needed I guess
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626724)
Time Spent: 3h 43m (was: 3.55h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 43m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626723=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626723
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 14:50
Start Date: 22/Jul/21 14:50
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674875763
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/RecordReaderWrapper.java
##
@@ -69,7 +70,14 @@ static RecordReader create(InputFormat inputFormat,
HiveInputFormat.HiveInputSpl
JobConf jobConf, Reporter reporter) throws IOException {
int headerCount = Utilities.getHeaderCount(tableDesc);
int footerCount = Utilities.getFooterCount(tableDesc, jobConf);
-RecordReader innerReader =
inputFormat.getRecordReader(split.getInputSplit(), jobConf, reporter);
+
+RecordReader innerReader = null;
+try {
+ innerReader = inputFormat.getRecordReader(split.getInputSplit(), jobConf,
reporter);
+} catch (InterruptedIOException iioe) {
+ // If reading from the underlying record reader is interrupted, return a
no-op record reader
+ return new ZeroRowsInputFormat().getRecordReader(split.getInputSplit(),
jobConf, reporter);
Review comment:
why is it better to return with no-op record reader instead of letting
this codepath fail and handle the exception somewhere else? doesn't this mask
issues?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626723)
Time Spent: 3.55h (was: 3h 23m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.55h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626721=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626721
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 14:47
Start Date: 22/Jul/21 14:47
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674871945
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java
##
@@ -4117,28 +4118,33 @@ public void testAuthForNotificationAPIs() throws
Exception {
createDB(dbName, driver);
NotificationEventResponse rsp =
metaStoreClient.getNextNotification(firstEventId, 0, null);
assertEquals(1, rsp.getEventsSize());
+
// Test various scenarios
-// Remove the proxy privilege and the auth should fail (in reality the
proxy setting should not be changed on the fly)
-hconf.unset(proxySettingName);
-// Need to explicitly update ProxyUsers
-ProxyUsers.refreshSuperUserGroupsConfiguration(hconf);
-// Verify if the auth should fail
-Exception ex = null;
+// Remove the proxy privilege by reseting proxy configuration to default
value.
+// The auth should fail (in reality the proxy setting should not be
changed on the fly)
+// Pretty hacky: Affects both instances of HMS
+ProxyUsers.refreshSuperUserGroupsConfiguration();
+
try {
rsp = metaStoreClient.getNextNotification(firstEventId, 0, null);
+ Assert.fail("Get Next Nofitication should have failed due to no proxy
auth");
} catch (TException e) {
- ex = e;
Review comment:
I have no idea how can we hit this catch, but having it empty is always
a red sign, have you checked if at least a log.debug is useful here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626721)
Time Spent: 3h 23m (was: 3h 13m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 23m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=626719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626719
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 22/Jul/21 14:45
Start Date: 22/Jul/21 14:45
Worklog Time Spent: 10m
Work Description: abstractdog commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r674869601
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/BaseReplicationAcrossInstances.java
##
@@ -55,14 +56,15 @@ static void internalBeforeClassSetup(Map
overrides, Class clazz)
throws Exception {
conf = new HiveConf(clazz);
conf.set("dfs.client.use.datanode.hostname", "true");
-conf.set("hadoop.proxyuser." + Utils.getUGI().getShortUserName() +
".hosts", "*");
Review comment:
this is for impersonation according to
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Superusers.html,
don't we want to test this scenario anymore?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 626719)
Time Spent: 3h 13m (was: 3.05h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 13m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=619976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-619976
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 07/Jul/21 13:31
Start Date: 07/Jul/21 13:31
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r665373095
##
File path:
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java
##
@@ -137,6 +137,8 @@
// This starts the reader in the background.
rr.start();
return result;
+} catch (IOException ioe) {
Review comment:
Hey @pgaref,
Ya, this is required. Based on the `InvalidInputException` (which is a
subclass of `IOException`) changes in HDFS, this code is require to pass the
`InvalidInputException` up to the caller directly, otherwise, in the
`Exception` block, it gets wrapped in yet another `IOException` and that caller
is no longer able to detect the `InvalidInputException`.
I hope that makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 619976)
Time Spent: 3.05h (was: 2h 53m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.05h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=619869=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-619869 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 07/Jul/21 09:44 Start Date: 07/Jul/21 09:44 Worklog Time Spent: 10m Work Description: pgaref commented on a change in pull request #1742: URL: https://github.com/apache/hive/pull/1742#discussion_r665214848 ## File path: ql/src/test/results/clientpositive/llap/check_constraint.q.out ## @@ -2415,14 +2415,14 @@ STAGE PLANS: outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5 Statistics: Num rows: 1 Data size: 409 Basic stats: COMPLETE Column stats: NONE Select Operator - expressions: _col1 (type: string), _col0 (type: int), _col4 (type: string), _col5 (type: struct), _col2 (type: string), _col3 (type: int) + expressions: _col0 (type: int), _col4 (type: string), _col3 (type: int), _col1 (type: string), _col5 (type: struct), _col2 (type: string) Review comment: Are these q.out changes expected? What is the root cause? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 619869) Time Spent: 2h 53m (was: 2h 43m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 53m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=619868=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-619868
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 07/Jul/21 09:42
Start Date: 07/Jul/21 09:42
Worklog Time Spent: 10m
Work Description: pgaref commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r665213804
##
File path:
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapInputFormat.java
##
@@ -137,6 +137,8 @@
// This starts the reader in the background.
rr.start();
return result;
+} catch (IOException ioe) {
Review comment:
is this needed? Exception should catch everything right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 619868)
Time Spent: 2h 43m (was: 2.55h)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 43m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=619867=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-619867
]
ASF GitHub Bot logged work on HIVE-24484:
-
Author: ASF GitHub Bot
Created on: 07/Jul/21 09:40
Start Date: 07/Jul/21 09:40
Worklog Time Spent: 10m
Work Description: pgaref commented on a change in pull request #1742:
URL: https://github.com/apache/hive/pull/1742#discussion_r665212475
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java
##
@@ -119,16 +120,15 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws Throwable {
.run("insert into table encrypted_table values (2,'value2')")
.dump(primaryDbName, dumpWithClause);
-replica
-.run("repl load " + primaryDbName + " into " + replicatedDbName
-+ " with('hive.repl.add.raw.reserved.namespace'='true', "
-+ "'hive.repl.replica.external.table.base.dir'='" +
replica.externalTableWarehouseRoot + "', "
-+ "'distcp.options.pugpbx'='',
'distcp.options.skipcrccheck'='')")
-.run("use " + replicatedDbName)
-.run("repl status " + replicatedDbName)
-.verifyResult(tuple.lastReplicationId)
-.run("select value from encrypted_table")
-.verifyFailure(new String[] { "value1", "value2" });
+try {
+ replica.run("repl load " + primaryDbName + " into " + replicatedDbName
+ + " with('hive.repl.add.raw.reserved.namespace'='true', " +
"'hive.repl.replica.external.table.base.dir'='"
+ + replica.externalTableWarehouseRoot + "', "
+ + "'distcp.options.pugpbx'='', 'distcp.options.skipcrccheck'='')");
+ Assert.fail("Test should have thrown an exception because
cross-encryption-zone is not allowed for RAW");
+} catch (IOException ioe) {
+ // ignore
Review comment:
Hey @belugabehr just read the detailed comment on the JIRA about this
but I believe we should add some explanation here as well for clarity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 619867)
Time Spent: 2.55h (was: 2h 23m)
> Upgrade Hadoop to 3.3.1
> ---
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.55h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=616656=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-616656 ] David Mollitor logged work on HIVE-24484: - Author: David Mollitor Created on: 29/Jun/21 18:21 Start Date: 29/Jun/21 18:21 Worklog Time Spent: 0.05h Issue Time Tracking --- Worklog Id: (was: 616656) Time Spent: 2h 23m (was: 2h 20m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 23m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=614531=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614531 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 24/Jun/21 14:11 Start Date: 24/Jun/21 14:11 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1742: URL: https://github.com/apache/hive/pull/1742 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 614531) Time Spent: 2h 20m (was: 2h 10m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=614530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614530 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 24/Jun/21 14:08 Start Date: 24/Jun/21 14:08 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #1742: URL: https://github.com/apache/hive/pull/1742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 614530) Time Spent: 2h 10m (was: 2h) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=614216=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614216 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 23/Jun/21 20:35 Start Date: 23/Jun/21 20:35 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1742: URL: https://github.com/apache/hive/pull/1742 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 614216) Time Spent: 2h (was: 1h 50m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24484) Upgrade Hadoop to 3.3.1
[ https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=614211=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-614211 ] ASF GitHub Bot logged work on HIVE-24484: - Author: ASF GitHub Bot Created on: 23/Jun/21 20:31 Start Date: 23/Jun/21 20:31 Worklog Time Spent: 10m Work Description: belugabehr closed pull request #1742: URL: https://github.com/apache/hive/pull/1742 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 614211) Time Spent: 1h 50m (was: 1h 40m) > Upgrade Hadoop to 3.3.1 > --- > > Key: HIVE-24484 > URL: https://issues.apache.org/jira/browse/HIVE-24484 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
