[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17367301#comment-17367301
]
Steve Loughran commented on HIVE-17133:
---
Is this ready to go in? even without a new test?
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17133.1.patch
>
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003680#comment-17003680
]
Hive QA commented on HIVE-17133:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12989496/HIVE-17133.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 17786 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.cli.TestHiveCli.testCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testCommentStripping (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testDatabaseOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testErrOutput (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testHelp (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInValidCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidDatabaseOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidOptions (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testInvalidOptions2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSetHeaderValue (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSetPromptValue (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd3 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSourceCmd4 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSqlFromCmd (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testSqlFromCmdWithDBName (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB1 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB2 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseCurrentDB3 (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testUseInvalidDB (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testVariables (batchId=206)
org.apache.hive.beeline.cli.TestHiveCli.testVariablesForSource (batchId=206)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20021/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20021/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20021/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12989496 - PreCommit-HIVE-Build
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17133.1.patch
>
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003662#comment-17003662
]
Hive QA commented on HIVE-17133:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20021/dev-support/hive-personality.sh
|
| git revision | master / 4dcbfb3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20021/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17133.1.patch
>
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17003635#comment-17003635
]
Rui Li commented on HIVE-17133:
---
While HADOOP-14683 has been fixed, it seems we still have to fix on our side.
[~sershe] [~xuefuz] Would you mind have a look at the patch?
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
> Attachments: HIVE-17133.1.patch
>
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110072#comment-16110072
]
Sergey Shelukhin commented on HIVE-17133:
-
The solution would be to upgrade to 2.8.2 once that is released. We'd always
build with the old signature and thus support old and new versions but not the
ones in the middle (need to verify that we do indeed refer to a method with old
signature when building).
Thus, Hive will not support Hadoop 2.8.0 and 2.8.1.
The problem is if we release something referring to the new signature soon (or
have already). We might need 2.2.1 and 2.3.0 if these were built against Hadoop
2.8.0/1, as far as I understand cc [~owen.omalley] [~pxiong]
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099305#comment-16099305
]
Sergey Shelukhin commented on HIVE-17133:
-
Filed HADOOP-14683. Let's see what the response is, and go from there.
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099228#comment-16099228
]
Sergey Shelukhin commented on HIVE-17133:
-
I think we do... we should still be able run on it.
We'd probably need a shim. Or we can ask to re-add the method in 2.8.1 (or
whatever) and not support 2.8.0.
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17133) NoSuchMethodError in Hadoop FileStatus.compareTo
[
https://issues.apache.org/jira/browse/HIVE-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094475#comment-16094475
]
Rui Li commented on HIVE-17133:
---
I'm aware Hive is on Hadoop 2.8.0 now. Wondering whether we still support older
versions.
> NoSuchMethodError in Hadoop FileStatus.compareTo
>
>
> Key: HIVE-17133
> URL: https://issues.apache.org/jira/browse/HIVE-17133
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>
> The stack trace is:
> {noformat}
> Caused by: java.lang.NoSuchMethodError:
> org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
> at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
> at java.util.TimSort.sort(TimSort.java:234)
> at java.util.Arrays.sort(Arrays.java:1512)
> at java.util.ArrayList.sort(ArrayList.java:1454)
> at java.util.Collections.sort(Collections.java:175)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)
> {noformat}
> I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop
> 2.7.2 is:
> https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
> In Hadoop 2.8.0 it becomes:
> https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
> I think that breaks binary compatibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
