[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011352#comment-15011352 ] Tsuyoshi Ozawa commented on HADOOP-8419: Backported this to branch-2 for targeting 2.8.0. > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011496#comment-15011496 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #678 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/678/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011368#comment-15011368 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-trunk-Commit #8819 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8819/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011580#comment-15011580 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Mapreduce-trunk #2620 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2620/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011688#comment-15011688 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #691 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/691/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011594#comment-15011594 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Yarn-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1416/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011928#comment-15011928 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #615 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/615/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012974#comment-15012974 ] Yu Li commented on HADOOP-8419: --- Thanks for checking and backporting this old patch [~ozawa]. :-) > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012006#comment-15012006 ] Hudson commented on HADOOP-8419: FAILURE: Integrated in Hadoop-Hdfs-trunk #2553 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2553/]) Moved HADOOP-8419 in CHANGES.txt from 3.0.0 to 2.8.0. (ozawa: rev 169921e8abc2efef1bda69ad0d4b530488c639d8) * hadoop-common-project/hadoop-common/CHANGES.txt > GzipCodec NPE upon reset with IBM JDK > - > > Key: HADOOP-8419 > URL: https://issues.apache.org/jira/browse/HADOOP-8419 > Project: Hadoop Common > Issue Type: Bug > Components: io >Affects Versions: 1.0.3 >Reporter: Luke Lu >Assignee: Yu Li > Labels: gzip, ibm-jdk > Fix For: 1.1.2, 2.0.5-alpha, 2.8.0 > > Attachments: HADOOP-8419-branch-1.patch, > HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk-v2.patch, > HADOOP-8419-trunk.patch > > > The GzipCodec will NPE upon reset after finish when the native zlib codec is > not loaded. When the native zlib is loaded the codec creates a > CompressorOutputStream that doesn't have the problem, otherwise, the > GZipCodec uses GZIPOutputStream which is extended to provide the resetState > method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, > GZIPOutputStream#finish will release the underlying deflater, which causes > NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK > doesn't have this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13691041#comment-13691041 ] Eric Yang commented on HADOOP-8419: --- Yes this is committed to trunk. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Fix For: 3.0.0, 1.1.2, 2.0.5-alpha Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13688712#comment-13688712 ] Suresh Srinivas commented on HADOOP-8419: - [~eyang] Is this fix committed to trunk. If so can you please mark the fixed version as such. If not, why is this in BUG FIXES section in CHANGES.txt in trunk. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Fix For: 1.1.2 Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579875#comment-13579875 ] Eric Yang commented on HADOOP-8419: --- Matt, the failures is not related to this patch in HDFS trunk failure case. HDFS test case is failing due to HADOOP-9067. I also reverted this patch completely and reran the test case to be sure that this patch was not trigger any strange hidden behavior. Amir, the test case has been tested on power platform on ppc Jenkins server. We did not see the regression issue that you mentioned. Could you provide IBM JDK version number that has this regression fixed in zLinux? This will help us to make sure the logic is only target specific version ranges of IBM jdk that have this bug. Since you are using the same ppc Jenkins server to do the test, I don't think you will see any regression. I am inclined to close this issue baring no objections. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Fix For: 1.1.2 Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579222#comment-13579222 ] Amir Sanjar commented on HADOOP-8419: - Matt, please do not add this patch to the next release.. It causes regression on IBM S390x running zLinux. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Fix For: 1.1.2 Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13579372#comment-13579372 ] Yu Li commented on HADOOP-8419: --- Hi Matt, from the Husdson auto-comment I cannot locate the reason why integration with hdfs-trunk and mapred-trunk failed, can you give me some hints to resolve the issue? Thanks. Hi Amir, can you tell me more about the regression with detailed error/exception message? I think we should improve the patch rather than leave the known issue open, thanks. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Fix For: 1.1.2 Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13564000#comment-13564000 ] Matt Foley commented on HADOOP-8419: I think When Eric committed it, he intended to mark fixVersion to be 1.1.2 and 3.0.0, not just targetVersions. I'm marking it fixed in 1.1.2. However, I'm concerned about the integration failure noted above for Hdfs-trunk and Mapreduce-trunk. So I'm not yet marking it fixed in 3.0.0. Can you please check into this? Thanks. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551034#comment-13551034 ] Hudson commented on HADOOP-8419: Integrated in Hadoop-Yarn-trunk #93 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/93/]) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431740) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431739) Result = SUCCESS eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431740 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressionStreamReuse.java eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431739 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/GzipCodec.java GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551096#comment-13551096 ] Hudson commented on HADOOP-8419: Integrated in Hadoop-Hdfs-trunk #1282 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1282/]) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431740) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431739) Result = FAILURE eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431740 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressionStreamReuse.java eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431739 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/GzipCodec.java GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551106#comment-13551106 ] Hudson commented on HADOOP-8419: Integrated in Hadoop-Mapreduce-trunk #1310 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1310/]) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431740) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431739) Result = FAILURE eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431740 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressionStreamReuse.java eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431739 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/GzipCodec.java GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550600#comment-13550600 ] Eric Yang commented on HADOOP-8419: --- +1, I just committed this, thank you Yu. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13550606#comment-13550606 ] Hudson commented on HADOOP-8419: Integrated in Hadoop-trunk-Commit #3215 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3215/]) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431740) HADOOP-8419. Fixed GzipCode NPE reset for IBM JDK. (Yu Li via eyang) (Revision 1431739) Result = SUCCESS eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431740 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCompressionStreamReuse.java eyang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1431739 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/GzipCodec.java GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch, HADOOP-8419-trunk-v2.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[
https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539235#comment-13539235
]
Yu Li commented on HADOOP-8419:
---
Thanks Luke for the comments, I have updated and resubmit the patch.
For the updated patch, all test-commit UT passed in my env, and below is the
result of test-patch:
{color:green}+1 overall{color}.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
GzipCodec NPE upon reset with IBM JDK
-
Key: HADOOP-8419
URL: https://issues.apache.org/jira/browse/HADOOP-8419
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 1.0.3
Reporter: Luke Lu
Assignee: Yu Li
Labels: gzip, ibm-jdk
Attachments: HADOOP-8419-branch-1.patch,
HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch,
HADOOP-8419-trunk-v2.patch
The GzipCodec will NPE upon reset after finish when the native zlib codec is
not loaded. When the native zlib is loaded the codec creates a
CompressorOutputStream that doesn't have the problem, otherwise, the
GZipCodec uses GZIPOutputStream which is extended to provide the resetState
method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10,
GZIPOutputStream#finish will release the underlying deflater, which causes
NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK
doesn't have this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[
https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539240#comment-13539240
]
Hadoop QA commented on HADOOP-8419:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12562314/HADOOP-8419-trunk-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1924//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1924//console
This message is automatically generated.
GzipCodec NPE upon reset with IBM JDK
-
Key: HADOOP-8419
URL: https://issues.apache.org/jira/browse/HADOOP-8419
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 1.0.3
Reporter: Luke Lu
Assignee: Yu Li
Labels: gzip, ibm-jdk
Attachments: HADOOP-8419-branch-1.patch,
HADOOP-8419-branch1-v2.patch, HADOOP-8419-trunk.patch,
HADOOP-8419-trunk-v2.patch
The GzipCodec will NPE upon reset after finish when the native zlib codec is
not loaded. When the native zlib is loaded the codec creates a
CompressorOutputStream that doesn't have the problem, otherwise, the
GZipCodec uses GZIPOutputStream which is extended to provide the resetState
method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10,
GZIPOutputStream#finish will release the underlying deflater, which causes
NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK
doesn't have this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13534117#comment-13534117 ] Luke Lu commented on HADOOP-8419: - Since this is restricted to some IBM JDK 6 releases. We should restrict override to these release as well, a la changes in HBASE-7145. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[
https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493960#comment-13493960
]
Yu Li commented on HADOOP-8419:
---
The result of test-patch:
{color:green}+1 overall{color}.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
GzipCodec NPE upon reset with IBM JDK
-
Key: HADOOP-8419
URL: https://issues.apache.org/jira/browse/HADOOP-8419
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 1.0.3
Reporter: Luke Lu
Assignee: Yu Li
Labels: gzip, ibm-jdk
Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch
The GzipCodec will NPE upon reset after finish when the native zlib codec is
not loaded. When the native zlib is loaded the codec creates a
CompressorOutputStream that doesn't have the problem, otherwise, the
GZipCodec uses GZIPOutputStream which is extended to provide the resetState
method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10,
GZIPOutputStream#finish will release the underlying deflater, which causes
NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK
doesn't have this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493965#comment-13493965 ] Yu Li commented on HADOOP-8419: --- Test result on branch-1: Both with and w/o my patch, below UT cases failed, not sure whether it's env issue, but from error message it should be irrelavant with compression: [junit] Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.067 sec [junit] Running org.apache.hadoop.hdfs.TestRestartDFS [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 16.016 sec [junit] Running org.apache.hadoop.hdfs.TestSafeMode [junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 64.601 sec [junit] Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint [junit] Tests run: 3, Failures: 0, Errors: 3, Time elapsed: 41.901 sec [junit] Running org.apache.hadoop.hdfs.server.namenode.TestStartup [junit] Tests run: 3, Failures: 2, Errors: 0, Time elapsed: 44.583 sec All cases with error has error messages like: === Edit log corruption detected: corruption length = 9748 toleration length = 0; the corruption is intolerable. java.io.IOException: Edit log corruption detected: corruption length = 9748 toleration length = 0; the corruption is intolerable. at org.apache.hadoop.hdfs.server.namenode.FSEditLog.checkEndOfLog(FSEditLog.java:608) === The case with failure has error message like: === java.io.IOException: Failed to parse edit log (/home/biadmin/hadoop/build/test/data/dfs/chkpt/current/edits) at position 555, edit log length is 690, opcode=0, isTolerationEnabled=false, Rec ent opcode offsets=[65 124 244 388] at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:84) === GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493966#comment-13493966 ] Yu Li commented on HADOOP-8419: --- Test result on trunk: Both with and w/o my patch, below UT case failed, but from error message it should be irrelavant with compression: === Tests in error: testRDNS(org.apache.hadoop.net.TestDNS): DNS server failure [response code 2] Tests run: 1784, Failures: 0, Errors: 1, Skipped: 18 [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main SUCCESS [1.702s] [INFO] Apache Hadoop Project POM . SUCCESS [3.812s] [INFO] Apache Hadoop Annotations . SUCCESS [1.312s] [INFO] Apache Hadoop Project Dist POM SUCCESS [0.245s] [INFO] Apache Hadoop Assemblies .. SUCCESS [0.335s] [INFO] Apache Hadoop Auth SUCCESS [6.754s] [INFO] Apache Hadoop Auth Examples ... SUCCESS [0.322s] [INFO] Apache Hadoop Common .. FAILURE [16:42.921s] [INFO] Apache Hadoop Common Project .. SKIPPED === From the UT log we could see below error message: === Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 15.459 sec FAILURE! testRDNS(org.apache.hadoop.net.TestDNS) Time elapsed: 15233 sec ERROR! javax.naming.ServiceUnavailableException: DNS server failure [response code 2]; remaining name '81.122.30.9.in-addr.arpa' at com.sun.jndi.dns.DnsClient.checkResponseCode(DnsClient.java:594) at com.sun.jndi.dns.DnsClient.isMatchResponse(DnsClient.java:553) === GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494133#comment-13494133 ] Luke Lu commented on HADOOP-8419: - Yu, Please click Submit patch to let jenkins to review the trunk patch. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[
https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494558#comment-13494558
]
Hadoop QA commented on HADOOP-8419:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12552831/HADOOP-8419-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1729//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1729//console
This message is automatically generated.
GzipCodec NPE upon reset with IBM JDK
-
Key: HADOOP-8419
URL: https://issues.apache.org/jira/browse/HADOOP-8419
Project: Hadoop Common
Issue Type: Bug
Components: io
Affects Versions: 1.0.3
Reporter: Luke Lu
Assignee: Yu Li
Labels: gzip, ibm-jdk
Attachments: HADOOP-8419-branch-1.patch, HADOOP-8419-trunk.patch
The GzipCodec will NPE upon reset after finish when the native zlib codec is
not loaded. When the native zlib is loaded the codec creates a
CompressorOutputStream that doesn't have the problem, otherwise, the
GZipCodec uses GZIPOutputStream which is extended to provide the resetState
method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10,
GZIPOutputStream#finish will release the underlying deflater, which causes
NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK
doesn't have this issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493374#comment-13493374 ] Luke Lu commented on HADOOP-8419: - The patch lgtm. Thanks Yu! Can you post a patch for trunk as well? That way we can leverage the trunk unit tests. Also please post ant test and test-patch results for branch-1 for Sun JDK as well. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493011#comment-13493011 ] Yu Li commented on HADOOP-8419: --- More details about this issue: In IBM JDK, GZIPOutputStream class calls the deflater's end method as part of GZIPOutputStream.finish(), so deflater's reset can't be called after, while in SUN or Open JDK implementation this end method won't be called. To work-around this issue, we need to override the finish method of coresponding classes which extends GZIPOutputStream, thus won't depend on implementation of differenct JDK. And since the needed writeTrailer, writeInt and writeShort all become private method in JDK6(SUN/IBM/OPEN JDK), we also need to add these 3 methods in the patch. GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8419) GzipCodec NPE upon reset with IBM JDK
[ https://issues.apache.org/jira/browse/HADOOP-8419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493024#comment-13493024 ] Yu Li commented on HADOOP-8419: --- Attached the patch for branch-1 GzipCodec NPE upon reset with IBM JDK - Key: HADOOP-8419 URL: https://issues.apache.org/jira/browse/HADOOP-8419 Project: Hadoop Common Issue Type: Bug Components: io Affects Versions: 1.0.3 Reporter: Luke Lu Assignee: Yu Li Labels: gzip, ibm-jdk Attachments: HADOOP-8419-branch-1.patch The GzipCodec will NPE upon reset after finish when the native zlib codec is not loaded. When the native zlib is loaded the codec creates a CompressorOutputStream that doesn't have the problem, otherwise, the GZipCodec uses GZIPOutputStream which is extended to provide the resetState method. Since IBM JDK 6 SR9 FP2 including the current JDK 6 SR10, GZIPOutputStream#finish will release the underlying deflater, which causes NPE upon reset. This seems to be an IBM JDK quirk as Sun JDK and OpenJDK doesn't have this issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
