[ https://issues.apache.org/jira/browse/HADOOP-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16371568#comment-16371568 ]
genericqa commented on HADOOP-6852: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 7 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 24s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m 39s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 17m 49s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-client-modules/hadoop-client-minicluster {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 8s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 36s{color} | {color:green} root: The patch generated 0 new + 69 unchanged - 11 fixed = 69 total (was 80) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 44s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-client-modules/hadoop-client-minicluster {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 8s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 55s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}139m 40s{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 39s{color} | {color:green} hadoop-client-minicluster in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 45s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}267m 8s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | HADOOP-6852 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12911369/HADOOP-6852.04.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux ca835c10b45f 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 004b722 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HADOOP-Build/14176/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt | | Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/14176/testReport/ | | Max. process+thread count | 1640 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-client-modules/hadoop-client-minicluster U: . | | Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/14176/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > apparent bug in concatenated-bzip2 support (decoding) > ----------------------------------------------------- > > Key: HADOOP-6852 > URL: https://issues.apache.org/jira/browse/HADOOP-6852 > Project: Hadoop Common > Issue Type: Bug > Components: io > Affects Versions: 0.22.0 > Environment: Linux x86_64 running 32-bit Hadoop, JDK 1.6.0_15 > Reporter: Greg Roelofs > Assignee: Zsolt Venczel > Priority: Major > Attachments: HADOOP-6852.01.patch, HADOOP-6852.02.patch, > HADOOP-6852.03.patch, HADOOP-6852.04.patch > > > The following simplified code (manually picked out of testMoreBzip2() in > https://issues.apache.org/jira/secure/attachment/12448272/HADOOP-6835.v4.trunk-hadoop-mapreduce.patch) > triggers a "java.io.IOException: bad block header" in > org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.initBlock( > CBZip2InputStream.java:527): > {noformat} > JobConf jobConf = new JobConf(defaultConf); > CompressionCodec bzip2 = new BZip2Codec(); > ReflectionUtils.setConf(bzip2, jobConf); > localFs.delete(workDir, true); > // copy multiple-member test file to HDFS > String fn2 = "testCompressThenConcat.txt" + bzip2.getDefaultExtension(); > Path fnLocal2 = new > Path(System.getProperty("test.concat.data","/tmp"),fn2); > Path fnHDFS2 = new Path(workDir, fn2); > localFs.copyFromLocalFile(fnLocal2, fnHDFS2); > FileInputFormat.setInputPaths(jobConf, workDir); > final FileInputStream in2 = new FileInputStream(fnLocal2.toString()); > CompressionInputStream cin2 = bzip2.createInputStream(in2); > LineReader in = new LineReader(cin2); > Text out = new Text(); > int numBytes, totalBytes=0, lineNum=0; > while ((numBytes = in.readLine(out)) > 0) { > ++lineNum; > totalBytes += numBytes; > } > in.close(); > {noformat} > The specified file is also included in the H-6835 patch linked above, and > some additional debug output is included in the commented-out test loop > above. (Only in the linked, "v4" version of the patch, however--I'm about to > remove the debug stuff for checkin.) > It's possible I've done something completely boneheaded here, but the file, > at least, checks out in a subsequent set of subtests and with stock bzip2 > itself. Only the code above is problematic; it reads through the first > concatenated chunk (17 lines of text) just fine but chokes on the header of > the second one. Altogether, the test file contains 84 lines of text and 4 > concatenated bzip2 files. > (It's possible this is a mapreduce issue rather than common, but note that > the identical gzip test works fine. Possibly it's related to the > stream-vs-decompressor dichotomy, though; intentionally not supported?) -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org