[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15711204#comment-15711204 ] Weiwei Yang commented on HDFS-11156: Regarding to the jenkins job result # The findbugs error was from a recent commit HDFS-10930, not from this patch. # The checkstyle issues were not caused by this patch either, 1st issue was the method too long, 2nd the code style was using nested blocks in switch-case block. # The 2 UT failures also not related, I have run them locally and they all succeed. They seemed to be intermittent failures, HDFS-9599 and HDFS-11030 might be related. Hello [~liuml07] Would you please help to review the patch ? Thanks a lot. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch, HDFS-11156.05.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1571#comment-1571 ] Hadoop QA commented on HDFS-11156: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 42s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 21s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 31s{color} | {color:orange} hadoop-hdfs-project: The patch generated 2 new + 237 unchanged - 0 fixed = 239 total (was 237) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}102m 9s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}130m 47s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.server.namenode.TestDecommissioningStatus | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HDFS-11156 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12841220/HDFS-11156.05.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 0d7e5a7f263a 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 1f7613b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | findbugs | https://builds.apache.org/job/PreCommit-HDFS-Build/17725/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html | | checkstyle |
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710484#comment-15710484 ] Andrew Wang commented on HDFS-11156: bq. this API returns BlockLocations[] + FileStatus, it has more information but also means more work for clients to parse and more stuff on network. getFileBlockLocations API should be good in most cases. I'm guessing your client already has code to parse a FileStatus, so is this really more work? This depends on your usecase, but often the client will want a FileStatus too. Anyway, I'm okay with a GETFILEBLOCKLOCATIONS API if that's what fits best. Thanks for working on this [~cheersyang]! > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709000#comment-15709000 ] Weiwei Yang commented on HDFS-11156: Hello [~szetszwo] Thanks for the input, I am happy with that :). Thanks for [~liuml07] and [~andrew.wang] to join the discussion, appreciate your help :). I'll upload a patch tomorrow since most of work has already done in previous patch. I have also removed "incompatible change" tag. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707977#comment-15707977 ] Tsz Wo Nicholas Sze commented on HDFS-11156: The use of underscore in GET_BLOCK_LOCATIONS is to emphasize that it is an internal API. {quote} I agree with Mingliang Liu's suggestion that to #Provide a new OP GETFILEBLOCKLOCATIONS for webhdfs that returns BlockLocation[] to comply with FileSystem API {quote} +1 on adding a new GETFILEBLOCKLOCATIONS for FileSystem#getFileBlockLocations and keep GET_BLOCK_LOCATIONS unchanged. Thanks for working on this, [~cheersyang]. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707367#comment-15707367 ] Weiwei Yang commented on HDFS-11156: Hello [~liuml07], [~andrew.wang] I agree with [~liuml07]'s suggestion that to # Provide a new OP {{GETFILEBLOCKLOCATIONS}} for webhdfs that returns BlockLocation[] to comply with FileSystem API # Add documentation for GETFILEBLOCKLOCATIONS for webhdfs, along with the info of its response json type, In our case, client is trying to switch their application from calling java API to webhdfs, then failed to find the equivalent API of {{getFileBlockLocations}}. No doc provided, we figured by reading the source code but end up with an unexpected output from current GET_BLOCK_LOCATIONS. This is not user-friendly. I am going to remove the tag of "Incompatible changes" because adding a new API with well documented will maintain the compatibility, and provide a FileSystem compliant way to user to get block locations via rest api. Further more, what's the meaning of keep a "private unstable op" in web hdfs? Webhdfs APIs are not private, you can't stop user calling them. If it is considered as unstable, how about something in document {noformat} Get File Block Locations Submit a HTTP GET request curl -i -L "http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS ... Deprecated : use GETFILEBLOCKLOCATIONS instead. {noformat} To Andrew's suggestion : bq. have you also considered implementing listLocatedStatus, which is IMO the better API since it returns both listing and locations in a single call? this API returns BlockLocations[] + FileStatus, it has more information but also means more work for clients to parse and more stuff on network. getFileBlockLocations API should be good in most cases. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15706889#comment-15706889 ] Andrew Wang commented on HDFS-11156: I looked at the git blame, and HDFS-2527 which added the private/unstable annotation also did a rename from GETFILEBLOCKLOCATIONS to GET_BLOCK_LOCATIONS. I'm guessing it wasn't exposed initially to keep the public WebHDFS API as narrow as possible. [~szetszwo] do you have any historical perspective on this? Could you also provide more information about your usecase? Is there an out-of-tree client you are maintaining? Also, given that GET_BLOCK_LOCATIONS is effectively stable, can you just use that instead (like the Java WebHdfsFileSystem implementation)? If we do need to add a new API (which I'm okay with in the abstract), have you also considered implementing listLocatedStatus, which is IMO the better API since it returns both listing and locations in a single call? > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705922#comment-15705922 ] Mingliang Liu commented on HDFS-11156: -- They are two different problems. P1. The mismatch of REST {{getFileBlockLocations}} with FileSystem API is a public/well-known feature which was designed to be this. The general contract of REST API, as Weiwei pointed out above, is to support the FileSystem interface. So I believe this is not the case. Moreover, mismatching APIs should be well-documented which is extremely useful to new users. {{GET_BLOCK_LOCATIONS}} is a private unstable op per its documentation, see [here|https://github.com/apache/hadoop/blob/d8bab3dcb693b2773ede9a6e4f71ae85ee056f79/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/resources/GetOpParam.java#L37-L37]. It's even not on the page of WebHDFS REST API documentation. P2. The mismatch was an omission but we don't want to fix it for the sake of wire compatibility. I can understand that our users may have been using this and it's considered to some degree public/stable. I agree with the community decision that we should break the wire compatibility only for a good reason. I propose that we create a new OP named {{GETFILEBLOCKLOCATIONS}} for REST API. The name pattern follows other REST APIs (no underscores); it returns {{BlockLocations[]}} as other FileSystem does. {{WebHdfsFileSystem}} should not be affected as it returns {{BlockLocation[]}} anyway. Thanks, > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704729#comment-15704729 ] Hadoop QA commented on HDFS-11156: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} hadoop-hdfs-project: The patch generated 0 new + 230 unchanged - 4 fixed = 230 total (was 234) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 57s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 10s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}100m 7s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.blockmanagement.TestReplicationPolicy | | | hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HDFS-11156 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840821/HDFS-11156.04.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux c9648c977431 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 67d9f28 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17694/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17694/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project | |
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704644#comment-15704644 ] Weiwei Yang commented on HDFS-11156: Hello [~andrew.wang] Thanks for providing your thought. >From the [webhdfs >document|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html] > {{The HTTP REST API supports the complete FileSystem/FileContext interface for HDFS}} user would expect the BlockLocation[] instead of LocatedBlocks according to [FileSystem#getFileBlockLocations|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html#getFileBlockLocations-org.apache.hadoop.fs.Path-long-long-] interface. I checked rest of APIs, they seemed good like {{GETFILESTATUS (see FileSystem.getFileStatus)}} Isn't this a bug worth to fix? I am hoping you can reconsider this. Thank you > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15704573#comment-15704573 ] Andrew Wang commented on HDFS-11156: Hi folks, I'm -1 on this change. Our users rely on WebHDFS in particular for compatibility across major versions. In general, we should not break wire compatibility unless there are *very* good reasons, and we're trying to preserve wire compatibility even for Hadoop 3. WebHDFS also does not expose the FileSystem interface, it exposes the HDFS interface. This means we sometimes return extra stuff like what we have in LocatedBlock vs the generic BlockLocation. This explains the mismatch with the FileSystem API. I'd like to close this as WONTFIX if you agree, thanks. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch, HDFS-11156.04.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703132#comment-15703132 ] Mingliang Liu commented on HDFS-11156: -- The patch looks good to me overall. +1 I have several minor comments: # Do we need to specially handle the case of {{list.isEmpty()}}? Looks pretty similar to the {{else}} clause. {code} 593 } else if (list.isEmpty()) { 594 return new String[0]; 595 } else { 596 final String[] array = new String[list.size()]; 597 int i = 0; 598 for (Object object : list) { 599 array[i++] = object.toString(); 600 } 601 return array; 602 } {code} # Are you considering using {{assertArrayEquals()}}? {code:title=testWebHdfsGetBlockLocationsWithStorageType()} 899 for(int j=0; jWebhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688779#comment-15688779 ] Weiwei Yang commented on HDFS-11156: Thanks a lot [~liuml07] > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688745#comment-15688745 ] Mingliang Liu commented on HDFS-11156: -- It may happen this week or after the holiday. Thanks, > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688666#comment-15688666 ] Weiwei Yang commented on HDFS-11156: Hello Mingliang Liu Would you help to review this patch ? I have built latest trunk and tested with this patch, use {code} http://:/webhdfs/v1/tmp/test1.img?op=GET_BLOCK_LOCATIONS=0=1=yangww {code} it returns output {code} {"BlockLocations":[{"cachedHosts":[],"corrupt":false,"hosts":["deale1.fyre.ibm.com"],"length":134217728,"names":["9.30.167.122:9866"],"offset":0,"storageIds":["DS-2c136bcd-d658-4853-8b7d-9ecb354e6093"],"storageTypes":["DISK"],"topologyPaths":["/default-rack/9.30.167.122:9866"]}]} {code} client would easily parse this to BlockLocation array using json parser or object mapper, just like the test case code I added to {{TestWebHDFS}}. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688566#comment-15688566 ] Weiwei Yang commented on HDFS-11156: I don't think the test failures in latest jenkins job related to this patch, I run them successfully locally, also 1st jenkins job UT was clean, v2/v3 patch were just checkstyle fixes. > Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API > -- > > Key: HDFS-11156 > URL: https://issues.apache.org/jira/browse/HDFS-11156 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 2.7.3 >Reporter: Weiwei Yang >Assignee: Weiwei Yang > Attachments: HDFS-11156.01.patch, HDFS-11156.02.patch, > HDFS-11156.03.patch > > > Following webhdfs REST API > {code} > http://:/webhdfs/v1/?op=GET_BLOCK_LOCATIONS=0=1 > {code} > will get a response like > {code} > { > "LocatedBlocks" : { > "fileLength" : 1073741824, > "isLastBlockComplete" : true, > "isUnderConstruction" : false, > "lastLocatedBlock" : { ... }, > "locatedBlocks" : [ {...} ] > } > } > {code} > This represents for *o.a.h.h.p.LocatedBlocks*. However according to > *FileSystem* API, > {code} > public BlockLocation[] getFileBlockLocations(Path p, long start, long len) > {code} > clients would expect an array of BlockLocation. This mismatch should be > fixed. Marked as Incompatible change as this will change the output of the > GET_BLOCK_LOCATIONS API. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15687118#comment-15687118 ] Hadoop QA commented on HDFS-11156: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 59s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 30s{color} | {color:green} hadoop-hdfs-project: The patch generated 0 new + 231 unchanged - 4 fixed = 231 total (was 235) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 54s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 37s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 93m 47s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker | | | hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR | | | hadoop.hdfs.server.balancer.TestBalancer | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HDFS-11156 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840030/HDFS-11156.03.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 1fc06549b7a4 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a926f89 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17633/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17633/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs-client
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15686763#comment-15686763 ] Hadoop QA commented on HDFS-11156: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 34s{color} | {color:orange} hadoop-hdfs-project: The patch generated 1 new + 230 unchanged - 4 fixed = 231 total (was 234) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 56s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 91m 7s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}121m 53s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HDFS-11156 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12840009/HDFS-11156.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 60cd3d32f1a4 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a926f89 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/17631/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/17631/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/17631/testReport/ | | modules | C:
[jira] [Commented] (HDFS-11156) Webhdfs rest api GET_BLOCK_LOCATIONS output doesn't comply with FileSystem API
[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15686250#comment-15686250 ] Hadoop QA commented on HDFS-11156: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 6s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 30s{color} | {color:orange} hadoop-hdfs-project: The patch generated 3 new + 231 unchanged - 3 fixed = 234 total (was 234) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 48s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 53s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 61m 54s{color} | {color:green} hadoop-hdfs in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 88m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs-project/hadoop-hdfs | | | Private method org.apache.hadoop.hdfs.web.JsonUtil.toJsonArray(DatanodeInfo[]) is never called At JsonUtil.java:called At JsonUtil.java:[lines 189-198] | | | Private method org.apache.hadoop.hdfs.web.JsonUtil.toJsonMap(ExtendedBlock) is never called At JsonUtil.java:called At JsonUtil.java:[lines 139-148] | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:a9ad5d6 | | JIRA Issue | HDFS-11156 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12839964/HDFS-11156.01.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 99268116dc01 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 6f80742 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle |