[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154050#comment-16154050 ] Steve Loughran commented on HADOOP-13998: - see HADOOP-14838 for branch-2 backport > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Fix For: 3.0.0-beta1 > > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150233#comment-16150233 ] Steve Loughran commented on HADOOP-13998: - Have to update the testing.md doc on that, maybe a list of "intermittently unreliable tests". And @flakytest attribute would be cute,,,yetus could generate a report and if it were a @flakytest then it could be less fussy about a veto. However, that'd potentially put us off tracking down problems. Finishing off the commit today > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149826#comment-16149826 ] Aaron Fabbri commented on HADOOP-13998: --- +1 on v5 patch (non-binding, see upstream VOTE). Didn't re-read the whole thing carefully but tested in us-west-2. All S3A integration and unit tests passed w/ and w/o s3guard. On the s3guard run, I noticed one failure which went away when I re-ran it: {{ITestS3GuardToolDynamoDB#testPruneCommandCLI}}. This is a sleep-based prune test case that may in rare cases fail (because your computer / network / dynamodb goes out to lunch for >1 second). In the future, we should add a comment to the test case saying it may transiently fail. These tests are still useful, kinda wish we had a way to mark them as flaky-but-useful. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149638#comment-16149638 ] Aaron Fabbri commented on HADOOP-13998: --- Applying and testing patch 005 now. For Yetus results above: Javac warnings are just the AmazonS3Client constructor deprecation: {code} * @deprecated use {@link AmazonS3ClientBuilder#defaultClient()} */ @Deprecated public AmazonS3Client() { this(new S3CredentialsProviderChain()); } {code} Checkstyle issues are line length where we cannot break (e.g. javadoc link urls), method length (innerRename), and "more than 7 method parameters" on two functions. The all seem like reasonable exceptions. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch, HADOOP-13998-005.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149458#comment-16149458 ] Hadoop QA commented on HADOOP-13998: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m 1s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 59 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 57s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 58s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 18s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 37s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 25s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 11m 25s{color} | {color:red} root generated 2 new + 1283 unchanged - 1 fixed = 1285 total (was 1284) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 12s{color} | {color:orange} root: The patch generated 4 new + 205 unchanged - 4 fixed = 209 total (was 209) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 24s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 8s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 38s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} hadoop-project in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 30s{color} | {color:green} hadoop-assemblies in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 17s{color} | {color:green} hadoop-common in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 1s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 44s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}123m 52s{color} | {color:black} {color} | \\ \\ || Subsystem ||
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138149#comment-16138149 ] Steve Loughran commented on HADOOP-13998: - I'm doing the info command along with a bit of a rework of the CLI code; just the aggregate set of niggles. I don't want to hold the vote up though as it'll need work and is non-critical > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129461#comment-16129461 ] Sean Mackrory commented on HADOOP-13998: I'm happy with this patch. +1 to an 's3guard info' tool. I also made a few nit-picky comments about grouping S3Guard things together better in the config-related files. But other than that I've tested this to my satisfaction and reviewed the code with no concerns worth holding up a merge. I can address my nit-picky thoughts next time I'm editing one of those files. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129181#comment-16129181 ] Aaron Fabbri commented on HADOOP-13998: --- FYI -Ds3guard (local) tests all passed once I removed parallel-tests option. I think there are some lingering issues with root contract tests and encryption integration tests in parallel test mode. I'm +1 (nonbinding) on this patch. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128702#comment-16128702 ] Steve Loughran commented on HADOOP-13998: - +I'll make a patch of the changes between HADOOP-13345 branch & the latest patch here for review/commit to that branch pre-vote > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128695#comment-16128695 ] Steve Loughran commented on HADOOP-13998: - OK. Do we have any other showstoppers? Having played with the tool, I'd really like a "s3guard info URI" command to list the s3guard state of a bucket (is it protected, is it encrypted, endpoint, with some opts for the command to fail if no s3guard or encryption. Why? No easy way to verify that a bucket is protected, not once you have complex per-bucket setup. But that could go in after atop trunk easily enough; no reason not to hold this merge up. *If everyone is happy with this patch, it's time to raise it as a vote in hadoop-common. I'll draw people's attention to it on the list now* > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128169#comment-16128169 ] Aaron Fabbri commented on HADOOP-13998: --- v4 applied cleanly. S3A tests w/o S3guard all passed. Added -Dparallel-tests and -Ds3guard and saw some failures (ITestS3AEncryptionSSEC stuff and a couple of ITestS3AContractRootDir). Rerunning w/o parallel mode then I'll run some tests with dynamo. > Merge initial S3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch, > HADOOP-13998-003.patch, HADOOP-13998-004.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127956#comment-16127956 ] Hadoop QA commented on HADOOP-13998: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 59 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 9s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 44s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 10s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 41s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 57s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 13m 57s{color} | {color:red} root generated 2 new + 1316 unchanged - 1 fixed = 1318 total (was 1317) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 8s{color} | {color:orange} root: The patch generated 4 new + 205 unchanged - 4 fixed = 209 total (was 209) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 22s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 11s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 30s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 23s{color} | {color:green} hadoop-project in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 25s{color} | {color:green} hadoop-assemblies in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 32s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 41s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}101m 18s{color} | {color:black} {color} | \\ \\ || Reason || Tests || |
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127597#comment-16127597 ] Hadoop QA commented on HADOOP-13998: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 59 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 27s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 10s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 59s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 20s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 13m 2s{color} | {color:red} root generated 2 new + 1316 unchanged - 1 fixed = 1318 total (was 1317) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 30s{color} | {color:orange} root: The patch generated 4 new + 205 unchanged - 4 fixed = 209 total (was 209) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 24s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 10s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 40s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 27s{color} | {color:green} hadoop-project in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 30s{color} | {color:green} hadoop-assemblies in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 30s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 8s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 47s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}102m 38s{color} | {color:black} {color} | \\ \\ || Reason || Tests || |
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127041#comment-16127041 ] Steve Loughran commented on HADOOP-13998: - # I want to do a checkstyle fix # we do need to follow the full vote for a branch merge; I believe I can be one of the voters. > Merge initial s3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126687#comment-16126687 ] Sean Mackrory commented on HADOOP-13998: Thanks, [~steve_l] - I'm reviewing (is a comitter +1 even binding on this? I seem to recall branch merges being PMC votes, but I don't see that in the by-laws). I ran tests locally as well and only got some typical S3N flakiness: {code} testListStatus(org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract) Time elapsed: 2.527 sec <<< FAILURE! java.lang.AssertionError: expected:<1> but was:<12> testRenameDirectoryAsExistingFile(org.apache.hadoop.fs.s3native.ITestJets3tNativeS3FileSystemContract) Time elapsed: 1.38 sec <<< FAILURE! java.lang.AssertionError: Source exists expected: but was: {code} > Merge initial s3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.0.0-beta1 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13998-001.patch, HADOOP-13998-002.patch > > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124737#comment-16124737 ] Hadoop QA commented on HADOOP-13998: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 59 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 19s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 6s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 57s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 19s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 23s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 50s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 10m 50s{color} | {color:red} root generated 2 new + 1344 unchanged - 1 fixed = 1346 total (was 1345) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 11s{color} | {color:orange} root: The patch generated 19 new + 205 unchanged - 4 fixed = 224 total (was 209) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 0s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 25s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 8s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 38s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} hadoop-project in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 30s{color} | {color:green} hadoop-assemblies in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 18s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 44s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 96m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || |
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124660#comment-16124660 ] Hadoop QA commented on HADOOP-13998: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 59 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 37s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m 9s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m 55s{color} | {color:green} trunk passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 19s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m 11s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 11s{color} | {color:red} root generated 2 new + 1344 unchanged - 1 fixed = 1346 total (was 1345) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 2m 16s{color} | {color:orange} root: The patch generated 31 new + 205 unchanged - 4 fixed = 236 total (was 209) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m 1s{color} | {color:green} There were no new shellcheck issues. {color} | | {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m 26s{color} | {color:green} There were no new shelldocs issues. {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 1s{color} | {color:red} The patch has 8 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 9s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-project hadoop-assemblies {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 36s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} hadoop-project in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 29s{color} | {color:green} hadoop-assemblies in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 24s{color} | {color:red} hadoop-common in the patch failed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 58s{color} | {color:green} hadoop-aws in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 43s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} |
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123360#comment-16123360 ] Steve Loughran commented on HADOOP-13998: - Status: review nearly ready. Once In I'm going to create a full diff to see what Yetus says...this will list all our style/findbugs issues, so it's a big change to review or ignore things. I'm also thinking of moving all tests from Lambdas to closures, so that a branch-2 will be nearly identical to this one. This will make integration easier, as well as aid backporting any phase II work. Mingliang did this internally, which I've got as a starting base. But as intelliJ does it itself, it may be simplest to use it for that, leaving only any other 8 -> 7 migration issue. Mingliang: what were the other java 7 migration issues other than us using Lambda expressions in tests. (FWIW, all the error handling in HADOOP-13786 uses L-exps in production code to wrap AWS SDK calls with retry logic, something we need to roll out across the FS to handle throttling and other transients. I really want to stay in java 8 there) > Merge initial s3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Steve Loughran > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120254#comment-16120254 ] Steve Loughran commented on HADOOP-13998: - We are pretty much done here, down to those review-of-spelling nits. I'm about do do a merge into s3guard of trunk again, as I can see things have diverged (if I mix builds, I get errors about commons-lang3 missing) > Merge initial s3guard release into trunk > > > Key: HADOOP-13998 > URL: https://issues.apache.org/jira/browse/HADOOP-13998 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Steve Loughran > > JIRA to link in all the things we think are needed for a preview/merge into > trunk -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org