[jira] [Comment Edited] (HADOOP-13998) initial s3guard preview

2017-06-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035280#comment-16035280 ] Sean Mackrory edited comment on HADOOP-13998 at 6/2/17 7:23 PM:

[jira] [Assigned] (HADOOP-14484) Ensure deleted parent directory tombstones are overwritten when implicitly recreated

2017-06-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14484: -- Assignee: Sean Mackrory > Ensure deleted parent directory tombstones are overwritten

[jira] [Assigned] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14457: -- Assignee: Sean Mackrory > create() does not notify metadataStore of parent

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.003.patch Attaching a patch that adds all the test

[jira] [Created] (HADOOP-14489) ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

2017-06-05 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14489: -- Summary: ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured Key: HADOOP-14489 URL: https://issues.apache.org/jira/browse/HADOOP-14489

[jira] [Commented] (HADOOP-14484) Ensure deleted parent directory tombstones are overwritten when implicitly recreated

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037033#comment-16037033 ] Sean Mackrory commented on HADOOP-14484: [~ste...@apache.org] - yeah that's what I ended up

[jira] [Resolved] (HADOOP-14484) Ensure deleted parent directory tombstones are overwritten when implicitly recreated

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory resolved HADOOP-14484. Resolution: Duplicate Resolving this as a duplicate, since I did end up doing it as part

[jira] [Updated] (HADOOP-14489) ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14489: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-13345 > ITestS3GuardConcurrentOps

[jira] [Commented] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041525#comment-16041525 ] Sean Mackrory commented on HADOOP-14448: That failure's my fault - I had pushed before running

[jira] [Updated] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14448: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Play nice with

[jira] [Commented] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041579#comment-16041579 ] Sean Mackrory commented on HADOOP-14448: >From the local Yetus run: {quote} | Vote |

[jira] [Updated] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14448: --- Status: Patch Available (was: Open) > Play nice with ITestS3AEncryptionSSEC >

[jira] [Updated] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14499: --- Attachment: HADOOP-14499-HADOOP-13345.001.patch If we're looking for a quick solution to

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041728#comment-16041728 ] Sean Mackrory commented on HADOOP-14457: +1 - works for me. I'm stepping out shortly, but can

[jira] [Assigned] (HADOOP-14506) Add create() contract test that verifies ancestor dir creation

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14506: -- Assignee: Sean Mackrory > Add create() contract test that verifies ancestor dir

[jira] [Commented] (HADOOP-14504) ProvidedFileStatusIterator#next() may throw IndexOutOfBoundsException

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042019#comment-16042019 ] Sean Mackrory commented on HADOOP-14504: +1 to the direct test in the .002 patch, too :) >

[jira] [Commented] (HADOOP-14504) ProvidedFileStatusIterator#next() may throw IndexOutOfBoundsException

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042015#comment-16042015 ] Sean Mackrory commented on HADOOP-14504: HADOOP-14499 was already filed for the extant findbugs

[jira] [Updated] (HADOOP-14506) Add create() contract test that verifies ancestor dir creation

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14506: --- Attachment: HADOOP-14506.001.patch Adding the test from HADOOP-14457 (I'll put the other

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033381#comment-16033381 ] Sean Mackrory commented on HADOOP-14457: Do also not the bounds-checking I added to

[jira] [Updated] (HADOOP-13221) s3a create() doesn't check for a parent path being a file

2017-06-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13221: --- Attachment: HADOOP-13321-test.patch Attaching a test for this originally written as part of

[jira] [Comment Edited] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033381#comment-16033381 ] Sean Mackrory edited comment on HADOOP-14457 at 6/1/17 6:05 PM: Do also

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.002.patch Thanks for the feedback,

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.006.patch > create() does not notify metadataStore of

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16042845#comment-16042845 ] Sean Mackrory commented on HADOOP-14457: There's a bit of overlap between this and HADOOP-14505,

[jira] [Commented] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043073#comment-16043073 ] Sean Mackrory commented on HADOOP-14488: I added DELAY_KEY_SUBSTRING to the work /

[jira] [Updated] (HADOOP-14506) Add create() contract test that verifies ancestor dir creation

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14506: --- Attachment: HADOOP-14506.002.patch Fixing the description and adding text to all asserts.

[jira] [Updated] (HADOOP-14506) Add create() contract test that verifies ancestor dir creation

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14506: --- Status: Patch Available (was: Open) > Add create() contract test that verifies ancestor

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.007.patch This needed a minor tweak to be rebase on

[jira] [Updated] (HADOOP-14505) simplify mkdirs() after S3Guard delete tracking change

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14505: --- Attachment: HADOOP-14505-HADOOP-13345.001.patch Attaching a patch. Is mostly changes

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-002.patch Attaching that update to the test, rebasing

[jira] [Commented] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043582#comment-16043582 ] Sean Mackrory commented on HADOOP-14488: And it actually just occurred to me that although this

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043602#comment-16043602 ] Sean Mackrory commented on HADOOP-14499: Well the idea behind prune is to not store metadata

[jira] [Commented] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043605#comment-16043605 ] Sean Mackrory commented on HADOOP-14488: Ah - developed on top of my latest patch for

[jira] [Commented] (HADOOP-14509) InconsistentAmazonS3Client adds extra paths to listStatus() after delete.

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043608#comment-16043608 ] Sean Mackrory commented on HADOOP-14509: We can maybe resolve this as a duplicate of

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.004.patch Attaching a patch that switches from

[jira] [Updated] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14433: --- Status: Patch Available (was: Open) >

[jira] [Updated] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14433: --- Summary: ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name

[jira] [Commented] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038922#comment-16038922 ] Sean Mackrory commented on HADOOP-14433: Yep - I've tested in parallel mode with -Ds3guard,

[jira] [Commented] (HADOOP-13760) S3Guard: add delete tracking

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037348#comment-16037348 ] Sean Mackrory commented on HADOOP-13760: [~ste...@apache.org] Note that the exception message

[jira] [Assigned] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14448: -- Assignee: Sean Mackrory > Play nice with ITestS3AEncryptionSSEC >

[jira] [Created] (HADOOP-14499) Findbufs warning in LocalMetadataStore

2017-06-06 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14499: -- Summary: Findbufs warning in LocalMetadataStore Key: HADOOP-14499 URL: https://issues.apache.org/jira/browse/HADOOP-14499 Project: Hadoop Common Issue

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.005.patch Rebasing on the latest merge... > create()

[jira] [Updated] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14433: --- Resolution: Fixed Target Version/s: HADOOP-13345 Status: Resolved

[jira] [Updated] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14433: --- Fix Version/s: HADOOP-13345 > ITestS3GuardConcurrentOps.testConcurrentTableCreations fails

[jira] [Commented] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations fails without table name configured

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039708#comment-16039708 ] Sean Mackrory commented on HADOOP-14433: Pushing shortly (findbugs warning is unrelated and

[jira] [Commented] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040884#comment-16040884 ] Sean Mackrory commented on HADOOP-14448: Ran against us-west-2. Just re-ran against eu-central-1

[jira] [Updated] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14448: --- Attachment: HADOOP-14448-HADOOP-13345.001.patch Attaching patch that skips the tests that

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041001#comment-16041001 ] Sean Mackrory commented on HADOOP-14457: So none of these are related to this patch. I wonder if

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.009.patch > create() does not notify metadataStore of

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.008.patch Yeah I was on the fence about doing the

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-003.patch Yeah something like that. The next

[jira] [Commented] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049974#comment-16049974 ] Sean Mackrory commented on HADOOP-14488: [~fabbri] Thanks - +1 to the changes as you describe

[jira] [Comment Edited] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048376#comment-16048376 ] Sean Mackrory edited comment on HADOOP-14457 at 6/13/17 8:40 PM: - Had

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048376#comment-16048376 ] Sean Mackrory commented on HADOOP-14457: Had lot of back-and-forth about this offline with

[jira] [Commented] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050439#comment-16050439 ] Sean Mackrory commented on HADOOP-14488: +1 to that patch. I can commit it this morning. Also,

[jira] [Updated] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was:

[jira] [Updated] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14499: --- Attachment: HADOOP-14499-HADOOP-13345.002.patch Attaching a patch that: * Resolves the

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045537#comment-16045537 ] Sean Mackrory commented on HADOOP-14457: A capabilities-based approach seems to me to be far

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-004.patch So after digging into making sure this

[jira] [Assigned] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14488: -- Assignee: Sean Mackrory > s3guard listStatus fails after renaming file into

[jira] [Commented] (HADOOP-14505) simplify mkdirs() after S3Guard delete tracking change

2017-06-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052030#comment-16052030 ] Sean Mackrory commented on HADOOP-14505: So the changes in HADOOP-13760 actually make a batch of

[jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking

2017-05-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13760: --- Attachment: HADOOP-13760-HADOOP-13345.005.patch Thanks for the analysis, [~fabbri]. The

[jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking

2017-05-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13760: --- Attachment: HADOOP-13760-HADOOP-13345.004.patch Attaching another patch. All tests are

[jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking

2017-05-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13760: --- Attachment: HADOOP-13760-HADOOP-13345.006.patch Thanks [~liuml07]. When the "tombstone"

[jira] [Created] (HADOOP-14457) LocalMetadataStore falsely reports empty parent directory authoritatively

2017-05-25 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14457: -- Summary: LocalMetadataStore falsely reports empty parent directory authoritatively Key: HADOOP-14457 URL: https://issues.apache.org/jira/browse/HADOOP-14457

[jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking

2017-05-24 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13760: --- Status: Patch Available (was: Open) > S3Guard: add delete tracking >

[jira] [Updated] (HADOOP-13760) S3Guard: add delete tracking

2017-05-24 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13760: --- Attachment: HADOOP-13760-HADOOP-13345.011.patch {quote}I'd also be ok with renaming

[jira] [Commented] (HADOOP-14448) Play nice with ITestS3AEncryptionSSEC

2017-06-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033073#comment-16033073 ] Sean Mackrory commented on HADOOP-14448: +1 to that. I took a deeper look at this yesterday:

[jira] [Commented] (HADOOP-13998) initial s3guard preview

2017-05-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030140#comment-16030140 ] Sean Mackrory commented on HADOOP-13998: I would generally support a merge where we are in

[jira] [Commented] (HADOOP-14489) ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037640#comment-16037640 ] Sean Mackrory commented on HADOOP-14489: So this has always succeeded for me because my

[jira] [Updated] (HADOOP-14433) ITestS3GuardConcurrentOps.testConcurrentTableCreations failing on local dynamo

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14433: --- Attachment: HADOOP-14433-HADOOP-13345.001.patch So this has always succeeded for me because

[jira] [Issue Comment Deleted] (HADOOP-14489) ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14489: --- Comment: was deleted (was: So this has always succeeded for me because my auth-keys.xml

[jira] [Resolved] (HADOOP-14489) ITestS3GuardConcurrentOps requires explicit DynamoDB table name to be configured

2017-06-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory resolved HADOOP-14489. Resolution: Fixed Resolving as a duplicate. Thanks [~liuml07] >

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039332#comment-16039332 ] Sean Mackrory commented on HADOOP-14457: Also, I looked a bit deeper into the failure that is

[jira] [Comment Edited] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039332#comment-16039332 ] Sean Mackrory edited comment on HADOOP-14457 at 6/6/17 6:01 PM: Also, I

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Status: Patch Available (was: Open) > create() does not notify metadataStore of parent

[jira] [Updated] (HADOOP-14835) mvn site build throws SAX errors

2017-09-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14835: --- Attachment: HADOOP-14835.003.patch +1 - but attaching a v3 patch that also adds a note to

[jira] [Moved] (HADOOP-14931) Isolation of native libraries in JARs

2017-10-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory moved HDFS-12595 to HADOOP-14931: --- Key: HADOOP-14931 (was: HDFS-12595) Project: Hadoop Common

[jira] [Commented] (HADOOP-14801) s3guard diff demand creates a new table

2017-08-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138736#comment-16138736 ] Sean Mackrory commented on HADOOP-14801: This is probably it. The forceCreate option was only

[jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system

2017-08-31 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149157#comment-16149157 ] Sean Mackrory commented on HADOOP-13887: Just wanted to comment that I gave [~moist] some

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.009.patch Thanks Yonger. I've attached a patch with some trivial

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252886#comment-16252886 ] Sean Mackrory commented on HADOOP-14475: Okay - thanks for the additional context. I think the

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254662#comment-16254662 ] Sean Mackrory commented on HADOOP-14475: {quote}DefaultMetricsSystem lifecycle API is confusing

[jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254662#comment-16254662 ] Sean Mackrory edited comment on HADOOP-14475 at 11/16/17 2:37 AM: --

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267086#comment-16267086 ] Sean Mackrory commented on HADOOP-14475: {quote}I think we'd probably be better off with

[jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269021#comment-16269021 ] Sean Mackrory edited comment on HADOOP-14475 at 11/28/17 4:44 PM: -- Yeah

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268964#comment-16268964 ] Sean Mackrory commented on HADOOP-14475: Okay - sounds like we're on the same page, then -

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269021#comment-16269021 ] Sean Mackrory commented on HADOOP-14475: Yeah FileSink seems to be too primitive for real-world

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268073#comment-16268073 ] Sean Mackrory commented on HADOOP-14475: [~iyonger] I don't believe I've changed anything that

[jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268073#comment-16268073 ] Sean Mackrory edited comment on HADOOP-14475 at 11/28/17 4:07 AM: --

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272970#comment-16272970 ] Sean Mackrory commented on HADOOP-14475: I never saw the issue you were referring to and can't

[jira] [Created] (HADOOP-15079) ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch

2017-11-30 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-15079: -- Summary: ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch Key: HADOOP-15079 URL: https://issues.apache.org/jira/browse/HADOOP-15079

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273554#comment-16273554 ] Sean Mackrory commented on HADOOP-14475: Filed HADOOP-15079. I'm quite certain it's unrelated to

[jira] [Commented] (HADOOP-15079) ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch

2017-11-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273638#comment-16273638 ] Sean Mackrory commented on HADOOP-15079: I suspected that simply removing the

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.015.patch Conflict presumably due to s3a output committer.

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267865#comment-16267865 ] Sean Mackrory commented on HADOOP-14475: One other thing worth mentioning is the compatibility

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.014.patch So I see the local hostname is already included in

[jira] [Commented] (HADOOP-15058) create-release site build outputs dummy shaded jars due to skipShade

2017-12-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277616#comment-16277616 ] Sean Mackrory commented on HADOOP-15058: Looks good to me! +1 > create-release site build

[jira] [Updated] (HADOOP-15080) Cat-X dependency on org.json via derived json-lib

2017-12-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-15080: --- Status: Patch Available (was: Open) Just "submitting" to get a Pre-Commit run - I'm not

[jira] [Commented] (HADOOP-15058) create-release site build outputs dummy shaded jars due to skipShade

2017-12-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277555#comment-16277555 ] Sean Mackrory commented on HADOOP-15058: [~andrew.wang] Sure I'll take a look. At first glance I

<    1   2   3   4   5   6   7   8   9   10   >