[jira] [Resolved] (HADOOP-14973) [s3a] Log StorageStatistics

2017-11-20 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory resolved HADOOP-14973. Resolution: Duplicate Resolving in favor of HADOOP-14475 since that's the right approach.

[jira] [Created] (HADOOP-15055) Add s3 metrics from AWS SDK to s3a metrics tracking

2017-11-20 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-15055: -- Summary: Add s3 metrics from AWS SDK to s3a metrics tracking Key: HADOOP-15055 URL: https://issues.apache.org/jira/browse/HADOOP-15055 Project: Hadoop Common

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-20 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.013.patch * Made S3AInstrumentation implement Closeable and use

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.012.patch - Dropping .stop(). Looking at the MetricsSystemImpl

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16257249#comment-16257249 ] Sean Mackrory commented on HADOOP-14475: Also note that I made the record name / source name

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.011.patch Same as .010. but with: - Checkstyle issues fixed, with

[jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256303#comment-16256303 ] Sean Mackrory edited comment on HADOOP-14475 at 11/17/17 1:51 AM: --

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.010.patch Attaching a patch trying out the idea of a separate

[jira] [Comment Edited] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254662#comment-16254662 ] Sean Mackrory edited comment on HADOOP-14475 at 11/16/17 2:37 AM: --

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254662#comment-16254662 ] Sean Mackrory commented on HADOOP-14475: {quote}DefaultMetricsSystem lifecycle API is confusing

[jira] [Updated] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14475: --- Attachment: HADOOP-14475.009.patch Thanks Yonger. I've attached a patch with some trivial

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252886#comment-16252886 ] Sean Mackrory commented on HADOOP-14475: Okay - thanks for the additional context. I think the

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251577#comment-16251577 ] Sean Mackrory commented on HADOOP-14475: Also some minor checkstyle issues - but if you can

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251543#comment-16251543 ] Sean Mackrory commented on HADOOP-14475: Thanks for the update [~iyonger]! Your .008 patch

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-11-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234542#comment-16234542 ] Sean Mackrory commented on HADOOP-14475: I tried this out on a cluster. I saw that if I

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225594#comment-16225594 ] Sean Mackrory commented on HADOOP-14475: Actually I do have a couple of thoughts on the patch: *

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225425#comment-16225425 ] Sean Mackrory commented on HADOOP-14475: This is needed: {code} -String targetMetricSource

[jira] [Commented] (HADOOP-14475) Metrics of S3A don't print out when enable it in Hadoop metrics property file

2017-10-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225089#comment-16225089 ] Sean Mackrory commented on HADOOP-14475: I'm taking a look at this. For what it's worth, before

[jira] [Commented] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222859#comment-16222859 ] Sean Mackrory commented on HADOOP-14973: {quote}discouraging any attempts to avoid

[jira] [Commented] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222715#comment-16222715 ] Sean Mackrory commented on HADOOP-14973: I think you misunderstand where these requests are

[jira] [Commented] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222441#comment-16222441 ] Sean Mackrory commented on HADOOP-14973: Yeah let me take a closer look at how well metrics2

[jira] [Updated] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-24 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14973: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-14831 > [s3a] Log StorageStatistics

[jira] [Updated] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-24 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14973: --- Summary: [s3a] Log StorageStatistics (was: Log StorageStatistics) > [s3a] Log

[jira] [Updated] (HADOOP-14973) Log StorageStatistics

2017-10-24 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14973: --- Affects Version/s: 3.0.0-beta1 2.8.1 > Log StorageStatistics >

[jira] [Created] (HADOOP-14973) Log StorageStatistics

2017-10-23 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14973: -- Summary: Log StorageStatistics Key: HADOOP-14973 URL: https://issues.apache.org/jira/browse/HADOOP-14973 Project: Hadoop Common Issue Type: Bug

[jira] [Updated] (HADOOP-14972) Histogram metrics types for latency, etc.

2017-10-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14972: --- Description: We'd like metrics to track latencies for various operations, such as latencies

[jira] [Created] (HADOOP-14972) Histogram metrics types for latency, etc.

2017-10-23 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14972: -- Summary: Histogram metrics types for latency, etc. Key: HADOOP-14972 URL: https://issues.apache.org/jira/browse/HADOOP-14972 Project: Hadoop Common

[jira] [Moved] (HADOOP-14931) Isolation of native libraries in JARs

2017-10-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory moved HDFS-12595 to HADOOP-14931: --- Key: HADOOP-14931 (was: HDFS-12595) Project: Hadoop Common

[jira] [Updated] (HADOOP-14835) mvn site build throws SAX errors

2017-09-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14835: --- Attachment: HADOOP-14835.003.patch +1 - but attaching a v3 patch that also adds a note to

[jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system

2017-08-31 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149157#comment-16149157 ] Sean Mackrory commented on HADOOP-13887: Just wanted to comment that I gave [~moist] some

[jira] [Commented] (HADOOP-14801) s3guard diff demand creates a new table

2017-08-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138736#comment-16138736 ] Sean Mackrory commented on HADOOP-14801: This is probably it. The forceCreate option was only

[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129461#comment-16129461 ] Sean Mackrory commented on HADOOP-13998: I'm happy with this patch. +1 to an 's3guard info'

[jira] [Commented] (HADOOP-13998) Merge initial s3guard release into trunk

2017-08-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126687#comment-16126687 ] Sean Mackrory commented on HADOOP-13998: Thanks, [~steve_l] - I'm reviewing (is a comitter +1

[jira] [Comment Edited] (HADOOP-14260) Configuration.dumpConfiguration should redact sensitive information

2017-08-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122623#comment-16122623 ] Sean Mackrory edited comment on HADOOP-14260 at 8/11/17 12:53 AM: -- +1,

[jira] [Commented] (HADOOP-14260) Configuration.dumpConfiguration should redact sensitive information

2017-08-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122623#comment-16122623 ] Sean Mackrory commented on HADOOP-14260: +1, pending a clean Yetus run. As a side-note, seeing

[jira] [Updated] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13952: --- Attachment: HADOOP-13952.001.patch Made the suggested changes, rebuilt and re-ran Yetus

[jira] [Updated] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13952: --- Attachment: HADOOP-13952.000.patch I added an "extra_dir" parameter so you can specify a

[jira] [Comment Edited] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114886#comment-16114886 ] Sean Mackrory edited comment on HADOOP-13952 at 8/4/17 7:51 PM: .000

[jira] [Updated] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13952: --- Status: Patch Available (was: Open) > tools dependency hooks are throwing errors >

[jira] [Commented] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16114453#comment-16114453 ] Sean Mackrory commented on HADOOP-13952: Changing findfileindir in dist-layout-stitching to only

[jira] [Comment Edited] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113722#comment-16113722 ] Sean Mackrory edited comment on HADOOP-13952 at 8/4/17 12:10 AM: - So:

[jira] [Commented] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113722#comment-16113722 ] Sean Mackrory commented on HADOOP-13952: So: hadoop-tool-dist is including these dependencies in

[jira] [Comment Edited] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113523#comment-16113523 ] Sean Mackrory edited comment on HADOOP-13952 at 8/3/17 9:29 PM: I

[jira] [Updated] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13952: --- Attachment: HADOOP-13952.preview.patch I couldn't find much official documentation on the

[jira] [Assigned] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-13952: -- Assignee: Sean Mackrory > tools dependency hooks are throwing errors >

[jira] [Commented] (HADOOP-13952) tools dependency hooks are throwing errors

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113086#comment-16113086 ] Sean Mackrory commented on HADOOP-13952: FYI most of these appear to have been fixed. What

[jira] [Updated] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14498: --- Attachment: HADOOP-14498.003.patch .003 patch address shellcheck issues and refactors the

[jira] [Updated] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14498: --- Attachment: HADOOP-14498.002.patch Attaching said patch It'd be nice to replace that

[jira] [Commented] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111994#comment-16111994 ] Sean Mackrory commented on HADOOP-14498: Aah that makes sense. I guess that's what the comment

[jira] [Updated] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14498: --- Status: Patch Available (was: Open) > HADOOP_OPTIONAL_TOOLS not parsed correctly >

[jira] [Updated] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14498: --- Attachment: HADOOP-14498.001.patch Now that HADOOP-13595 is in I had a look at this. It is

[jira] [Assigned] (HADOOP-14498) HADOOP_OPTIONAL_TOOLS not parsed correctly

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14498: -- Assignee: Sean Mackrory > HADOOP_OPTIONAL_TOOLS not parsed correctly >

[jira] [Updated] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-13595: --- Resolution: Fixed Fix Version/s: 3.0.0-beta1 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111341#comment-16111341 ] Sean Mackrory commented on HADOOP-13595: +1. Can commit shortly... > Rework hadoop_usage to be

[jira] [Commented] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1644#comment-1644 ] Sean Mackrory commented on HADOOP-13595: Ooh I see that now. Thanks. +1 from me, pending clean

[jira] [Commented] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1616#comment-1616 ] Sean Mackrory commented on HADOOP-13595: Thanks for the iteration. What's up with oifs stuff

[jira] [Created] (HADOOP-14717) Add StreamCapabilities support to s3a

2017-08-01 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14717: -- Summary: Add StreamCapabilities support to s3a Key: HADOOP-14717 URL: https://issues.apache.org/jira/browse/HADOOP-14717 Project: Hadoop Common Issue

[jira] [Comment Edited] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109889#comment-16109889 ] Sean Mackrory edited comment on HADOOP-13595 at 8/1/17 10:23 PM: - [~aw]

[jira] [Comment Edited] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109889#comment-16109889 ] Sean Mackrory edited comment on HADOOP-13595 at 8/1/17 10:22 PM: - [~aw]

[jira] [Commented] (HADOOP-13595) Rework hadoop_usage to be broken up by clients/daemons/etc.

2017-08-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109889#comment-16109889 ] Sean Mackrory commented on HADOOP-13595: [~aw] - I just reviewed this and did some testing. I'm

[jira] [Updated] (HADOOP-14505) simplify mkdirs() after S3Guard delete tracking change

2017-07-11 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14505: --- Assignee: Sean Mackrory Status: Patch Available (was: Open) Yes it is! That patch

[jira] [Updated] (HADOOP-14068) Add integration test version of TestMetadataStore for DynamoDB

2017-07-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14068: --- Attachment: HADOOP-14068-HADOOP-13345.008.patch I had been back to this, but after some

[jira] [Updated] (HADOOP-14548) S3Guard: issues running parallel tests w/ S3N

2017-07-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14548: --- Resolution: Fixed Target Version/s: HADOOP-13345 Status: Resolved

[jira] [Commented] (HADOOP-14548) S3Guard: issues running parallel tests w/ S3N

2017-07-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16078356#comment-16078356 ] Sean Mackrory commented on HADOOP-14548: {quote}S3n tests will all skip if you don't provide the

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-07-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.010.patch So attaching a patch that follows the

[jira] [Commented] (HADOOP-14576) DynamoDB tables may leave ACTIVE state after initial connection

2017-06-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061659#comment-16061659 ] Sean Mackrory commented on HADOOP-14576: {quote}do you think this is something to need for an

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16061643#comment-16061643 ] Sean Mackrory commented on HADOOP-14499: Filed HADOOP-14585 for that, as it's unrelated to the

[jira] [Created] (HADOOP-14585) Ensure controls in-place to prevent clients with significant clock skews pruning aggressively

2017-06-23 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14585: -- Summary: Ensure controls in-place to prevent clients with significant clock skews pruning aggressively Key: HADOOP-14585 URL:

[jira] [Updated] (HADOOP-14527) ITestS3GuardListConsistency is too slow

2017-06-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14527: --- Attachment: HADOOP-14527-HADOOP-13345.005.patch Pushed. I took the liberty of addressing

[jira] [Updated] (HADOOP-14527) ITestS3GuardListConsistency is too slow

2017-06-23 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14527: --- Resolution: Fixed Status: Resolved (was: Patch Available) >

[jira] [Created] (HADOOP-14577) testGetFileStatus failing

2017-06-22 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14577: -- Summary: testGetFileStatus failing Key: HADOOP-14577 URL: https://issues.apache.org/jira/browse/HADOOP-14577 Project: Hadoop Common Issue Type: Sub-task

[jira] [Created] (HADOOP-14576) DynamoDB tables may leave ACTIVE state after initial connection

2017-06-22 Thread Sean Mackrory (JIRA)
Sean Mackrory created HADOOP-14576: -- Summary: DynamoDB tables may leave ACTIVE state after initial connection Key: HADOOP-14576 URL: https://issues.apache.org/jira/browse/HADOOP-14576 Project:

[jira] [Commented] (HADOOP-14548) S3Guard: issues running parallel tests w/ S3N

2017-06-22 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16060095#comment-16060095 ] Sean Mackrory commented on HADOOP-14548: Those are actually all already in the sequential

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-22 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16059709#comment-16059709 ] Sean Mackrory commented on HADOOP-14499: Just noticed there are actually 2 findbugs entries, and

[jira] [Updated] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-22 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14499: --- Attachment: HADOOP-14499-HADOOP-13345.003.patch Fair enough. Same patch, but without the

[jira] [Commented] (HADOOP-14527) ITestS3GuardListConsistency is too slow

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058488#comment-16058488 ] Sean Mackrory commented on HADOOP-14527: +1 pending Yetus' cooperation. I can commit later. >

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058397#comment-16058397 ] Sean Mackrory commented on HADOOP-14499: That's from my comment on Jun 8th. Since we currently

[jira] [Commented] (HADOOP-14527) ITestS3GuardListConsistency is too slow

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058387#comment-16058387 ] Sean Mackrory commented on HADOOP-14527: +1 in general - although I'd prefer to remove the

[jira] [Commented] (HADOOP-14548) S3Guard: issues running parallel tests w/ S3N

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058365#comment-16058365 ] Sean Mackrory commented on HADOOP-14548: {code}Sean Mackrory any thoughts on #5?{code} The SSEC

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16057721#comment-16057721 ] Sean Mackrory commented on HADOOP-14499: I'm really quite mystified by that findbugs continuing

[jira] [Updated] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14499: --- Attachment: HADOOP-14499-HADOOP-13345.002.patch Attaching a patch that: * Resolves the

[jira] [Commented] (HADOOP-14505) simplify mkdirs() after S3Guard delete tracking change

2017-06-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052030#comment-16052030 ] Sean Mackrory commented on HADOOP-14505: So the changes in HADOOP-13760 actually make a batch of

[jira] [Updated] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was:

[jira] [Commented] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16050439#comment-16050439 ] Sean Mackrory commented on HADOOP-14488: +1 to that patch. I can commit it this morning. Also,

[jira] [Commented] (HADOOP-14488) s3guard w/ failure injection: listStatus fails after renaming file into directory

2017-06-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049974#comment-16049974 ] Sean Mackrory commented on HADOOP-14488: [~fabbri] Thanks - +1 to the changes as you describe

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-004.patch So after digging into making sure this

[jira] [Assigned] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HADOOP-14488: -- Assignee: Sean Mackrory > s3guard listStatus fails after renaming file into

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048376#comment-16048376 ] Sean Mackrory commented on HADOOP-14457: Had lot of back-and-forth about this offline with

[jira] [Comment Edited] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-13 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048376#comment-16048376 ] Sean Mackrory edited comment on HADOOP-14457 at 6/13/17 8:40 PM: - Had

[jira] [Commented] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045537#comment-16045537 ] Sean Mackrory commented on HADOOP-14457: A capabilities-based approach seems to me to be far

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.009.patch > create() does not notify metadataStore of

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-003.patch Yeah something like that. The next

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.008.patch Yeah I was on the fence about doing the

[jira] [Commented] (HADOOP-14509) InconsistentAmazonS3Client adds extra paths to listStatus() after delete.

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043608#comment-16043608 ] Sean Mackrory commented on HADOOP-14509: We can maybe resolve this as a duplicate of

[jira] [Commented] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043605#comment-16043605 ] Sean Mackrory commented on HADOOP-14488: Ah - developed on top of my latest patch for

[jira] [Commented] (HADOOP-14499) Findbugs warning in LocalMetadataStore

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043602#comment-16043602 ] Sean Mackrory commented on HADOOP-14499: Well the idea behind prune is to not store metadata

[jira] [Commented] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16043582#comment-16043582 ] Sean Mackrory commented on HADOOP-14488: And it actually just occurred to me that although this

[jira] [Updated] (HADOOP-14488) s3guard listStatus fails after renaming file into directory

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14488: --- Attachment: HADOOP-14488-HADOOP-13345-002.patch Attaching that update to the test, rebasing

[jira] [Updated] (HADOOP-14505) simplify mkdirs() after S3Guard delete tracking change

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14505: --- Attachment: HADOOP-14505-HADOOP-13345.001.patch Attaching a patch. Is mostly changes

[jira] [Updated] (HADOOP-14457) create() does not notify metadataStore of parent directories or ensure they're not existing files

2017-06-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HADOOP-14457: --- Attachment: HADOOP-14457-HADOOP-13345.007.patch This needed a minor tweak to be rebase on

  1   2   3   4   5   6   >