[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16343975#comment-16343975
]
Steve Moist commented on HADOOP-15006:
--
>what's your proposal for letting the client encryption be
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341776#comment-16341776
]
Steve Moist commented on HADOOP-15006:
--
Ok fixed it. The workflow with block output was causing it
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-15006:
-
Attachment: s3-cse-poc.patch
> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339855#comment-16339855
]
Steve Moist edited comment on HADOOP-15006 at 1/25/18 11:19 PM:
Found an
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-15006:
-
Attachment: (was: s3-cse-poc.patch)
> Encrypt S3A data client-side with Hadoop libraries &
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339855#comment-16339855
]
Steve Moist commented on HADOOP-15006:
--
I've attached a quick and dirty proof of concept. It's now
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-15006:
-
Attachment: s3-cse-poc.patch
> Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16320663#comment-16320663
]
Steve Moist commented on HADOOP-15006:
--
{quote}
you are going to have to handle the fact that its
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317345#comment-16317345
]
Steve Moist commented on HADOOP-15006:
--
{quote}
Do have any good links for further reading on the
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310101#comment-16310101
]
Steve Moist edited comment on HADOOP-15006 at 1/3/18 7:02 PM:
--
{quote}
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310101#comment-16310101
]
Steve Moist commented on HADOOP-15006:
--
{quote}
Before worrying about these, why not conduct some
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308621#comment-16308621
]
Steve Moist commented on HADOOP-15006:
--
{quote}
It appears there will be NO CHANGES to the KMS,
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16288396#comment-16288396
]
Steve Moist commented on HADOOP-15006:
--
I don't think anyone's started it. I posted the design doc
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286574#comment-16286574
]
Steve Moist commented on HADOOP-15006:
--
Hey [~fabbri] and [~steve_l], any traction on this?
>
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286572#comment-16286572
]
Steve Moist commented on HADOOP-14209:
--
Is this still something that is needed to be done?
>
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234365#comment-16234365
]
Steve Moist commented on HADOOP-13887:
--
>I can see the appeal of some form of support for this
[
https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-15006:
-
Attachment: S3-CSE Proposal.pdf
> Encrypt S3A data client-side with Hadoop libraries & Hadoop
Steve Moist created HADOOP-15006:
Summary: Encrypt S3A data client-side with Hadoop libraries &
Hadoop KMS
Key: HADOOP-15006
URL: https://issues.apache.org/jira/browse/HADOOP-15006
Project: Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-13887:
-
Summary: Encrypt S3A data client-side with AWS SDK (was: Support for
client-side encryption in
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16219616#comment-16219616
]
Steve Moist commented on HADOOP-13887:
--
[~steve_l] Should we continue discussion on this or table
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109860#comment-16109860
]
Steve Moist edited comment on HADOOP-13887 at 8/1/17 10:00 PM:
---
[~steve_l]
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109860#comment-16109860
]
Steve Moist commented on HADOOP-13887:
--
[~steve_l] I do like object tags to store the OEMI on, but
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096761#comment-16096761
]
Steve Moist commented on HADOOP-13887:
--
I think they can easily co-exist as distinct options. But
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16096558#comment-16096558
]
Steve Moist commented on HADOOP-13887:
--
Hey [~steve_l] and [~Igor Mazur], I've spent some time
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-13887:
-
Attachment: S3-CSE Proposal.pdf
> Support for client-side encryption in S3A file system
>
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16030185#comment-16030185
]
Steve Moist commented on HADOOP-13887:
--
So I've been taking a look at a different approach to S3
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15993019#comment-15993019
]
Steve Moist commented on HADOOP-14209:
--
Fixed all the formatting issues in trunk to be in
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15989151#comment-15989151
]
Steve Moist commented on HADOOP-14209:
--
Minor formatting change in trunk for the validation tests.
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-14209:
-
Status: Patch Available (was: In Progress)
> Remove @Ignore from valid S3a test.
>
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-14209:
-
Attachment: HADOOP-14209-branch-2-001.patch
HADOOP-14209-001.patch
Tested
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist reassigned HADOOP-14209:
Assignee: Steve Moist (was: Sean Mackrory)
> Remove @Ignore from valid S3a test.
>
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HADOOP-14209 started by Steve Moist.
> Remove @Ignore from valid S3a test.
> ---
>
[
https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983638#comment-15983638
]
Steve Moist commented on HADOOP-13887:
--
If you don't mind. I'd like to jump in with some thoughts.
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-14305:
-
Attachment: HADOOP-14305-branch-2-002.patch
> S3A SSE tests won't run in parallel: Bad request
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981459#comment-15981459
]
Steve Moist commented on HADOOP-14305:
--
us-west-2 (Oregon)
> S3A SSE tests won't run in parallel:
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-14305:
-
Status: Patch Available (was: In Progress)
> S3A SSE tests won't run in parallel: Bad request
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15980160#comment-15980160
]
Steve Moist commented on HADOOP-14305:
--
Uploaded a patch for trunk, if everyone is satisfied with
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-14305:
-
Attachment: HADOOP-14305-002.patch
> S3A SSE tests won't run in parallel: Bad request in
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973589#comment-15973589
]
Steve Moist commented on HADOOP-14305:
--
I've made the pom changes, but want to spend some time
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973319#comment-15973319
]
Steve Moist commented on HADOOP-14305:
--
Sounds good. I'll submit a few patches today.
> S3A SSE
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist reassigned HADOOP-14305:
Assignee: Steve Moist
> S3A SSE tests won't run in parallel: Bad request in directory
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HADOOP-14305 started by Steve Moist.
> S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971778#comment-15971778
]
Steve Moist commented on HADOOP-14305:
--
I'm pretty sure it's SSE-C related.
I disabled the SSE-C
[
https://issues.apache.org/jira/browse/HADOOP-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969466#comment-15969466
]
Steve Moist commented on HADOOP-14305:
--
{quote}
Steve Moist : have you been using this feature yet?
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967987#comment-15967987
]
Steve Moist commented on HADOOP-13075:
--
If I remember correctly, I kept getting the runs
Steve Moist created HADOOP-14209:
Summary: Remove @Ignore from valid S3a test.
Key: HADOOP-14209
URL: https://issues.apache.org/jira/browse/HADOOP-14209
Project: Hadoop Common
Issue Type:
[
https://issues.apache.org/jira/browse/HADOOP-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935012#comment-15935012
]
Steve Moist commented on HADOOP-14209:
--
This is in trunk and branch-2 and should be fixed in both
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870245#comment-15870245
]
Steve Moist commented on HADOOP-13075:
--
Is that something that we need to fix?
> Add support for
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-13075:
-
Attachment: HADOOP-13075-branch2.002.patch
HADOOP-13075-003.patch
Branch-2
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856794#comment-15856794
]
Steve Moist commented on HADOOP-13075:
--
So with SSE-KMS, there's 2 cases, one where the user
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854759#comment-15854759
]
Steve Moist commented on HADOOP-13075:
--
[~steve_l] I've asked around on our side and we don't have
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-13075:
-
Attachment: HADOOP-13075-002.patch
Corrected patch. The precondition lines are still too long
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Moist updated HADOOP-13075:
-
Attachment: HADOOP-13075-001.patch
trunk patch
> Add support for SSE-KMS and SSE-C in s3a
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843591#comment-15843591
]
Steve Moist commented on HADOOP-13075:
--
I ran some tests using the minicluster against the default
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840518#comment-15840518
]
Steve Moist commented on HADOOP-13075:
--
Updated with changes. One issue I found with refactoring
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840149#comment-15840149
]
Steve Moist commented on HADOOP-13075:
--
Ok, that's what I originally thought. I will leave the
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840118#comment-15840118
]
Steve Moist commented on HADOOP-13075:
--
[~ste...@apache.org] So you're saying don't change the name
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839021#comment-15839021
]
Steve Moist commented on HADOOP-13075:
--
Updated pull request with [~steve_l]'s comments.
> Add
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838654#comment-15838654
]
Steve Moist commented on HADOOP-13075:
--
[~fabbri] One of Frederico's original changes was removing
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838352#comment-15838352
]
Steve Moist commented on HADOOP-13075:
--
Sounds good, I'll make changes around the Enum and convert
[
https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838104#comment-15838104
]
Steve Moist commented on HADOOP-13075:
--
Hi, I've taken Federico's changes and updated them to be
61 matches
Mail list logo
