[jira] [Commented] (HADOOP-11660) Add support for hardware crc of HDFS checksums on ARM aarch64 architecture
[ https://issues.apache.org/jira/browse/HADOOP-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584776#comment-14584776 ] Edward Nevill commented on HADOOP-11660: Hi Andrew, Yes. Please see my comments of 4/Mar/15. The version of the CRC patch checked into Hadoop does support pipelining. On A57 without pipelining the raw CRC speed was 4.5X better. With pipelining it was 11X better. Regards, Ed. > Add support for hardware crc of HDFS checksums on ARM aarch64 architecture > -- > > Key: HADOOP-11660 > URL: https://issues.apache.org/jira/browse/HADOOP-11660 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.8.0 > Environment: ARM aarch64 development platform >Reporter: Edward Nevill >Assignee: Edward Nevill >Priority: Minor > Labels: performance > Fix For: 2.8.0 > > Attachments: jira-11660.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > This patch adds support for hardware crc for ARM's new 64 bit architecture > The patch is completely conditionalized on __aarch64__ > I have only added support for the non pipelined version as I benchmarked the > pipelined version on aarch64 and it showed no performance improvement. > The aarch64 version supports both Castagnoli and Zlib CRCs as both of these > are supported on ARM aarch64 hardwre. > To benchmark this I modified the test_bulk_crc32 test to print out the time > taken to CRC a 1MB dataset 1000 times. > Before: > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 2.55 > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 2.55 > After: > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 0.57 > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 0.57 > So this represents a 5X performance improvement on raw CRC calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12050) Enable MaxInactiveInterval for hadoop http auth token
[ https://issues.apache.org/jira/browse/HADOOP-12050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hzlu updated HADOOP-12050: -- Attachment: Enable-MaxInactiveInterval-for-hadoop-http-auth-toke.patch This is the correct one, combining HADOOP-12049 & HADOOP-12050. > Enable MaxInactiveInterval for hadoop http auth token > - > > Key: HADOOP-12050 > URL: https://issues.apache.org/jira/browse/HADOOP-12050 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Benoy Antony >Assignee: hzlu > Attachments: > Enable-MaxInactiveInterval-for-hadoop-http-auth-toke.patch > > > During http authentication, a cookie which contains the authentication token > is dropped. The expiry time of the authentication token can be configured via > hadoop.http.authentication.token.validity. The default value is 10 hours. > For clusters which require enhanced security, it is desirable to have a > configurable MaxInActiveInterval for the authentication token. If there is no > activity during MaxInActiveInterval, the authentication token will be > invalidated. > The MaxInActiveInterval will be less than > hadoop.http.authentication.token.validity. The default value will be 30 > minutes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12050) Enable MaxInactiveInterval for hadoop http auth token
[ https://issues.apache.org/jira/browse/HADOOP-12050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hzlu updated HADOOP-12050: -- Attachment: (was: 0002-Enable-MaxInactiveInterval-for-hadoop-http-auth-toke.patch) > Enable MaxInactiveInterval for hadoop http auth token > - > > Key: HADOOP-12050 > URL: https://issues.apache.org/jira/browse/HADOOP-12050 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Benoy Antony >Assignee: hzlu > > During http authentication, a cookie which contains the authentication token > is dropped. The expiry time of the authentication token can be configured via > hadoop.http.authentication.token.validity. The default value is 10 hours. > For clusters which require enhanced security, it is desirable to have a > configurable MaxInActiveInterval for the authentication token. If there is no > activity during MaxInActiveInterval, the authentication token will be > invalidated. > The MaxInActiveInterval will be less than > hadoop.http.authentication.token.validity. The default value will be 30 > minutes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-9565) Add a Blobstore interface to add to blobstore FileSystems
[
https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584724#comment-14584724
]
Chen He commented on HADOOP-9565:
-
Thank you for the explanation, [~ste...@apache.org]. You are right, the
._COPYING_ is added by CLI (distcp refers to this also) and hardcoded there.
IMHO, it may be more flexible if we can choose to not add ._COPYING_ by setting
a parameter like "OBJECTSTORE_NO_RENAME_IN_COPY".
> Add a Blobstore interface to add to blobstore FileSystems
> -
>
> Key: HADOOP-9565
> URL: https://issues.apache.org/jira/browse/HADOOP-9565
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3, fs/swift
>Affects Versions: 2.6.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Labels: BB2015-05-TBR
> Attachments: HADOOP-9565-001.patch, HADOOP-9565-002.patch,
> HADOOP-9565-003.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really
> blobstores, with different atomicity and consistency guarantees, by adding a
> {{Blobstore}} interface to add to them.
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that
> all blobstores implement at server-side copy operation as a substitute for
> rename.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584659#comment-14584659
]
Hudson commented on HADOOP-11971:
-
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2173 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2173/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584649#comment-14584649
]
Hudson commented on HADOOP-11971:
-
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #225 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/225/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12088) KMSClientProvider uses equalsIgnoreCase("application/json")
[
https://issues.apache.org/jira/browse/HADOOP-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584638#comment-14584638
]
Steve Loughran commented on HADOOP-12088:
-
yes to starts with, but the case conversion may also want to go
{{toLower(EN_US)}} (oh, and handle a null content-type), which the first check
does
> KMSClientProvider uses equalsIgnoreCase("application/json")
> ---
>
> Key: HADOOP-12088
> URL: https://issues.apache.org/jira/browse/HADOOP-12088
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.7.1
>Reporter: Steve Loughran
>Assignee: Brahma Reddy Battula
>
> {{KMSClientProvider.call()}} validates the content type returned by going
> {code}
> "application/json".equalsIgnoreCase(conn.getContentType())
> {code}
> that's not going to work in all locales, not if upper case is being returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12088) KMSClientProvider uses equalsIgnoreCase("application/json")
[
https://issues.apache.org/jira/browse/HADOOP-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated HADOOP-12088:
--
Priority: Major (was: Minor)
> KMSClientProvider uses equalsIgnoreCase("application/json")
> ---
>
> Key: HADOOP-12088
> URL: https://issues.apache.org/jira/browse/HADOOP-12088
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.7.1
>Reporter: Steve Loughran
>Assignee: Brahma Reddy Battula
>
> {{KMSClientProvider.call()}} validates the content type returned by going
> {code}
> "application/json".equalsIgnoreCase(conn.getContentType())
> {code}
> that's not going to work in all locales, not if upper case is being returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584626#comment-14584626
]
Hudson commented on HADOOP-11971:
-
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #216 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/216/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584618#comment-14584618
]
Hudson commented on HADOOP-11971:
-
SUCCESS: Integrated in Hadoop-Hdfs-trunk #2155 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2155/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11660) Add support for hardware crc of HDFS checksums on ARM aarch64 architecture
[ https://issues.apache.org/jira/browse/HADOOP-11660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584577#comment-14584577 ] Andrew Pinski commented on HADOOP-11660: > On aarch64 a crc32 takes 3 exec cycles, but it only has one crc unit, so if > we did the same on aarch64 and had say > crc32 w0, w0, x3 > crc32 w1, w1, x4 > crc32 w2, w2, x5 > The second crc32 w1, w1, x4 would have to wait for the 1st crc to complete > and the 3rd would have to wait for the 2nd, taking 9 cycles in any case, so > there is no benefit to pipelining. This is not true on some AARCH64 processors. For ThunderX, this is definitely not true. crc32 (32bits) is fully pipelined and the next one will issue right away. Though the latency of those instructions are 4 cycles long. So this will only take 6 cycles on ThunderX. Thanks, Andrew Pinski > Add support for hardware crc of HDFS checksums on ARM aarch64 architecture > -- > > Key: HADOOP-11660 > URL: https://issues.apache.org/jira/browse/HADOOP-11660 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.8.0 > Environment: ARM aarch64 development platform >Reporter: Edward Nevill >Assignee: Edward Nevill >Priority: Minor > Labels: performance > Fix For: 2.8.0 > > Attachments: jira-11660.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > This patch adds support for hardware crc for ARM's new 64 bit architecture > The patch is completely conditionalized on __aarch64__ > I have only added support for the non pipelined version as I benchmarked the > pipelined version on aarch64 and it showed no performance improvement. > The aarch64 version supports both Castagnoli and Zlib CRCs as both of these > are supported on ARM aarch64 hardwre. > To benchmark this I modified the test_bulk_crc32 test to print out the time > taken to CRC a 1MB dataset 1000 times. > Before: > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 2.55 > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 2.55 > After: > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 0.57 > CRC 1048576 bytes @ 512 bytes per checksum X 1000 iterations = 0.57 > So this represents a 5X performance improvement on raw CRC calculation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584561#comment-14584561
]
Hudson commented on HADOOP-11971:
-
FAILURE: Integrated in Hadoop-Yarn-trunk #957 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/957/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11971) Move test utilities for tracing from hadoop-hdfs to hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-11971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584556#comment-14584556
]
Hudson commented on HADOOP-11971:
-
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #227 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/227/])
HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to
hadoop-common. Contributed by Masatake Iwasaki. (aajisaka: rev
e4489d97e5f0cec601b93e1b883fa194a7353229)
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/SetSpanReceiver.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracing.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/tracing/TestTracingShortCircuitLocalRead.java
> Move test utilities for tracing from hadoop-hdfs to hadoop-common
> -
>
> Key: HADOOP-11971
> URL: https://issues.apache.org/jira/browse/HADOOP-11971
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tracing
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Fix For: 2.8.0
>
> Attachments: HADOOP-11971.001.patch
>
>
> Utilities used in TestTracing such as SetSpanReceiver should be moved to
> {{hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/tracing/}}
> in order to make it usable from yarn and other modules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12049) Control http authentication cookie persistence via configuration
[
https://issues.apache.org/jira/browse/HADOOP-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584524#comment-14584524
]
Hadoop QA commented on HADOOP-12049:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 15s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 29s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 55s | The applied patch generated 4
new checkstyle issues (total was 44, now 48). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 33s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | common tests | 5m 19s | Tests passed in
hadoop-auth. |
| {color:red}-1{color} | common tests | 37m 19s | Tests failed in
hadoop-common. |
| | | 83m 24s | |
\\
\\
|| Reason || Tests ||
| Timed out tests | org.apache.hadoop.http.TestAuthenticationSessionCookie |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739405/0001-Control-http-authentication-cookie-persistence.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / eef7b50 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/artifact/patchprocess/diffcheckstylehadoop-auth.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/artifact/patchprocess/whitespace.txt
|
| hadoop-auth test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/artifact/patchprocess/testrun_hadoop-auth.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/artifact/patchprocess/testrun_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/6962/console |
This message was automatically generated.
> Control http authentication cookie persistence via configuration
>
>
> Key: HADOOP-12049
> URL: https://issues.apache.org/jira/browse/HADOOP-12049
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.4.1
>Reporter: Benoy Antony
>Assignee: hzlu
> Labels: patch
> Fix For: 2.4.1
>
> Attachments: 0001-Control-http-authentication-cookie-persistence.patch
>
>
> During http authentication, a cookie is dropped. This is a persistent cookie.
> The cookie is valid across browser sessions.
> For clusters which require enhanced security, it is desirable to have a
> session cookie so that cookie gets deleted when the user closes browser
> session.
> It should be possible to specify cookie persistence (session or persistent)
> via configuration
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-12088) KMSClientProvider uses equalsIgnoreCase("application/json")
[
https://issues.apache.org/jira/browse/HADOOP-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584523#comment-14584523
]
Brahma Reddy Battula commented on HADOOP-12088:
---
As per [RFC|http://www.ietf.org/rfc/rfc4627.txt] specification type and sub
type can be followed by parameters separated by ";"
So, I think its sufficient to check as
{{conn.getContentType().startsWIth("application/json")}}
[~ste...@apache.org] please let me know your opinion.
> KMSClientProvider uses equalsIgnoreCase("application/json")
> ---
>
> Key: HADOOP-12088
> URL: https://issues.apache.org/jira/browse/HADOOP-12088
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.7.1
>Reporter: Steve Loughran
>Assignee: Brahma Reddy Battula
>Priority: Minor
>
> {{KMSClientProvider.call()}} validates the content type returned by going
> {code}
> "application/json".equalsIgnoreCase(conn.getContentType())
> {code}
> that's not going to work in all locales, not if upper case is being returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-9565) Add a Blobstore interface to add to blobstore FileSystems
[
https://issues.apache.org/jira/browse/HADOOP-9565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584501#comment-14584501
]
Steve Loughran commented on HADOOP-9565:
I don't know about {{.COPYING}}. that's not done in the swiftfs itself, it's
distcp/CLI
rename: same thing.
distcp assumes that directory rename is atomic and fast, so does a copy with
the suffix COPYING so that the (slow) upload can take place bit by bit. When
the upload is finished it does a quick rename() , only in swift and s3 that
rename is in fact a second copy.
the fix here is for distcp & fs cli to recognise object stores and act
differently, or adding an explicit 'don't rename" option for all object stores.
> Add a Blobstore interface to add to blobstore FileSystems
> -
>
> Key: HADOOP-9565
> URL: https://issues.apache.org/jira/browse/HADOOP-9565
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3, fs/swift
>Affects Versions: 2.6.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Labels: BB2015-05-TBR
> Attachments: HADOOP-9565-001.patch, HADOOP-9565-002.patch,
> HADOOP-9565-003.patch
>
>
> We can make the fact that some {{FileSystem}} implementations are really
> blobstores, with different atomicity and consistency guarantees, by adding a
> {{Blobstore}} interface to add to them.
> This could also be a place to add a {{Copy(Path,Path)}} method, assuming that
> all blobstores implement at server-side copy operation as a substitute for
> rename.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HADOOP-12050) Enable MaxInactiveInterval for hadoop http auth token
[ https://issues.apache.org/jira/browse/HADOOP-12050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hzlu updated HADOOP-12050: -- Attachment: 0002-Enable-MaxInactiveInterval-for-hadoop-http-auth-toke.patch Patch for Enable-MaxInactiveInterval-for-hadoop-http-auth-toke > Enable MaxInactiveInterval for hadoop http auth token > - > > Key: HADOOP-12050 > URL: https://issues.apache.org/jira/browse/HADOOP-12050 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Reporter: Benoy Antony >Assignee: hzlu > Attachments: > 0002-Enable-MaxInactiveInterval-for-hadoop-http-auth-toke.patch > > > During http authentication, a cookie which contains the authentication token > is dropped. The expiry time of the authentication token can be configured via > hadoop.http.authentication.token.validity. The default value is 10 hours. > For clusters which require enhanced security, it is desirable to have a > configurable MaxInActiveInterval for the authentication token. If there is no > activity during MaxInActiveInterval, the authentication token will be > invalidated. > The MaxInActiveInterval will be less than > hadoop.http.authentication.token.validity. The default value will be 30 > minutes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12049) Control http authentication cookie persistence via configuration
[ https://issues.apache.org/jira/browse/HADOOP-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hzlu updated HADOOP-12049: -- Attachment: 0001-Control-http-authentication-cookie-persistence.patch A patch for HADOOP-12049 > Control http authentication cookie persistence via configuration > > > Key: HADOOP-12049 > URL: https://issues.apache.org/jira/browse/HADOOP-12049 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.4.1 >Reporter: Benoy Antony >Assignee: hzlu > Labels: patch > Fix For: 2.4.1 > > Attachments: 0001-Control-http-authentication-cookie-persistence.patch > > > During http authentication, a cookie is dropped. This is a persistent cookie. > The cookie is valid across browser sessions. > For clusters which require enhanced security, it is desirable to have a > session cookie so that cookie gets deleted when the user closes browser > session. > It should be possible to specify cookie persistence (session or persistent) > via configuration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HADOOP-12049) Control http authentication cookie persistence via configuration
[ https://issues.apache.org/jira/browse/HADOOP-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hzlu updated HADOOP-12049: -- Fix Version/s: 2.4.1 Labels: patch (was: ) Target Version/s: 2.4.1 Affects Version/s: 2.4.1 Status: Patch Available (was: Open) > Control http authentication cookie persistence via configuration > > > Key: HADOOP-12049 > URL: https://issues.apache.org/jira/browse/HADOOP-12049 > Project: Hadoop Common > Issue Type: Improvement > Components: security >Affects Versions: 2.4.1 >Reporter: Benoy Antony >Assignee: hzlu > Labels: patch > Fix For: 2.4.1 > > > During http authentication, a cookie is dropped. This is a persistent cookie. > The cookie is valid across browser sessions. > For clusters which require enhanced security, it is desirable to have a > session cookie so that cookie gets deleted when the user closes browser > session. > It should be possible to specify cookie persistence (session or persistent) > via configuration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
