[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918676#comment-13918676
]
Colin Patrick McCabe commented on HDFS-6040:
Test failure is HDFS-5997, not related to this patch.
I'm going to add a suppression for the findbugs warning. It's not a new
warning (I'm not sure why it's showing up as new), just a consequence of
findbugs not understanding that the lock is held at the start of the function.
I also fixed an issue where DFSClients without libhadoop.so installed would get
an exception while creating the {{ShortCircuitCache}}, and adjusted some noisy
INFO logs down.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch
In ShortCircuitShnm, clean up some extra imports, fix a findbugs warning, and
throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918689#comment-13918689
]
Aaron T. Myers commented on HDFS-6040:
--
Instead of catching {{UnsupportedOperationException}} when creating the
{{DfsClientShmManager}}, I recommend just not creating it unless
{{NativeCodeLoader#isNativeCodeLoaded}} returns true. I think that's the more
common idiom used throughout the rest of the code base, e.g. in the JNI-based
groups mappings, the compression codecs, the NativeIO classes, etc.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch
* clean up some extra imports in ShortCircuitShm
* fix or suppress bogus findbugs warnings
* throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
* fix an issue where DFSClients got an exception when libhadoop.so was not
available
* adjust some noisy INFO log messages downward
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[ https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918692#comment-13918692 ] Colin Patrick McCabe commented on HDFS-6040: bq. Instead of catching UnsupportedOperationException when creating the DfsClientShmManager, I recommend just not creating it unless NativeCodeLoader#isNativeCodeLoaded returns true. ok. some ShortCircuitShm cleanups - Key: HDFS-6040 URL: https://issues.apache.org/jira/browse/HDFS-6040 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs-client Affects Versions: 2.4.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch, HDFS-6040.003.patch * clean up some extra imports in ShortCircuitShm * fix or suppress bogus findbugs warnings * throw an IOException rather than RuntimeException when the client asks the server for a bogus slot index. * fix an issue where DFSClients got an exception when libhadoop.so was not available * adjust some noisy INFO log messages downward -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918759#comment-13918759
]
Chris Nauroth commented on HDFS-6040:
-
{{DomainSocketWatcher}} also aborts if {{SystemUtils#IS_OS_WINDOWS}} is true.
I think we need a separate check for that, because Windows will have the native
code loaded, but no support for Unix domain sockets.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch,
HDFS-6040.003.patch, HDFS-6040.004.patch
* clean up some extra imports in ShortCircuitShm
* fix or suppress bogus findbugs warnings
* throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
* fix an issue where DFSClients got an exception when libhadoop.so was not
available
* adjust some noisy INFO log messages downward
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918772#comment-13918772
]
Colin Patrick McCabe commented on HDFS-6040:
I will check {{DomainSocketWatcher#loadingFailureReason}} directly so that we
handle Windows / other failure conditions properly.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch,
HDFS-6040.003.patch, HDFS-6040.004.patch, HDFS-6040.005.patch
* clean up some extra imports in ShortCircuitShm
* fix or suppress bogus findbugs warnings
* throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
* fix an issue where DFSClients got an exception when libhadoop.so was not
available
* adjust some noisy INFO log messages downward
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[ https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918883#comment-13918883 ] Andrew Wang commented on HDFS-6040: --- v5 rev LGTM, +1 pending again some ShortCircuitShm cleanups - Key: HDFS-6040 URL: https://issues.apache.org/jira/browse/HDFS-6040 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs-client Affects Versions: 2.4.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch, HDFS-6040.003.patch, HDFS-6040.004.patch, HDFS-6040.005.patch * clean up some extra imports in ShortCircuitShm * fix or suppress bogus findbugs warnings * throw an IOException rather than RuntimeException when the client asks the server for a bogus slot index. * fix an issue where DFSClients got an exception when libhadoop.so was not available * adjust some noisy INFO log messages downward -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[ https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918887#comment-13918887 ] Aaron T. Myers commented on HDFS-6040: -- +1, likewise it looks good to me. some ShortCircuitShm cleanups - Key: HDFS-6040 URL: https://issues.apache.org/jira/browse/HDFS-6040 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs-client Affects Versions: 2.4.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch, HDFS-6040.003.patch, HDFS-6040.004.patch, HDFS-6040.005.patch * clean up some extra imports in ShortCircuitShm * fix or suppress bogus findbugs warnings * throw an IOException rather than RuntimeException when the client asks the server for a bogus slot index. * fix an issue where DFSClients got an exception when libhadoop.so was not available * adjust some noisy INFO log messages downward -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918954#comment-13918954
]
Hadoop QA commented on HDFS-6040:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12632379/HDFS-6040.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6285//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6285//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6285//console
This message is automatically generated.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Blocker
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch,
HDFS-6040.003.patch, HDFS-6040.004.patch, HDFS-6040.005.patch
* clean up some extra imports in ShortCircuitShm
* fix or suppress bogus findbugs warnings
* throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
* fix an issue where DFSClients got an exception when libhadoop.so was not
available
* adjust some noisy INFO log messages downward
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918967#comment-13918967
]
Hadoop QA commented on HDFS-6040:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12632392/HDFS-6040.005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6286//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6286//console
This message is automatically generated.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Blocker
Attachments: HDFS-6040.001.patch, HDFS-6040.002.patch,
HDFS-6040.003.patch, HDFS-6040.004.patch, HDFS-6040.005.patch
* clean up some extra imports in ShortCircuitShm
* fix or suppress bogus findbugs warnings
* throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
* fix an issue where DFSClients got an exception when libhadoop.so was not
available
* adjust some noisy INFO log messages downward
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[ https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918493#comment-13918493 ] Andrew Wang commented on HDFS-6040: --- +1 pending Jenkins some ShortCircuitShm cleanups - Key: HDFS-6040 URL: https://issues.apache.org/jira/browse/HDFS-6040 Project: Hadoop HDFS Issue Type: Sub-task Components: hdfs-client Affects Versions: 2.4.0 Reporter: Colin Patrick McCabe Assignee: Colin Patrick McCabe Attachments: HDFS-6040.001.patch In ShortCircuitShnm, clean up some extra imports, fix a findbugs warning, and throw an IOException rather than RuntimeException when the client asks the server for a bogus slot index. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6040) some ShortCircuitShm cleanups
[
https://issues.apache.org/jira/browse/HDFS-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918638#comment-13918638
]
Hadoop QA commented on HDFS-6040:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12632337/HDFS-6040.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6278//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6278//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6278//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6278//console
This message is automatically generated.
some ShortCircuitShm cleanups
-
Key: HDFS-6040
URL: https://issues.apache.org/jira/browse/HDFS-6040
Project: Hadoop HDFS
Issue Type: Sub-task
Components: hdfs-client
Affects Versions: 2.4.0
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6040.001.patch
In ShortCircuitShnm, clean up some extra imports, fix a findbugs warning, and
throw an IOException rather than RuntimeException when the client asks the
server for a bogus slot index.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
