date:20140716

[jira] [Created] (HADOOP-10849) Implement conf substitution with UGI.current/loginUser

2014-07-16 Thread Gera Shegalov (JIRA)

Gera Shegalov created HADOOP-10849:
--

 Summary: Implement conf substitution with UGI.current/loginUser
 Key: HADOOP-10849
 URL: https://issues.apache.org/jira/browse/HADOOP-10849
 Project: Hadoop Common
  Issue Type: Improvement
  Components: conf
Affects Versions: 2.4.1
Reporter: Gera Shegalov
Assignee: Gera Shegalov


Many path properties and similar in hadoop code base would be easily configured 
if we had substitutions with {{UserGroupInformation#getCurrentUser}}. Currently 
we often use less elegant concatenation code if we want to express currentUser 
as opposed to ${user.name} system property representing the user owning the JVM.

This JIRA proposes the corresponding substitution support for keys 
{{ugi.current.user}} and {{ugi.login.user}}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Updated] (HADOOP-10849) Implement conf substitution with UGI.current/loginUser

2014-07-16 Thread Gera Shegalov (JIRA)


 [ 
https://issues.apache.org/jira/browse/HADOOP-10849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated HADOOP-10849:
---

Attachment: HADOOP-10849.v01.patch

v01 to give an idea.

 Implement conf substitution with UGI.current/loginUser
 --

 Key: HADOOP-10849
 URL: https://issues.apache.org/jira/browse/HADOOP-10849
 Project: Hadoop Common
  Issue Type: Improvement
  Components: conf
Affects Versions: 2.4.1
Reporter: Gera Shegalov
Assignee: Gera Shegalov
 Attachments: HADOOP-10849.v01.patch


 Many path properties and similar in hadoop code base would be easily 
 configured if we had substitutions with 
 {{UserGroupInformation#getCurrentUser}}. Currently we often use less elegant 
 concatenation code if we want to express currentUser as opposed to 
 ${user.name} system property representing the user owning the JVM.
 This JIRA proposes the corresponding substitution support for keys 
 {{ugi.current.user}} and {{ugi.login.user}}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-10720) KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API

2014-07-16 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-10720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063216#comment-14063216
 ] 

Hadoop QA commented on HADOOP-10720:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12655994/HADOOP-10720.11.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 2 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common hadoop-common-project/hadoop-kms:

  org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
  org.apache.hadoop.ipc.TestIPC
  org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4288//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4288//console

This message is automatically generated.

 KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API
 ---

 Key: HADOOP-10720
 URL: https://issues.apache.org/jira/browse/HADOOP-10720
 Project: Hadoop Common
  Issue Type: Improvement
  Components: security
Affects Versions: 3.0.0
Reporter: Alejandro Abdelnur
Assignee: Arun Suresh
 Attachments: COMBO.patch, COMBO.patch, COMBO.patch, COMBO.patch, 
 COMBO.patch, HADOOP-10720.1.patch, HADOOP-10720.10.patch, 
 HADOOP-10720.11.patch, HADOOP-10720.2.patch, HADOOP-10720.3.patch, 
 HADOOP-10720.4.patch, HADOOP-10720.5.patch, HADOOP-10720.6.patch, 
 HADOOP-10720.7.patch, HADOOP-10720.8.patch, HADOOP-10720.9.patch, 
 HADOOP-10720.patch, HADOOP-10720.patch, HADOOP-10720.patch, 
 HADOOP-10720.patch, HADOOP-10720.patch


 KMS client/server should implement support for generating encrypted keys and 
 decrypting them via the REST API being introduced by HADOOP-10719.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-10830) Missing lock in JavaKeyStoreProvider.createCredentialEntry

2014-07-16 Thread Vinayakumar B (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-10830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063236#comment-14063236
 ] 

Vinayakumar B commented on HADOOP-10830:


Hi [~benoyantony], thanks for the patch.

There is no need of double try-catch-finally blocks. existing itself you can 
re-use.
Like following..
{code}  @Override
  public CredentialEntry createCredentialEntry(String alias, char[] credential)
throws IOException {
writeLock.lock();
try {
  if (keyStore.containsAlias(alias) || cache.containsKey(alias)) {
throw new IOException(Credential  + alias +  already exists in 
+ this);
  }
  return innerSetCredential(alias, credential);
} catch (KeyStoreException e) {
  throw new IOException(Problem looking up credential  + alias +  in 
  + this, e);
} finally {
  writeLock.unlock();
}
  }{code}

anyway {{innerSetCredential(..)}} have its own try-catch block. So its fine to 
use one try-catch-finally block itself.

 Missing lock in JavaKeyStoreProvider.createCredentialEntry
 --

 Key: HADOOP-10830
 URL: https://issues.apache.org/jira/browse/HADOOP-10830
 Project: Hadoop Common
  Issue Type: Bug
  Components: security
Reporter: Benoy Antony
Assignee: Benoy Antony
 Attachments: HADOOP-10830.patch


 _JavaKeyStoreProvider_ uses _ReentrantReadWriteLock_  to provide thread 
 safety.
 The {{createCredentialEntry}} should hold _writeLock_ before adding the entry.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-9921) daemon scripts should remove pid file on stop call after stop or process is found not running

2014-07-16 Thread Vinayakumar B (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063238#comment-14063238
 ] 

Vinayakumar B commented on HADOOP-9921:
---

Hi [~umamaheswararao], can you review the patch?

 daemon scripts should remove pid file on stop call after stop or process is 
 found not running
 -

 Key: HADOOP-9921
 URL: https://issues.apache.org/jira/browse/HADOOP-9921
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Vinayakumar B
Assignee: Vinayakumar B
 Attachments: HADOOP-9921.patch


 daemon scripts should remove the pid file on stop call using daemon script.
 Should remove the pid file, even though process is not running.
 same pid file will be used by start command. At that time, if the same pid is 
 assigned to some other process, then start may fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-10131) NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if excluded nodes passed are not part of the cluster tree

2014-07-16 Thread Vinayakumar B (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-10131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063241#comment-14063241
]

Vinayakumar B commented on HADOOP-10131:

Hi [~umamaheswararao], can you review the patch? Thanks In advance

NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if
excluded nodes passed are not part of the cluster tree
-

Key: HADOOP-10131
URL: https://issues.apache.org/jira/browse/HADOOP-10131
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.0.5-alpha
Reporter: Vinayakumar B
Assignee: Vinayakumar B
Attachments: HADOOP-10131.patch, HDFS-5112.patch

I got File /hdfs_COPYING_ could only be replicated to 0 nodes instead of
minReplication (=1). There are 1 datanode(s) running and 1 node(s) are
excluded in this operation. in the following case
1. 2 DNs cluster,
2. One of the datanodes was not responding from last 10 min, but about to
detect as dead at NN.
3. Tried to write one file, for the block NN allocated both DNs.
4. Client While creating the pipeline took some time to detect one node
failure.
5. Before client detects pipeline failure, NN side dead node was removed from
cluster map.
6. Now, client has abandoned previous block and asked for new block with dead
node in excluded list and got above exception even though one more node was
available live.
When I dig this more, found that,
{{NetWorkTopology#countNumOfAvailableNodes()}} is not giving correct count
when the excludeNodes passed from client are not part of the cluster map.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-10794) A hadoop cluster needs clock synchronization

2014-07-16 Thread Zhijie Shen (JIRA)

[
https://issues.apache.org/jira/browse/HADOOP-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063270#comment-14063270
]

Zhijie Shen commented on HADOOP-10794:
--

I'm afraid each component may more or less have made the assumption that clock
is synchronized across the cluster. Unfortunately, the problem that made us be
aware of this issue came from MR: an elapsed time that based the the diff of
the timestamps on two hosts became negative. Having a try with a cluster whose
hosts are completely out of sync in terms of clock may give us the idea how bad
the situation is.

According to the prior discussion, it seems to be good to have a mechanism to
check the clocks across a cluster, report unacceptable asynchronization,
doesn't it? On the other hand, it is arguable whether this mechanism should be
part of Hadoop or other cluster monitoring tools.

Anyway, IMHO, it's good to at least let users be aware of the clock
synchronization assumption (via jira, mailing list or etc.), and the risks of
being out of sync, such that they can be more careful about this factor when
setting up a cluster.

A hadoop cluster needs clock synchronization

Key: HADOOP-10794
URL: https://issues.apache.org/jira/browse/HADOOP-10794
Project: Hadoop Common
Issue Type: Bug
Reporter: Zhijie Shen

As a distributed system, a hadoop cluster wants the clock on all the
participating hosts synchronized. Otherwise, some problems might happen. For
example, in YARN-2251, due to the clock on the host for the task container
falls behind that on the host of the AM container, the computed elapsed time
(the diff between the timestamps produced on two hosts) becomes negative.
In YARN-2251, we tried to mask the negative elapsed time. However, we should
seek for a decent long term solution, such as providing mechanism to do and
check clock synchronization.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-9921) daemon scripts should remove pid file on stop call after stop or process is found not running

2014-07-16 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063275#comment-14063275
 ] 

Hadoop QA commented on HADOOP-9921:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12600746/HADOOP-9921.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common:

  org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
  org.apache.hadoop.ipc.TestIPC
  org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4289//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4289//console

This message is automatically generated.

 daemon scripts should remove pid file on stop call after stop or process is 
 found not running
 -

 Key: HADOOP-9921
 URL: https://issues.apache.org/jira/browse/HADOOP-9921
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Vinayakumar B
Assignee: Vinayakumar B
 Attachments: HADOOP-9921.patch


 daemon scripts should remove the pid file on stop call using daemon script.
 Should remove the pid file, even though process is not running.
 same pid file will be used by start command. At that time, if the same pid is 
 assigned to some other process, then start may fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HADOOP-10131) NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if excluded nodes passed are not part of the cluster tree

2014-07-16 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HADOOP-10131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063280#comment-14063280
 ] 

Hadoop QA commented on HADOOP-10131:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12616044/HADOOP-10131.patch
  against trunk revision .

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common:

  org.apache.hadoop.ipc.TestIPC
  org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem
  org.apache.hadoop.fs.TestSymlinkLocalFSFileContext
  org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4290//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-HADOOP-Build/4290//console

This message is automatically generated.

 NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if 
 excluded nodes passed are not part of the cluster tree
 -

 Key: HADOOP-10131
 URL: https://issues.apache.org/jira/browse/HADOOP-10131
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 3.0.0, 2.0.5-alpha
Reporter: Vinayakumar B
Assignee: Vinayakumar B
 Attachments: HADOOP-10131.patch, HDFS-5112.patch


 I got File /hdfs_COPYING_ could only be replicated to 0 nodes instead of 
 minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are 
 excluded in this operation. in the following case
 1. 2 DNs cluster,
 2. One of the datanodes was not responding from last 10 min, but about to 
 detect as dead at NN.
 3. Tried to write one file, for the block NN allocated both DNs.
 4. Client While creating the pipeline took some time to detect one node 
 failure.
 5. Before client detects pipeline failure, NN side dead node was removed from 
 cluster map.
 6. Now, client has abandoned previous block and asked for new block with dead 
 node in excluded list and got above exception even though one more node was 
 available live.
 When I dig this more, found that,
 {{NetWorkTopology#countNumOfAvailableNodes()}} is not giving correct count 
 when the excludeNodes passed from client are not part of the cluster map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

1 2 >

1 - 100 of 115 matches

Mail list logo