[jira] [Commented] (HADOOP-14428) s3a: mkdir appears to be broken
[ https://issues.apache.org/jira/browse/HADOOP-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013557#comment-16013557 ] Mingliang Liu commented on HADOOP-14428: I'll test this tomorrow. I've no idea how the unit tests fools me. This is a blocker for sure. Thanks, > s3a: mkdir appears to be broken > --- > > Key: HADOOP-14428 > URL: https://issues.apache.org/jira/browse/HADOOP-14428 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Affects Versions: 3.0.0-alpha2, HADOOP-13345 >Reporter: Aaron Fabbri >Priority: Blocker > > Reproduction is: > hadoop fs -mkdir s3a://my-bucket/dir/ > hadoop fs -ls s3a://my-bucket/dir/ > ls: `s3a://my-bucket/dir/': No such file or directory > I believe this is a regression from HADOOP-14255. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013483#comment-16013483
]
Hadoop QA commented on HADOOP-14430:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
25s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 34s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 25s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14430 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868452/HADOOP-14430-001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 270c97e8b372 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / d87a63a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12341/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12341/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12341/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12341/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> the accessTime of FileStatus got by
[jira] [Commented] (HADOOP-14416) Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013481#comment-16013481 ] Sivaguru Sankaridurg commented on HADOOP-14416: --- Thanks [~liuml07] > Path starting with 'wasb:///' not resolved correctly while authorizing with > WASB-Ranger > --- > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt, > SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013468#comment-16013468
]
Hadoop QA commented on HADOOP-14429:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
22s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
24s{color} | {color:green} hadoop-common-project/hadoop-common generated 0 new
+ 18 unchanged - 1 fixed = 18 total (was 19) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
30s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14429 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868449/HADOOP-14429-001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux b031b5318334 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 18c494a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12340/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12340/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12340/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL:
[jira] [Commented] (HADOOP-14412) HostsFileReader#getHostDetails is very expensive on large clusters
[ https://issues.apache.org/jira/browse/HADOOP-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013467#comment-16013467 ] Hudson commented on HADOOP-14412: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11741 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11741/]) HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large (rohithsharmaks: rev d87a63a9019d74a1c338c724e050952843a153e5) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java * (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestHostsFileReader.java * (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/HostsFileReader.java > HostsFileReader#getHostDetails is very expensive on large clusters > -- > > Key: HADOOP-14412 > URL: https://issues.apache.org/jira/browse/HADOOP-14412 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Fix For: 2.9.0, 3.0.0-alpha3, 2.8.2 > > Attachments: HADOOP-14412.001.patch, HADOOP-14412.002.patch, > HADOOP-14412-branch-2.001.patch, HADOOP-14412-branch-2.002.patch, > HADOOP-14412-branch-2.002.patch, HADOOP-14412-branch-2.8.002.patch > > > After upgrading one of our large clusters to 2.8 we noticed many IPC server > threads of the resourcemanager spending time in NodesListManager#isValidNode > which in turn was calling HostsFileReader#getHostDetails. The latter is > creating complete copies of the include and exclude sets for every node > heartbeat, and these sets are not small due to the size of the cluster. > These copies are causing multiple resizes of the underlying HashSets being > filled and creating lots of garbage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14379) In federation mode,"hdfs dfsadmin -report" command can not be used
[ https://issues.apache.org/jira/browse/HADOOP-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013448#comment-16013448 ] lixinglong commented on HADOOP-14379: - [~chris.douglas] [~ajisakaa] Thanks for your comments ！ > In federation mode,"hdfs dfsadmin -report" command can not be used > -- > > Key: HADOOP-14379 > URL: https://issues.apache.org/jira/browse/HADOOP-14379 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0-alpha2 >Reporter: lixinglong >Assignee: lixinglong > Attachments: HADOOP-14379.001.patch, HADOOP-14379.002.patch, > HADOOP-14379.003.patch, HADOOP-14379.004.patch, HADOOP-14379.005.patch, > HADOOP-14379.006.patch > > > In federation mode,"hdfs dfsadmin -report" command can not be used,as follows: > hdfs@zdh102:~> hdfs dfsadmin -report > report: FileSystem viewfs://nsX/ is not an HDFS file system > Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning] > After adding new features,"hdfs dfsadmin -report" command can be used,as > follows: > hdfs@zdh102:~> hdfs dfsadmin -report > hdfs://nameservice > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.103:50010 (zdh103) > Hostname: zdh103 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68901670912 (64.17 GB) > DFS Remaining: 193527250944 (180.24 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.74% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:42 CST 2017 > Name: 10.43.183.104:50010 (zdh104) > Hostname: zdh104 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68800688128 (64.08 GB) > DFS Remaining: 193628233728 (180.33 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.78% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > hdfs://nameservice1 > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 12:36:06 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% >
[jira] [Updated] (HADOOP-14412) HostsFileReader#getHostDetails is very expensive on large clusters
[ https://issues.apache.org/jira/browse/HADOOP-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated HADOOP-14412: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.2 3.0.0-alpha3 2.9.0 Status: Resolved (was: Patch Available) committed to trunk/branch-2/branch-2.8. thanks Jason for the patch! > HostsFileReader#getHostDetails is very expensive on large clusters > -- > > Key: HADOOP-14412 > URL: https://issues.apache.org/jira/browse/HADOOP-14412 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Fix For: 2.9.0, 3.0.0-alpha3, 2.8.2 > > Attachments: HADOOP-14412.001.patch, HADOOP-14412.002.patch, > HADOOP-14412-branch-2.001.patch, HADOOP-14412-branch-2.002.patch, > HADOOP-14412-branch-2.002.patch, HADOOP-14412-branch-2.8.002.patch > > > After upgrading one of our large clusters to 2.8 we noticed many IPC server > threads of the resourcemanager spending time in NodesListManager#isValidNode > which in turn was calling HostsFileReader#getHostDetails. The latter is > creating complete copies of the include and exclude sets for every node > heartbeat, and these sets are not small due to the size of the cluster. > These copies are causing multiple resizes of the underlying HashSets being > filled and creating lots of garbage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Description:
{code}
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
{code}
from code above, we can see that the getFsAction method doesnot modify the
action generated by FsAction action = FsAction.NONE,which means it return
FsAction.NONE all the time;
was:
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE,which means it return FsAction.NONE all the time;
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
>
> {code}
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> {code}
> from code above, we can see that the getFsAction method doesnot modify the
> action generated by FsAction action = FsAction.NONE,which means it return
> FsAction.NONE all the time;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Description:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
{{long accessTime = 0}} in code below;
{code} private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
} {code}
was:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
{long accessTime = 0} in code below;
{code} private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
} {code}
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14430-001.patch
>
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> {{long accessTime = 0}} in code below;
> {code} private FileStatus getFileStatus(ChannelSftp channel, LsEntry
> sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> } {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Description:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
{long accessTime = 0} in code below;
{code} private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
} {code}
was:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
}
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14430-001.patch
>
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> {long accessTime = 0} in code below;
> {code} private FileStatus getFileStatus(ChannelSftp channel, LsEntry
> sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> } {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14412) HostsFileReader#getHostDetails is very expensive on large clusters
[ https://issues.apache.org/jira/browse/HADOOP-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013429#comment-16013429 ] Rohith Sharma K S commented on HADOOP-14412: committing shortly > HostsFileReader#getHostDetails is very expensive on large clusters > -- > > Key: HADOOP-14412 > URL: https://issues.apache.org/jira/browse/HADOOP-14412 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: HADOOP-14412.001.patch, HADOOP-14412.002.patch, > HADOOP-14412-branch-2.001.patch, HADOOP-14412-branch-2.002.patch, > HADOOP-14412-branch-2.002.patch, HADOOP-14412-branch-2.8.002.patch > > > After upgrading one of our large clusters to 2.8 we noticed many IPC server > threads of the resourcemanager spending time in NodesListManager#isValidNode > which in turn was calling HostsFileReader#getHostDetails. The latter is > creating complete copies of the include and exclude sets for every node > heartbeat, and these sets are not small due to the size of the cluster. > These copies are causing multiple resizes of the underlying HashSets being > filled and creating lots of garbage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Comment: was deleted
(was: get access time by SftpATTRS getATime method)
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14430-001.patch
>
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Attachment: HADOOP-14430-001.patch
get access time by SftpATTRS getATime method
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14430-001.patch
>
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Status: Patch Available (was: Open)
get access time by SftpATTRS getATime method
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14430-001.patch
>
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
[
https://issues.apache.org/jira/browse/HADOOP-14430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14430:
-
Description:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
}
was:
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
}
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> -
>
> Key: HADOOP-14430
> URL: https://issues.apache.org/jira/browse/HADOOP-14430
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
>
> the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
> always 0
> private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
> Path parentPath) throws IOException {
> SftpATTRS attr = sftpFile.getAttrs();
>……
> long modTime = attr.getMTime() * 1000; // convert to milliseconds
> long accessTime = 0;
> ……
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Component/s: fs
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE,which means it return FsAction.NONE all the time;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14430) the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is always 0
Hongyuan Li created HADOOP-14430:
Summary: the accessTime of FileStatus got by SFTPFileSystem's
getFileStatus method is always 0
Key: HADOOP-14430
URL: https://issues.apache.org/jira/browse/HADOOP-14430
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 3.0.0-alpha2
Reporter: Hongyuan Li
Assignee: Hongyuan Li
Priority: Trivial
the accessTime of FileStatus got by SFTPFileSystem's getFileStatus method is
always 0
private FileStatus getFileStatus(ChannelSftp channel, LsEntry sftpFile,
Path parentPath) throws IOException {
SftpATTRS attr = sftpFile.getAttrs();
……
long modTime = attr.getMTime() * 1000; // convert to milliseconds
long accessTime = 0;
……
}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14379) In federation mode,"hdfs dfsadmin -report" command can not be used
[
https://issues.apache.org/jira/browse/HADOOP-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013421#comment-16013421
]
Hadoop QA commented on HADOOP-14379:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
45s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
47s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 47s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
47s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
23s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 43s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14379 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868443/HADOOP-14379.006.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a1390a2b9ca7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 18c494a |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12339/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12339/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12339/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12339/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12339/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Description:
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE,which means it return FsAction.NONE all the time;
was:
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE;
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE,which means it return FsAction.NONE all the time;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Affects Version/s: 3.0.0-alpha2
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Status: Patch Available (was: Open)
modify FTPFileSystem getFsAction method to get correct FsAction
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Attachment: HADOOP-14429-001.patch
upload patch
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Attachment: (was: HADOOP-14429-001.patch)
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Attachment: HADOOP-14429-001.patch
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
> Attachments: HADOOP-14429-001.patch
>
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14429) FTPFileSystem getFsAction method does not work
[
https://issues.apache.org/jira/browse/HADOOP-14429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14429:
-
Description:
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE;
was:
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE;
> FTPFileSystem getFsAction method does not work
> --
>
> Key: HADOOP-14429
> URL: https://issues.apache.org/jira/browse/HADOOP-14429
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
>Priority: Trivial
>
> private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
> FsAction action = FsAction.NONE;
> if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
> action.or(FsAction.READ);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
> action.or(FsAction.WRITE);
> }
> if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
> action.or(FsAction.EXECUTE);
> }
> return action;
> }
> This method doesnot modify the action generated by FsAction action =
> FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14429) FTPFileSystem getFsAction method does not work
Hongyuan Li created HADOOP-14429:
Summary: FTPFileSystem getFsAction method does not work
Key: HADOOP-14429
URL: https://issues.apache.org/jira/browse/HADOOP-14429
Project: Hadoop Common
Issue Type: Bug
Reporter: Hongyuan Li
Assignee: Hongyuan Li
Priority: Trivial
private FsAction getFsAction(int accessGroup, FTPFile ftpFile) {
FsAction action = FsAction.NONE;
if (ftpFile.hasPermission(accessGroup, FTPFile.READ_PERMISSION)) {
action.or(FsAction.READ);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.WRITE_PERMISSION)) {
action.or(FsAction.WRITE);
}
if (ftpFile.hasPermission(accessGroup, FTPFile.EXECUTE_PERMISSION)) {
action.or(FsAction.EXECUTE);
}
return action;
}
This method doesnot modify the action generated by FsAction action =
FsAction.NONE;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14379) In federation mode,"hdfs dfsadmin -report" command can not be used
[ https://issues.apache.org/jira/browse/HADOOP-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lixinglong updated HADOOP-14379: Attachment: HADOOP-14379.006.patch > In federation mode,"hdfs dfsadmin -report" command can not be used > -- > > Key: HADOOP-14379 > URL: https://issues.apache.org/jira/browse/HADOOP-14379 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0-alpha2 >Reporter: lixinglong >Assignee: lixinglong > Attachments: HADOOP-14379.001.patch, HADOOP-14379.002.patch, > HADOOP-14379.003.patch, HADOOP-14379.004.patch, HADOOP-14379.005.patch, > HADOOP-14379.006.patch > > > In federation mode,"hdfs dfsadmin -report" command can not be used,as follows: > hdfs@zdh102:~> hdfs dfsadmin -report > report: FileSystem viewfs://nsX/ is not an HDFS file system > Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning] > After adding new features,"hdfs dfsadmin -report" command can be used,as > follows: > hdfs@zdh102:~> hdfs dfsadmin -report > hdfs://nameservice > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.103:50010 (zdh103) > Hostname: zdh103 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68901670912 (64.17 GB) > DFS Remaining: 193527250944 (180.24 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.74% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:42 CST 2017 > Name: 10.43.183.104:50010 (zdh104) > Hostname: zdh104 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68800688128 (64.08 GB) > DFS Remaining: 193628233728 (180.33 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.78% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > hdfs://nameservice1 > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 12:36:06 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu
[jira] [Commented] (HADOOP-14428) s3a: mkdir appears to be broken
[
https://issues.apache.org/jira/browse/HADOOP-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013356#comment-16013356
]
Aaron Fabbri commented on HADOOP-14428:
---
Just confirming after:
{code}
git revert b053fdc547ddbb6322674142a14010683006d123
{code}
I no longer get the "No such file or directory"
> s3a: mkdir appears to be broken
> ---
>
> Key: HADOOP-14428
> URL: https://issues.apache.org/jira/browse/HADOOP-14428
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.0.0-alpha2, HADOOP-13345
>Reporter: Aaron Fabbri
>Priority: Blocker
>
> Reproduction is:
> hadoop fs -mkdir s3a://my-bucket/dir/
> hadoop fs -ls s3a://my-bucket/dir/
> ls: `s3a://my-bucket/dir/': No such file or directory
> I believe this is a regression from HADOOP-14255.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14428) s3a: mkdir appears to be broken
[ https://issues.apache.org/jira/browse/HADOOP-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013357#comment-16013357 ] Aaron Fabbri commented on HADOOP-14428: --- FYI [~ste...@apache.org] [~liuml07]. Let me know if you guys can reproduce it easily. > s3a: mkdir appears to be broken > --- > > Key: HADOOP-14428 > URL: https://issues.apache.org/jira/browse/HADOOP-14428 > Project: Hadoop Common > Issue Type: Bug > Components: fs/s3 >Affects Versions: 3.0.0-alpha2, HADOOP-13345 >Reporter: Aaron Fabbri >Priority: Blocker > > Reproduction is: > hadoop fs -mkdir s3a://my-bucket/dir/ > hadoop fs -ls s3a://my-bucket/dir/ > ls: `s3a://my-bucket/dir/': No such file or directory > I believe this is a regression from HADOOP-14255. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14428) s3a: mkdir appears to be broken
Aaron Fabbri created HADOOP-14428: - Summary: s3a: mkdir appears to be broken Key: HADOOP-14428 URL: https://issues.apache.org/jira/browse/HADOOP-14428 Project: Hadoop Common Issue Type: Bug Components: fs/s3 Affects Versions: 3.0.0-alpha2, HADOOP-13345 Reporter: Aaron Fabbri Priority: Blocker Reproduction is: hadoop fs -mkdir s3a://my-bucket/dir/ hadoop fs -ls s3a://my-bucket/dir/ ls: `s3a://my-bucket/dir/': No such file or directory I believe this is a regression from HADOOP-14255. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14274) Azure: Simplify Ranger-WASB policy model
[ https://issues.apache.org/jira/browse/HADOOP-14274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013309#comment-16013309 ] Mingliang Liu commented on HADOOP-14274: [~sisan...@microsoft.com] can you add releases notes for this JIRA? That should summarize major changes, and be helpful for others to get the context. Thanks, > Azure: Simplify Ranger-WASB policy model > > > Key: HADOOP-14274 > URL: https://issues.apache.org/jira/browse/HADOOP-14274 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14274-001.patch, HADOOP-14274.002.patch, > HADOOP-14274.003.patch, TestingForRegressionWithSecureModeSwitchedOff.txt, > TestingWithSecureModeSwitchedOnSuccess.txt > > > This improvement seeks to simplify the WASB-Ranger policy model -- both the > policy specification and policy enforcement. > More specifically, WASB-Ranger checks do not follow the same policy model and > enforcement as Ranger-HDFS. > Ranger-HDFS hands off to HDFS-ACLs when a policy-match is not found. The > handoff requires the Ranger policies follow the same model as HDFS-ACLs. This > is not true with Ranger+WASB. > We seek to simplify the policy specification and enforcement by dropping the > 'x' bit altogether. > This JIRA tracks this improvement, alongwith a few more minor bugfixes that > were found during Ranger-WASB testing. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14407) DistCp - Introduce a configurable copy buffer size
[ https://issues.apache.org/jira/browse/HADOOP-14407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013280#comment-16013280 ] Yongjun Zhang commented on HADOOP-14407: Hi [~omkarksa], The patch looks good, except a few cosmetic things. - The patch failed to apply. Try using "git diff --no-prefix HEAD~ " to generate the patch. - some trailing white spaces, use "git apply --whitespace fix" to remove. - DistCpOptions.java, line exceeding 80 chars Thanks. > DistCp - Introduce a configurable copy buffer size > -- > > Key: HADOOP-14407 > URL: https://issues.apache.org/jira/browse/HADOOP-14407 > Project: Hadoop Common > Issue Type: Improvement > Components: tools/distcp >Affects Versions: 2.9.0 >Reporter: Omkar Aradhya K S >Assignee: Omkar Aradhya K S > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14407.001.patch > > > Currently, the RetriableFileCopyCommand has a fixed copy buffer size of just > 8KB. We have noticed in our performance tests that with bigger buffer sizes > we saw upto ~3x performance boost. Hence, making the copy buffer size a > configurable setting via the new parameter . -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-11457) Correction to BUILDING.txt: Update FindBugs to 3.0.0
[ https://issues.apache.org/jira/browse/HADOOP-11457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor reassigned HADOOP-11457: - Assignee: Andras Bokor (was: Rick Kellogg) > Correction to BUILDING.txt: Update FindBugs to 3.0.0 > > > Key: HADOOP-11457 > URL: https://issues.apache.org/jira/browse/HADOOP-11457 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Affects Versions: 2.6.0 >Reporter: Rick Kellogg >Assignee: Andras Bokor >Priority: Minor > Labels: newbie > Attachments: HADOOP-11457-1.patch > > > Based on Hadoop QA message shown in HADOOP-11402, it appears FindBugs 2.0.3 > is used. Latest release is 3.0.0. > Updating BUILDING.txt to reflect usage of v 2.0.3 instead of 1.3.9. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-12557) Add information in BUILDING.txt about the need for the FINDBUGS_HOME environment variable for docs builds.
[
https://issues.apache.org/jira/browse/HADOOP-12557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor reassigned HADOOP-12557:
-
Assignee: Andras Bokor (was: huangzheng)
> Add information in BUILDING.txt about the need for the FINDBUGS_HOME
> environment variable for docs builds.
> --
>
> Key: HADOOP-12557
> URL: https://issues.apache.org/jira/browse/HADOOP-12557
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation
>Affects Versions: 2.7.1
>Reporter: Chris Nauroth
>Assignee: Andras Bokor
>Priority: Minor
> Attachments: HADOOP-12557.patch
>
>
> BUILDING.txt mentions Findbugs 1.3.9 as a requirement, but it doesn't mention
> that the {{FINDBUGS_HOME}} environment variable must point to the base
> directory of the Findbugs installation when running builds with {{-Pdocs}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14419) Remove findbugs report from docs profile
[ https://issues.apache.org/jira/browse/HADOOP-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16013071#comment-16013071 ] Akira Ajisaka commented on HADOOP-14419: +1, thanks Andras. > Remove findbugs report from docs profile > > > Key: HADOOP-14419 > URL: https://issues.apache.org/jira/browse/HADOOP-14419 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Minor > Attachments: HADOOP-14419.01.patch, HADOOP-14419.02.patch > > > Based on [~aw]'s comments on HADOOP-12557 findbugs report is not needed in > the distro. > Let's remove it. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-7464) hadoop fs -stat '{glob}' gives null with combo of absolute and non-existent files
[
https://issues.apache.org/jira/browse/HADOOP-7464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor reassigned HADOOP-7464:
Assignee: Andras Bokor
> hadoop fs -stat '{glob}' gives null with combo of absolute and non-existent
> files
> -
>
> Key: HADOOP-7464
> URL: https://issues.apache.org/jira/browse/HADOOP-7464
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Affects Versions: 0.20.2
> Environment: CDH3u0
>Reporter: Jay Hacker
>Assignee: Andras Bokor
>Priority: Minor
>
> I'm trying to {{hadoop fs -stat}} a list of HDFS files all at once, because
> doing them one at a time is slow. stat doesn't accept multiple arguments, so
> I'm using a glob of the form '\{file1,file2\}' (quoted from the shell). I've
> discovered this doesn't work for me because the glob expands non-existent
> files to nothing, and I get nothing back from stat. It would be nice to be
> able to use stat for this, but perhaps that's more of a feature request.
> However, in the process, I discovered that with relative pathnames, I get
> back the stats for the existing files. With absolute filenames, I get back
> {{stat: null}}.
> $ hadoop fs -touchz file1 file2
> $ hadoop fs -stat '\{file1,file2\}'
> 2011-07-15 21:21:19
> 2011-07-15 21:21:19
> $ hadoop fs -stat '\{file1,file2,nonexistent\}'
> 2011-07-15 21:21:19
> 2011-07-15 21:21:19
> $ hadoop fs -stat '\{user/me/file1,/user/me/file2\}'
> 2011-07-15 21:21:19
> 2011-07-15 21:21:19
> $ hadoop fs -stat '\{/user/me/file1,/user/me/file2,nonexistent\}'
> stat: null
> Perhaps I'm doing something dumb, but it seems like stat should give the same
> results whether you use relative or absolute paths.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14416) Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012952#comment-16012952 ] Hudson commented on HADOOP-14416: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11739 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11739/]) HADOOP-14416. Path starting with 'wasb:///' not resolved correctly while (liuml07: rev b415c6fe743242acf1d1d3eb7ea7091d90d2c0d4) * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestWasbRemoteCallHelper.java * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/MockWasbAuthorizerImpl.java * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/AbstractWasbTestBase.java * (edit) hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/TestNativeAzureFileSystemAuthorization.java * (edit) hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java > Path starting with 'wasb:///' not resolved correctly while authorizing with > WASB-Ranger > --- > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Fix For: 2.9.0, 3.0.0-alpha3 > > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt, > SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14416) Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger
[
https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HADOOP-14416:
---
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha3
2.9.0
Status: Resolved (was: Patch Available)
Committed to {{branch-2}} and {{trunk}} branches. Thanks for your contribution
[~sisan...@microsoft.com].
> Path starting with 'wasb:///' not resolved correctly while authorizing with
> WASB-Ranger
> ---
>
> Key: HADOOP-14416
> URL: https://issues.apache.org/jira/browse/HADOOP-14416
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, security
>Reporter: Sivaguru Sankaridurg
>Assignee: Sivaguru Sankaridurg
> Fix For: 2.9.0, 3.0.0-alpha3
>
> Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch,
> HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt,
> SecureRunLogs.txt
>
>
> Bug found while launching spark-shell.
> Repro-steps :
> 1. Create a spark cluster with wasb-acls enabled.
> 2. Change spark history log directory configurations to
> wasb:///hdp/spark2-events
> 3. Launching the spark shell should fail.
> The above scenario works fine with clusters that dont have wasb-acl
> authorization enabled.
> Note : wasb:/// resolves correctly on fs shell.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
[
https://issues.apache.org/jira/browse/HADOOP-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012874#comment-16012874
]
Hadoop QA commented on HADOOP-14427:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
34s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} hadoop-common-project/hadoop-common: The patch
generated 0 new + 51 unchanged - 1 fixed = 51 total (was 52) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
20s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14427 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868357/HADOOP-14427-01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1ddb3d6708e2 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 89a8edc |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12338/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12338/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12338/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Avoid reloading of Configuration in ViewFileSystem creation.
>
>
> Key: HADOOP-14427
>
[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to consistent S3 endpoints
[
https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012868#comment-16012868
]
Hadoop QA commented on HADOOP-13786:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 25m
45s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 39 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
7s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 19m
3s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
52s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 2m
52s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
44s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
13s{color} | {color:green} HADOOP-13345 passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 13m 18s{color}
| {color:red} root generated 1 new + 777 unchanged - 1 fixed = 778 total (was
778) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 5s{color} | {color:orange} root: The patch generated 51 new + 100 unchanged
- 23 fixed = 151 total (was 123) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-registry in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
27s{color} | {color:red} hadoop-aws in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
53s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
58s{color} | {color:green} hadoop-yarn-registry in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
54s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
59s{color} | {color:green} hadoop-aws in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}148m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:612578f |
| JIRA Issue | HADOOP-13786 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868340/HADOOP-13786-HADOOP-13345-027.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux 2e8e45e81351 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017
[jira] [Updated] (HADOOP-14416) Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14416: --- Summary: Path starting with 'wasb:///' not resolved correctly while authorizing with WASB-Ranger (was: Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)) > Path starting with 'wasb:///' not resolved correctly while authorizing with > WASB-Ranger > --- > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt, > SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
[
https://issues.apache.org/jira/browse/HADOOP-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012847#comment-16012847
]
Brahma Reddy Battula commented on HADOOP-14427:
---
me too +1
> Avoid reloading of Configuration in ViewFileSystem creation.
>
>
> Key: HADOOP-14427
> URL: https://issues.apache.org/jira/browse/HADOOP-14427
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
> Attachments: HADOOP-14427-01.patch
>
>
> Avoid {{new Configuration()}} in below code. during viewfilesystem creation
> {code}public InternalDirOfViewFs(final InodeTree.INodeDir dir,
> final long cTime, final UserGroupInformation ugi, URI uri)
> throws URISyntaxException {
> myUri = uri;
> try {
> initialize(myUri, new Configuration());
> } catch (IOException e) {
> throw new RuntimeException("Cannot occur");
> }
> theInternalDir = dir;
> creationTime = cTime;
> this.ugi = ugi;
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
[
https://issues.apache.org/jira/browse/HADOOP-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012819#comment-16012819
]
Mingliang Liu commented on HADOOP-14427:
+1
> Avoid reloading of Configuration in ViewFileSystem creation.
>
>
> Key: HADOOP-14427
> URL: https://issues.apache.org/jira/browse/HADOOP-14427
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
> Attachments: HADOOP-14427-01.patch
>
>
> Avoid {{new Configuration()}} in below code. during viewfilesystem creation
> {code}public InternalDirOfViewFs(final InodeTree.INodeDir dir,
> final long cTime, final UserGroupInformation ugi, URI uri)
> throws URISyntaxException {
> myUri = uri;
> try {
> initialize(myUri, new Configuration());
> } catch (IOException e) {
> throw new RuntimeException("Cannot occur");
> }
> theInternalDir = dir;
> creationTime = cTime;
> this.ugi = ugi;
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13866) Upgrade netty-all to 4.1.1.Final
[
https://issues.apache.org/jira/browse/HADOOP-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012773#comment-16012773
]
Ted Yu commented on HADOOP-13866:
-
The shading discussion thread mentioned above hasn't led to a JIRA yet.
There would be better visibility once the discussion draws some conclusion.
> Upgrade netty-all to 4.1.1.Final
>
>
> Key: HADOOP-13866
> URL: https://issues.apache.org/jira/browse/HADOOP-13866
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha1
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HADOOP-13866.v1.patch, HADOOP-13866.v2.patch,
> HADOOP-13866.v3.patch, HADOOP-13866.v4.patch, HADOOP-13866.v6.patch,
> HADOOP-13866.v7.patch, HADOOP-13866.v8.patch, HADOOP-13866.v8.patch,
> HADOOP-13866.v8.patch, HADOOP-13866.v9.patch
>
>
> netty-all 4.1.1.Final is stable release which we should upgrade to.
> See bottom of HADOOP-12927 for related discussion.
> This issue was discovered since hbase 2.0 uses 4.1.1.Final of netty.
> When launching mapreduce job from hbase, /grid/0/hadoop/yarn/local/
> usercache/hbase/appcache/application_1479850535804_0008/container_e01_1479850535804_0008_01_05/mr-framework/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar
> (from hdfs) is ahead of 4.1.1.Final jar (from hbase) on the classpath.
> Resulting in the following exception:
> {code}
> 2016-12-01 20:17:26,678 WARN [Default-IPC-NioEventLoopGroup-1-1]
> io.netty.util.concurrent.DefaultPromise: An exception was thrown by
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete()
> java.lang.NoSuchMethodError:
> io.netty.buffer.ByteBuf.retainedDuplicate()Lio/netty/buffer/ByteBuf;
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:272)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:262)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
> at
> io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13866) Upgrade netty-all to 4.1.1.Final
[
https://issues.apache.org/jira/browse/HADOOP-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012763#comment-16012763
]
Hadoop QA commented on HADOOP-13866:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red} HADOOP-13866 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-13866 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12852194/HADOOP-13866.v9.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12337/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Upgrade netty-all to 4.1.1.Final
>
>
> Key: HADOOP-13866
> URL: https://issues.apache.org/jira/browse/HADOOP-13866
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha1
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HADOOP-13866.v1.patch, HADOOP-13866.v2.patch,
> HADOOP-13866.v3.patch, HADOOP-13866.v4.patch, HADOOP-13866.v6.patch,
> HADOOP-13866.v7.patch, HADOOP-13866.v8.patch, HADOOP-13866.v8.patch,
> HADOOP-13866.v8.patch, HADOOP-13866.v9.patch
>
>
> netty-all 4.1.1.Final is stable release which we should upgrade to.
> See bottom of HADOOP-12927 for related discussion.
> This issue was discovered since hbase 2.0 uses 4.1.1.Final of netty.
> When launching mapreduce job from hbase, /grid/0/hadoop/yarn/local/
> usercache/hbase/appcache/application_1479850535804_0008/container_e01_1479850535804_0008_01_05/mr-framework/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar
> (from hdfs) is ahead of 4.1.1.Final jar (from hbase) on the classpath.
> Resulting in the following exception:
> {code}
> 2016-12-01 20:17:26,678 WARN [Default-IPC-NioEventLoopGroup-1-1]
> io.netty.util.concurrent.DefaultPromise: An exception was thrown by
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete()
> java.lang.NoSuchMethodError:
> io.netty.buffer.ByteBuf.retainedDuplicate()Lio/netty/buffer/ByteBuf;
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:272)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:262)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
> at
> io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
[
https://issues.apache.org/jira/browse/HADOOP-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated HADOOP-14427:
---
Status: Patch Available (was: Open)
> Avoid reloading of Configuration in ViewFileSystem creation.
>
>
> Key: HADOOP-14427
> URL: https://issues.apache.org/jira/browse/HADOOP-14427
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
> Attachments: HADOOP-14427-01.patch
>
>
> Avoid {{new Configuration()}} in below code. during viewfilesystem creation
> {code}public InternalDirOfViewFs(final InodeTree.INodeDir dir,
> final long cTime, final UserGroupInformation ugi, URI uri)
> throws URISyntaxException {
> myUri = uri;
> try {
> initialize(myUri, new Configuration());
> } catch (IOException e) {
> throw new RuntimeException("Cannot occur");
> }
> theInternalDir = dir;
> creationTime = cTime;
> this.ugi = ugi;
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
[
https://issues.apache.org/jira/browse/HADOOP-14427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinayakumar B updated HADOOP-14427:
---
Attachment: HADOOP-14427-01.patch
Attached the patch.
> Avoid reloading of Configuration in ViewFileSystem creation.
>
>
> Key: HADOOP-14427
> URL: https://issues.apache.org/jira/browse/HADOOP-14427
> Project: Hadoop Common
> Issue Type: Bug
> Components: viewfs
>Reporter: Vinayakumar B
>Assignee: Vinayakumar B
> Attachments: HADOOP-14427-01.patch
>
>
> Avoid {{new Configuration()}} in below code. during viewfilesystem creation
> {code}public InternalDirOfViewFs(final InodeTree.INodeDir dir,
> final long cTime, final UserGroupInformation ugi, URI uri)
> throws URISyntaxException {
> myUri = uri;
> try {
> initialize(myUri, new Configuration());
> } catch (IOException e) {
> throw new RuntimeException("Cannot occur");
> }
> theInternalDir = dir;
> creationTime = cTime;
> this.ugi = ugi;
> }{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14427) Avoid reloading of Configuration in ViewFileSystem creation.
Vinayakumar B created HADOOP-14427:
--
Summary: Avoid reloading of Configuration in ViewFileSystem
creation.
Key: HADOOP-14427
URL: https://issues.apache.org/jira/browse/HADOOP-14427
Project: Hadoop Common
Issue Type: Bug
Components: viewfs
Reporter: Vinayakumar B
Assignee: Vinayakumar B
Avoid {{new Configuration()}} in below code. during viewfilesystem creation
{code}public InternalDirOfViewFs(final InodeTree.INodeDir dir,
final long cTime, final UserGroupInformation ugi, URI uri)
throws URISyntaxException {
myUri = uri;
try {
initialize(myUri, new Configuration());
} catch (IOException e) {
throw new RuntimeException("Cannot occur");
}
theInternalDir = dir;
creationTime = cTime;
this.ugi = ugi;
}{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14412) HostsFileReader#getHostDetails is very expensive on large clusters
[
https://issues.apache.org/jira/browse/HADOOP-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012695#comment-16012695
]
Hadoop QA commented on HADOOP-14412:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
48s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
1s{color} | {color:green} branch-2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
23s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
45s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
36s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
2s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} branch-2 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} branch-2 passed with JDK v1.7.0_131 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
4s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
39s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
39s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 43s{color} | {color:orange} root: The patch generated 1 new + 24 unchanged -
2 fixed = 25 total (was 26) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
47s{color} | {color:green} hadoop-common in the patch passed with JDK
v1.7.0_131. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 41m 59s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}199m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.7.0_131 Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:8515d35 |
| JIRA Issue | HADOOP-14412 |
| JIRA
[jira] [Commented] (HADOOP-13866) Upgrade netty-all to 4.1.1.Final
[
https://issues.apache.org/jira/browse/HADOOP-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012689#comment-16012689
]
Robert Kanter commented on HADOOP-13866:
Any updates on this?
> Upgrade netty-all to 4.1.1.Final
>
>
> Key: HADOOP-13866
> URL: https://issues.apache.org/jira/browse/HADOOP-13866
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha1
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: HADOOP-13866.v1.patch, HADOOP-13866.v2.patch,
> HADOOP-13866.v3.patch, HADOOP-13866.v4.patch, HADOOP-13866.v6.patch,
> HADOOP-13866.v7.patch, HADOOP-13866.v8.patch, HADOOP-13866.v8.patch,
> HADOOP-13866.v8.patch, HADOOP-13866.v9.patch
>
>
> netty-all 4.1.1.Final is stable release which we should upgrade to.
> See bottom of HADOOP-12927 for related discussion.
> This issue was discovered since hbase 2.0 uses 4.1.1.Final of netty.
> When launching mapreduce job from hbase, /grid/0/hadoop/yarn/local/
> usercache/hbase/appcache/application_1479850535804_0008/container_e01_1479850535804_0008_01_05/mr-framework/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar
> (from hdfs) is ahead of 4.1.1.Final jar (from hbase) on the classpath.
> Resulting in the following exception:
> {code}
> 2016-12-01 20:17:26,678 WARN [Default-IPC-NioEventLoopGroup-1-1]
> io.netty.util.concurrent.DefaultPromise: An exception was thrown by
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete()
> java.lang.NoSuchMethodError:
> io.netty.buffer.ByteBuf.retainedDuplicate()Lio/netty/buffer/ByteBuf;
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:272)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:262)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
> at
> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
> at
> io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14411) HDFS architecture documentation describes outdated placement policy
[
https://issues.apache.org/jira/browse/HADOOP-14411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012626#comment-16012626
]
Akira Ajisaka commented on HADOOP-14411:
LGTM, +1
> HDFS architecture documentation describes outdated placement policy
> ---
>
> Key: HADOOP-14411
> URL: https://issues.apache.org/jira/browse/HADOOP-14411
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: dud
>Assignee: Chen Liang
>Priority: Minor
> Labels: documentation
> Attachments: HADOOP-14411.001.patch, HADOOP-14411.002.patch
>
>
> Hello
> I've noticed inconsistencies in the default block placement policy between
> what one can read on several websites and what is stated in the [official
> documentation|https://git-wip-us.apache.org/repos/asf?p=hadoop.git;a=blob;f=hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md;h=bda53a9c77b41308eef3abf5b478158b2ea03be0;hb=HEAD#l105].
> After digging, I've found out that the proper default placement policy is :
> {code}
> * The replica placement strategy is that if the writer is on a datanode,
> * the 1st replica is placed on the local machine,
> * otherwise a random datanode. The 2nd replica is placed on a datanode
> * that is on a different rack. The 3rd replica is placed on a datanode
> * which is on a different node of the rack as the second replica.
> {code}
> [source|https://git-wip-us.apache.org/repos/asf?p=hadoop.git;a=blob;f=hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java#l45]
> HADOOP-5734 has been opened regarding this mistake several years ago and has
> eventually been merged but unfortunately this fix has been overwritten by
> [~tucu00] in [SVN
> commit|https://svn.apache.org/viewvc?view=revision=1425527].
> dud
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13786) Add S3Guard committer for zero-rename commits to consistent S3 endpoints
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13786: Status: Patch Available (was: Open) > Add S3Guard committer for zero-rename commits to consistent S3 endpoints > > > Key: HADOOP-13786 > URL: https://issues.apache.org/jira/browse/HADOOP-13786 > Project: Hadoop Common > Issue Type: New Feature > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13786-HADOOP-13345-001.patch, > HADOOP-13786-HADOOP-13345-002.patch, HADOOP-13786-HADOOP-13345-003.patch, > HADOOP-13786-HADOOP-13345-004.patch, HADOOP-13786-HADOOP-13345-005.patch, > HADOOP-13786-HADOOP-13345-006.patch, HADOOP-13786-HADOOP-13345-006.patch, > HADOOP-13786-HADOOP-13345-007.patch, HADOOP-13786-HADOOP-13345-009.patch, > HADOOP-13786-HADOOP-13345-010.patch, HADOOP-13786-HADOOP-13345-011.patch, > HADOOP-13786-HADOOP-13345-012.patch, HADOOP-13786-HADOOP-13345-013.patch, > HADOOP-13786-HADOOP-13345-015.patch, HADOOP-13786-HADOOP-13345-016.patch, > HADOOP-13786-HADOOP-13345-017.patch, HADOOP-13786-HADOOP-13345-018.patch, > HADOOP-13786-HADOOP-13345-019.patch, HADOOP-13786-HADOOP-13345-020.patch, > HADOOP-13786-HADOOP-13345-021.patch, HADOOP-13786-HADOOP-13345-022.patch, > HADOOP-13786-HADOOP-13345-023.patch, HADOOP-13786-HADOOP-13345-024.patch, > HADOOP-13786-HADOOP-13345-025.patch, HADOOP-13786-HADOOP-13345-026.patch, > HADOOP-13786-HADOOP-13345-027.patch, objectstore.pdf, s3committer-master.zip > > > A goal of this code is "support O(1) commits to S3 repositories in the > presence of failures". Implement it, including whatever is needed to > demonstrate the correctness of the algorithm. (that is, assuming that s3guard > provides a consistent view of the presence/absence of blobs, show that we can > commit directly). > I consider ourselves free to expose the blobstore-ness of the s3 output > streams (ie. not visible until the close()), if we need to use that to allow > us to abort commit operations. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13786) Add S3Guard committer for zero-rename commits to consistent S3 endpoints
[
https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13786:
Attachment: HADOOP-13786-HADOOP-13345-027.patch
Patch 027
# fix javadoc, checkstyle complaints, especially by encapsulating fields in the
json-serialized data.
# strip out stuff which is superfluous now the Magic committer is using the
staging committer commit code
# tests for the {{uploadFile(File, dest)}} routine used by the staging
committer; now handles 0 byte files & has a leaner API
This is now starting to feel like it is coming together. I'm still focusing on
the staging committer(s) as the ones to be ready to use first, but the fact
that things are unified and code cut is a sign of tightening up.
TODO
* the bits in the code with TODO in them.
* isolated tests for the Task class
* instrumentation
* scale tests.
I'm looking at doing the scale tests in spark code, downstream of spark itself.
Spark now pulls in the hadoop-aws JARs (& azure) if built right, so
out-the-git-repo spark/master is ready to be used in the tests. Doing it in
spark will (a) make it easier for me to write queries running in the test JVM
itself, which offers: parallelism, easier to inject faults, and to get some log
output when it all goes wrong.
Testing: s3 ireland. I'm doing a test with s3guard local right now, and things
are failing there, so somehow I've broken that. Moral: we should start asking
for test runs against s3guard as well as "classic"
> Add S3Guard committer for zero-rename commits to consistent S3 endpoints
>
>
> Key: HADOOP-13786
> URL: https://issues.apache.org/jira/browse/HADOOP-13786
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Steve Loughran
>Assignee: Steve Loughran
> Attachments: HADOOP-13786-HADOOP-13345-001.patch,
> HADOOP-13786-HADOOP-13345-002.patch, HADOOP-13786-HADOOP-13345-003.patch,
> HADOOP-13786-HADOOP-13345-004.patch, HADOOP-13786-HADOOP-13345-005.patch,
> HADOOP-13786-HADOOP-13345-006.patch, HADOOP-13786-HADOOP-13345-006.patch,
> HADOOP-13786-HADOOP-13345-007.patch, HADOOP-13786-HADOOP-13345-009.patch,
> HADOOP-13786-HADOOP-13345-010.patch, HADOOP-13786-HADOOP-13345-011.patch,
> HADOOP-13786-HADOOP-13345-012.patch, HADOOP-13786-HADOOP-13345-013.patch,
> HADOOP-13786-HADOOP-13345-015.patch, HADOOP-13786-HADOOP-13345-016.patch,
> HADOOP-13786-HADOOP-13345-017.patch, HADOOP-13786-HADOOP-13345-018.patch,
> HADOOP-13786-HADOOP-13345-019.patch, HADOOP-13786-HADOOP-13345-020.patch,
> HADOOP-13786-HADOOP-13345-021.patch, HADOOP-13786-HADOOP-13345-022.patch,
> HADOOP-13786-HADOOP-13345-023.patch, HADOOP-13786-HADOOP-13345-024.patch,
> HADOOP-13786-HADOOP-13345-025.patch, HADOOP-13786-HADOOP-13345-026.patch,
> HADOOP-13786-HADOOP-13345-027.patch, objectstore.pdf, s3committer-master.zip
>
>
> A goal of this code is "support O(1) commits to S3 repositories in the
> presence of failures". Implement it, including whatever is needed to
> demonstrate the correctness of the algorithm. (that is, assuming that s3guard
> provides a consistent view of the presence/absence of blobs, show that we can
> commit directly).
> I consider ourselves free to expose the blobstore-ness of the s3 output
> streams (ie. not visible until the close()), if we need to use that to allow
> us to abort commit operations.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13786) Add S3Guard committer for zero-rename commits to consistent S3 endpoints
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13786: Status: Open (was: Patch Available) > Add S3Guard committer for zero-rename commits to consistent S3 endpoints > > > Key: HADOOP-13786 > URL: https://issues.apache.org/jira/browse/HADOOP-13786 > Project: Hadoop Common > Issue Type: New Feature > Components: fs/s3 >Affects Versions: HADOOP-13345 >Reporter: Steve Loughran >Assignee: Steve Loughran > Attachments: HADOOP-13786-HADOOP-13345-001.patch, > HADOOP-13786-HADOOP-13345-002.patch, HADOOP-13786-HADOOP-13345-003.patch, > HADOOP-13786-HADOOP-13345-004.patch, HADOOP-13786-HADOOP-13345-005.patch, > HADOOP-13786-HADOOP-13345-006.patch, HADOOP-13786-HADOOP-13345-006.patch, > HADOOP-13786-HADOOP-13345-007.patch, HADOOP-13786-HADOOP-13345-009.patch, > HADOOP-13786-HADOOP-13345-010.patch, HADOOP-13786-HADOOP-13345-011.patch, > HADOOP-13786-HADOOP-13345-012.patch, HADOOP-13786-HADOOP-13345-013.patch, > HADOOP-13786-HADOOP-13345-015.patch, HADOOP-13786-HADOOP-13345-016.patch, > HADOOP-13786-HADOOP-13345-017.patch, HADOOP-13786-HADOOP-13345-018.patch, > HADOOP-13786-HADOOP-13345-019.patch, HADOOP-13786-HADOOP-13345-020.patch, > HADOOP-13786-HADOOP-13345-021.patch, HADOOP-13786-HADOOP-13345-022.patch, > HADOOP-13786-HADOOP-13345-023.patch, HADOOP-13786-HADOOP-13345-024.patch, > HADOOP-13786-HADOOP-13345-025.patch, HADOOP-13786-HADOOP-13345-026.patch, > objectstore.pdf, s3committer-master.zip > > > A goal of this code is "support O(1) commits to S3 repositories in the > presence of failures". Implement it, including whatever is needed to > demonstrate the correctness of the algorithm. (that is, assuming that s3guard > provides a consistent view of the presence/absence of blobs, show that we can > commit directly). > I consider ourselves free to expose the blobstore-ness of the s3 output > streams (ie. not visible until the close()), if we need to use that to allow > us to abort commit operations. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14146) KerberosAuthenticationHandler should authenticate with SPN in AP-REQ
[ https://issues.apache.org/jira/browse/HADOOP-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012522#comment-16012522 ] Daryn Sharp commented on HADOOP-14146: -- [~drankye], thanks for the reviews! Don't mean to be pedantic but could you please provide an explicit +1? BTW, we've had great success with this patch internally. > KerberosAuthenticationHandler should authenticate with SPN in AP-REQ > > > Key: HADOOP-14146 > URL: https://issues.apache.org/jira/browse/HADOOP-14146 > Project: Hadoop Common > Issue Type: Bug > Components: security >Affects Versions: 2.5.0 >Reporter: Daryn Sharp >Assignee: Daryn Sharp > Attachments: HADOOP-14146.1.patch, HADOOP-14146.patch > > > Many attempts (HADOOP-10158, HADOOP-11628, HADOOP-13565) have tried to add > multiple SPN host and/or realm support to spnego authentication. The basic > problem is the server tries to guess and/or brute force what SPN the client > used. The server should just decode the SPN from the AP-REQ. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14421) TestAdlFileSystemContractLive#testListStatus assertion failed
[
https://issues.apache.org/jira/browse/HADOOP-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012519#comment-16012519
]
John Zhuge commented on HADOOP-14421:
-
Thanks [~vishwajeet.dusane]. Hope this is just a test code reliability issue,
not the dreaded backend inconsistency. It'd nice to improve the test code
without masking backend inconsistency.
> TestAdlFileSystemContractLive#testListStatus assertion failed
> -
>
> Key: HADOOP-14421
> URL: https://issues.apache.org/jira/browse/HADOOP-14421
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/adl
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: Atul Sikaria
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:273
> expected:<1> but was:<11>
> {noformat}
> Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 35.118 sec
> <<< FAILURE! - in org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive
> testListStatus(org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive)
> Time elapsed: 0.518 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<1> but was:<11>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.hadoop.fs.FileSystemContractBaseTest.testListStatus(FileSystemContractBaseTest.java:273)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at
> org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive.runTest(TestAdlFileSystemContractLive.java:60)
> {noformat}
> This is the first time we saw the issue. The test store {{rwj2dm}} was
> created on the fly and destroyed after the test.
> The code base does not have HADOOP-14230 which cleans up the test dir better.
> Trying to determine whether this might help.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14421) TestAdlFileSystemContractLive#testListStatus assertion failed
[
https://issues.apache.org/jira/browse/HADOOP-14421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012510#comment-16012510
]
Vishwajeet Dusane commented on HADOOP-14421:
Looks like HADOOP-14230 should help in this case. I am trying to repro this
issue and add an appropriate fix around it which should help other test cases
as well.
> TestAdlFileSystemContractLive#testListStatus assertion failed
> -
>
> Key: HADOOP-14421
> URL: https://issues.apache.org/jira/browse/HADOOP-14421
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/adl
>Affects Versions: 2.8.0
>Reporter: John Zhuge
>Assignee: Atul Sikaria
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.testListStatus:273
> expected:<1> but was:<11>
> {noformat}
> Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 35.118 sec
> <<< FAILURE! - in org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive
> testListStatus(org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive)
> Time elapsed: 0.518 sec <<< FAILURE!
> junit.framework.AssertionFailedError: expected:<1> but was:<11>
> at junit.framework.Assert.fail(Assert.java:57)
> at junit.framework.Assert.failNotEquals(Assert.java:329)
> at junit.framework.Assert.assertEquals(Assert.java:78)
> at junit.framework.Assert.assertEquals(Assert.java:234)
> at junit.framework.Assert.assertEquals(Assert.java:241)
> at junit.framework.TestCase.assertEquals(TestCase.java:409)
> at
> org.apache.hadoop.fs.FileSystemContractBaseTest.testListStatus(FileSystemContractBaseTest.java:273)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at
> org.apache.hadoop.fs.adl.live.TestAdlFileSystemContractLive.runTest(TestAdlFileSystemContractLive.java:60)
> {noformat}
> This is the first time we saw the issue. The test store {{rwj2dm}} was
> created on the fly and destroyed after the test.
> The code base does not have HADOOP-14230 which cleans up the test dir better.
> Trying to determine whether this might help.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13887) Support for client-side encryption in S3A file system
[ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012491#comment-16012491 ] Igor Mazur commented on HADOOP-13887: - [~ste...@apache.org] What do you think about propagating S3 file Metadata through FileSystem.getXAttr() ? This might be useful in general and later we can use those data to emulate client side encryption. > Support for client-side encryption in S3A file system > - > > Key: HADOOP-13887 > URL: https://issues.apache.org/jira/browse/HADOOP-13887 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.8.0 >Reporter: Jeeyoung Kim >Assignee: Igor Mazur >Priority: Minor > Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, > HADOOP-13887-branch-2-003.patch, HADOOP-13897-branch-2-004.patch, > HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch, > HADOOP-13897-branch-2-008.patch, HADOOP-13897-branch-2-009.patch, > HADOOP-13897-branch-2-010.patch, HADOOP-13897-branch-2-012.patch, > HADOOP-13897-branch-2-014.patch, HADOOP-13897-trunk-011.patch, > HADOOP-13897-trunk-013.patch, HADOOP-14171-001.patch > > > Expose the client-side encryption option documented in Amazon S3 > documentation - > http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html > Currently this is not exposed in Hadoop but it is exposed as an option in AWS > Java SDK, which Hadoop currently includes. It should be trivial to propagate > this as a parameter passed to the S3client used in S3AFileSystem.java -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14297) Update the documentation about the new ec codecs config keys
[
https://issues.apache.org/jira/browse/HADOOP-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012397#comment-16012397
]
Hadoop QA commented on HADOOP-14297:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14297 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868316/HADOOP-14297.04.patch
|
| Optional Tests | asflicense mvnsite |
| uname | Linux 8045558e7a8f 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11
16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12334/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Update the documentation about the new ec codecs config keys
>
>
> Key: HADOOP-14297
> URL: https://issues.apache.org/jira/browse/HADOOP-14297
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Kai Sasaki
>Assignee: Kai Sasaki
> Attachments: HADOOP-14297.01.patch, HADOOP-14297.02.patch,
> HADOOP-14297.03.patch, HADOOP-14297.04.patch
>
>
> In HADOOP-13665,
> io.erasurecode.codec.{rs-legacy.rawcoder,rs.rawcoder,xor.rawcoder} are no
> more used.
> It is necessary to update {{HDFSErasureCoding.md}} to show new config keys
> io.erasurecode.codec.{rs-legacy.rawcoders,rs.rawcoders,xor.rawcoders} instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14424) Add CRC32C performance test.
[
https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012363#comment-16012363
]
Hadoop QA commented on HADOOP-14424:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
18s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
0s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 29s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 7 new + 13 unchanged - 0 fixed = 20 total (was 13) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
26s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14424 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868301/HADOOP-14424.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux ee463c09e5f0 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12333/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12333/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12333/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12333/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add CRC32C performance test.
>
>
> Key: HADOOP-14424
> URL:
[jira] [Updated] (HADOOP-14297) Update the documentation about the new ec codecs config keys
[
https://issues.apache.org/jira/browse/HADOOP-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kai Sasaki updated HADOOP-14297:
Attachment: HADOOP-14297.04.patch
> Update the documentation about the new ec codecs config keys
>
>
> Key: HADOOP-14297
> URL: https://issues.apache.org/jira/browse/HADOOP-14297
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: documentation
>Reporter: Kai Sasaki
>Assignee: Kai Sasaki
> Attachments: HADOOP-14297.01.patch, HADOOP-14297.02.patch,
> HADOOP-14297.03.patch, HADOOP-14297.04.patch
>
>
> In HADOOP-13665,
> io.erasurecode.codec.{rs-legacy.rawcoder,rs.rawcoder,xor.rawcoder} are no
> more used.
> It is necessary to update {{HDFSErasureCoding.md}} to show new config keys
> io.erasurecode.codec.{rs-legacy.rawcoders,rs.rawcoders,xor.rawcoders} instead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14412) HostsFileReader#getHostDetails is very expensive on large clusters
[ https://issues.apache.org/jira/browse/HADOOP-14412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated HADOOP-14412: Attachment: HADOOP-14412-branch-2.002.patch Uploading the same branch-2 patch again to trigger a Jenkins run. > HostsFileReader#getHostDetails is very expensive on large clusters > -- > > Key: HADOOP-14412 > URL: https://issues.apache.org/jira/browse/HADOOP-14412 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 2.8.0 >Reporter: Jason Lowe >Assignee: Jason Lowe > Attachments: HADOOP-14412.001.patch, HADOOP-14412.002.patch, > HADOOP-14412-branch-2.001.patch, HADOOP-14412-branch-2.002.patch, > HADOOP-14412-branch-2.002.patch, HADOOP-14412-branch-2.8.002.patch > > > After upgrading one of our large clusters to 2.8 we noticed many IPC server > threads of the resourcemanager spending time in NodesListManager#isValidNode > which in turn was calling HostsFileReader#getHostDetails. The latter is > creating complete copies of the include and exclude sets for every node > heartbeat, and these sets are not small due to the size of the cluster. > These copies are causing multiple resizes of the underlying HashSets being > filled and creating lots of garbage. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14424) Add CRC32C performance test.
[ https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiXin Ge updated HADOOP-14424: -- Attachment: HADOOP-14424.patch try to fix [VisibilityModifier] style warnings. not very sure > Add CRC32C performance test. > > > Key: HADOOP-14424 > URL: https://issues.apache.org/jira/browse/HADOOP-14424 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-alpha2 >Reporter: LiXin Ge >Assignee: LiXin Ge >Priority: Minor > Labels: test > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-14424.patch > > > The default checksum algorithm of Hadoop is CRC32C, so we'd better add a new > test to compare Crc32C chunked verification implementations. > This test is based on Crc32PerformanceTest, what I have done in this test is: > 1.CRC32C performance test. > 2.CRC32C is not supported by java.util.zip in JAVA JDK, so just remove it > from this test. > 3.User can choose either direct buffer or non-directly buffer to run this > test manually. > 4.Take use of verifyChunkedSumsByteArray for native to support non-directly > native test. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14424) Add CRC32C performance test.
[ https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiXin Ge updated HADOOP-14424: -- Attachment: (was: HADOOP-14424.patch) > Add CRC32C performance test. > > > Key: HADOOP-14424 > URL: https://issues.apache.org/jira/browse/HADOOP-14424 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-alpha2 >Reporter: LiXin Ge >Assignee: LiXin Ge >Priority: Minor > Labels: test > Fix For: 3.0.0-alpha2 > > > The default checksum algorithm of Hadoop is CRC32C, so we'd better add a new > test to compare Crc32C chunked verification implementations. > This test is based on Crc32PerformanceTest, what I have done in this test is: > 1.CRC32C performance test. > 2.CRC32C is not supported by java.util.zip in JAVA JDK, so just remove it > from this test. > 3.User can choose either direct buffer or non-directly buffer to run this > test manually. > 4.Take use of verifyChunkedSumsByteArray for native to support non-directly > native test. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012186#comment-16012186
]
Hadoop QA commented on HADOOP-14422:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
37s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14422 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868267/HADOOP-14422-005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4c83579a2b86 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12332/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12332/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12332/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
>
[jira] [Commented] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012138#comment-16012138
]
Hadoop QA commented on HADOOP-14422:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
34s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 6s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.fs.viewfs.TestViewFileSystemLocalFileSystem |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14422 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868266/HADOOP-14422-005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 487c004195f2 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12331/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12331/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12331/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12331/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
>
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: (was: HADOOP-14422-005.patch)
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422-005.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: HADOOP-14422-005.patch
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422-005.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: HADOOP-14422-005.patch
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422-005.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: (was: HADOOP-14422-005.patch)
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422-005.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: HADOOP-14422-005.patch
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422-005.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14379) In federation mode,"hdfs dfsadmin -report" command can not be used
[
https://issues.apache.org/jira/browse/HADOOP-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012059#comment-16012059
]
Hadoop QA commented on HADOOP-14379:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
43s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
45s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
45s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 45s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 35s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 4 new + 192 unchanged - 0 fixed = 196 total (was 192) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
46s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 45s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14379 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868257/HADOOP-14379.005.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 963d8043709b 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12330/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
|
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12330/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12330/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12330/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs.txt
|
| checkstyle |
[jira] [Commented] (HADOOP-14416) Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012047#comment-16012047 ] Sivaguru Sankaridurg commented on HADOOP-14416: --- Hi [~liuml07]. I fixed all the checkstyle comments -- except one regarding the file-length. I'll fix that with a separate patch. > Path not resolving correctly while authorizing with WASB-Ranger when it > starts with 'wasb:///' (triple-slash) > - > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt, > SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14416) Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)
[
https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012035#comment-16012035
]
Hadoop QA commented on HADOOP-14416:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 11s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 1
new + 39 unchanged - 3 fixed = 40 total (was 42) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
21s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14416 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868253/HADOOP-14416.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 674d7e9a9e99 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12329/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12329/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12329/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Path not resolving correctly while authorizing with WASB-Ranger when it
> starts with 'wasb:///' (triple-slash)
> -
>
> Key: HADOOP-14416
> URL: https://issues.apache.org/jira/browse/HADOOP-14416
> Project: Hadoop Common
> Issue Type: Bug
>
[jira] [Commented] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012032#comment-16012032
]
Hadoop QA commented on HADOOP-14422:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
33s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
55s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 55s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
34s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 38s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14422 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868248/HADOOP-14422-004.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 8e3fc5586b55 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/patch-compile-root.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/patch-compile-root.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12328/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
|
| unit |
[jira] [Commented] (HADOOP-14426) Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
[
https://issues.apache.org/jira/browse/HADOOP-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012025#comment-16012025
]
Hadoop QA commented on HADOOP-14426:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 19m
5s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
6s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14426 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868245/HADOOP-14426-001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit xml |
| uname | Linux 10e231da7e40 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12327/testReport/ |
| modules | C: hadoop-project U: hadoop-project |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12327/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
> -
>
> Key: HADOOP-14426
> URL: https://issues.apache.org/jira/browse/HADOOP-14426
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Jiajia Li
>Assignee: Jiajia Li
> Attachments: HADOOP-14426-001.patch
>
>
> Apache Kerby 1.0.0 with some bug fixes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14379) In federation mode,"hdfs dfsadmin -report" command can not be used
[ https://issues.apache.org/jira/browse/HADOOP-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lixinglong updated HADOOP-14379: Attachment: HADOOP-14379.005.patch > In federation mode,"hdfs dfsadmin -report" command can not be used > -- > > Key: HADOOP-14379 > URL: https://issues.apache.org/jira/browse/HADOOP-14379 > Project: Hadoop Common > Issue Type: New Feature >Affects Versions: 3.0.0-alpha2 >Reporter: lixinglong >Assignee: lixinglong > Attachments: HADOOP-14379.001.patch, HADOOP-14379.002.patch, > HADOOP-14379.003.patch, HADOOP-14379.004.patch, HADOOP-14379.005.patch > > > In federation mode,"hdfs dfsadmin -report" command can not be used,as follows: > hdfs@zdh102:~> hdfs dfsadmin -report > report: FileSystem viewfs://nsX/ is not an HDFS file system > Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning] > After adding new features,"hdfs dfsadmin -report" command can be used,as > follows: > hdfs@zdh102:~> hdfs dfsadmin -report > hdfs://nameservice > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > Name: 10.43.183.103:50010 (zdh103) > Hostname: zdh103 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68901670912 (64.17 GB) > DFS Remaining: 193527250944 (180.24 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.74% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:42 CST 2017 > Name: 10.43.183.104:50010 (zdh104) > Hostname: zdh104 > Rack: /default > Decommission Status : Normal > Configured Capacity: 262429138944 (244.41 GB) > DFS Used: 217088 (212 KB) > Non DFS Used: 68800688128 (64.08 GB) > DFS Remaining: 193628233728 (180.33 GB) > DFS Used%: 0.00% > DFS Remaining%: 73.78% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 10:44:41 CST 2017 > hdfs://nameservice1 > Configured Capacity: 2682939863040 (2.44 TB) > Present Capacity: 2170811387904 (1.97 TB) > DFS Remaining: 2170810589184 (1.97 TB) > DFS Used: 798720 (780 KB) > DFS Used%: 0.00% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > Missing blocks (with replication factor 1): 0 > - > Live datanodes (4): > Name: 10.43.156.126:50010 (zdh126) > Hostname: zdh126 > Rack: /default > Decommission Status : Normal > Configured Capacity: 1578524934144 (1.44 TB) > DFS Used: 217088 (212 KB) > Non DFS Used: 163274125312 (152.06 GB) > DFS Remaining: 1415250591744 (1.29 TB) > DFS Used%: 0.00% > DFS Remaining%: 89.66% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 12:36:06 CST 2017 > Name: 10.43.183.102:50010 (zdh102) > Hostname: zdh102 > Rack: /default > Decommission Status : Normal > Configured Capacity: 579556651008 (539.75 GB) > DFS Used: 147456 (144 KB) > Non DFS Used: 211151990784 (196.65 GB) > DFS Remaining: 368404512768 (343.10 GB) > DFS Used%: 0.00% > DFS Remaining%: 63.57% > Configured Cache Capacity: 0 (0 B) > Cache Used: 0 (0 B) > Cache Remaining: 0 (0 B) > Cache Used%: 100.00% > Cache Remaining%: 0.00% > Xceivers: 2 > Last contact: Thu May 04 12:36:05 CST 2017 >
[jira] [Commented] (HADOOP-14424) Add CRC32C performance test.
[
https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16012008#comment-16012008
]
Hadoop QA commented on HADOOP-14424:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
27s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
13s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 37s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 12 new + 13 unchanged - 0 fixed = 25 total (was 13) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
45s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14424 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868240/HADOOP-14424.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 88db4147e228 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12326/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12326/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12326/testReport/ |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12326/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add CRC32C performance test.
>
>
> Key: HADOOP-14424
> URL:
[jira] [Updated] (HADOOP-14416) Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivaguru Sankaridurg updated HADOOP-14416: -- Attachment: HADOOP-14416.003.patch Fixed checkstyle comments from 002.patch > Path not resolving correctly while authorizing with WASB-Ranger when it > starts with 'wasb:///' (triple-slash) > - > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, HADOOP-14416.003.patch, Non-SecureRun-Logs.txt, > SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: HADOOP-14422-004.patch
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422-004.patch, HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14426) Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
[ https://issues.apache.org/jira/browse/HADOOP-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiajia Li updated HADOOP-14426: --- Status: Patch Available (was: Open) > Upgrade Kerby version from 1.0.0-RC2 to 1.0.0 > - > > Key: HADOOP-14426 > URL: https://issues.apache.org/jira/browse/HADOOP-14426 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jiajia Li >Assignee: Jiajia Li > Attachments: HADOOP-14426-001.patch > > > Apache Kerby 1.0.0 with some bug fixes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14426) Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
[ https://issues.apache.org/jira/browse/HADOOP-14426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiajia Li updated HADOOP-14426: --- Attachment: HADOOP-14426-001.patch > Upgrade Kerby version from 1.0.0-RC2 to 1.0.0 > - > > Key: HADOOP-14426 > URL: https://issues.apache.org/jira/browse/HADOOP-14426 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Jiajia Li >Assignee: Jiajia Li > Attachments: HADOOP-14426-001.patch > > > Apache Kerby 1.0.0 with some bug fixes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14426) Upgrade Kerby version from 1.0.0-RC2 to 1.0.0
Jiajia Li created HADOOP-14426: -- Summary: Upgrade Kerby version from 1.0.0-RC2 to 1.0.0 Key: HADOOP-14426 URL: https://issues.apache.org/jira/browse/HADOOP-14426 Project: Hadoop Common Issue Type: Improvement Reporter: Jiajia Li Assignee: Jiajia Li Apache Kerby 1.0.0 with some bug fixes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011921#comment-16011921
]
Hadoop QA commented on HADOOP-14422:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
44s{color} | {color:red} hadoop-common-project/hadoop-common in trunk has 19
extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
44s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
11s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 11s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
43s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 45s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14422 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868237/HADOOP-14422-003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e700639ed0c3 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
|
| mvninstall |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/patch-mvninstall-hadoop-common-project_hadoop-common.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/patch-compile-root.txt
|
| javac |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/patch-compile-root.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/patch-mvnsite-hadoop-common-project_hadoop-common.txt
|
| findbugs |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12324/artifact/patchprocess/patch-findbugs-hadoop-common-project_hadoop-common.txt
|
| unit |
[jira] [Commented] (HADOOP-14416) Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)
[
https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011904#comment-16011904
]
Hadoop QA commented on HADOOP-14416:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 11s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 2
new + 39 unchanged - 3 fixed = 41 total (was 42) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
21s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14416 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12868239/HADOOP-14416.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux afb5938fb83f 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / c48f297 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12325/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12325/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/12325/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Path not resolving correctly while authorizing with WASB-Ranger when it
> starts with 'wasb:///' (triple-slash)
> -
>
> Key: HADOOP-14416
> URL: https://issues.apache.org/jira/browse/HADOOP-14416
> Project: Hadoop Common
> Issue Type: Bug
>
[jira] [Updated] (HADOOP-14424) Add CRC32C performance test.
[ https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiXin Ge updated HADOOP-14424: -- Attachment: HADOOP-14424.patch update the patch against the checkstyle warnings > Add CRC32C performance test. > > > Key: HADOOP-14424 > URL: https://issues.apache.org/jira/browse/HADOOP-14424 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-alpha2 >Reporter: LiXin Ge >Assignee: LiXin Ge >Priority: Minor > Labels: test > Fix For: 3.0.0-alpha2 > > Attachments: HADOOP-14424.patch > > > The default checksum algorithm of Hadoop is CRC32C, so we'd better add a new > test to compare Crc32C chunked verification implementations. > This test is based on Crc32PerformanceTest, what I have done in this test is: > 1.CRC32C performance test. > 2.CRC32C is not supported by java.util.zip in JAVA JDK, so just remove it > from this test. > 3.User can choose either direct buffer or non-directly buffer to run this > test manually. > 4.Take use of verifyChunkedSumsByteArray for native to support non-directly > native test. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14424) Add CRC32C performance test.
[ https://issues.apache.org/jira/browse/HADOOP-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LiXin Ge updated HADOOP-14424: -- Attachment: (was: HADOOP-14424.patch) > Add CRC32C performance test. > > > Key: HADOOP-14424 > URL: https://issues.apache.org/jira/browse/HADOOP-14424 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.0.0-alpha2 >Reporter: LiXin Ge >Assignee: LiXin Ge >Priority: Minor > Labels: test > Fix For: 3.0.0-alpha2 > > > The default checksum algorithm of Hadoop is CRC32C, so we'd better add a new > test to compare Crc32C chunked verification implementations. > This test is based on Crc32PerformanceTest, what I have done in this test is: > 1.CRC32C performance test. > 2.CRC32C is not supported by java.util.zip in JAVA JDK, so just remove it > from this test. > 3.User can choose either direct buffer or non-directly buffer to run this > test manually. > 4.Take use of verifyChunkedSumsByteArray for native to support non-directly > native test. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14416) Path not resolving correctly while authorizing with WASB-Ranger when it starts with 'wasb:///' (triple-slash)
[ https://issues.apache.org/jira/browse/HADOOP-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivaguru Sankaridurg updated HADOOP-14416: -- Attachment: HADOOP-14416.002.patch Fixed checkstyle comments from 001.patch submission > Path not resolving correctly while authorizing with WASB-Ranger when it > starts with 'wasb:///' (triple-slash) > - > > Key: HADOOP-14416 > URL: https://issues.apache.org/jira/browse/HADOOP-14416 > Project: Hadoop Common > Issue Type: Bug > Components: fs/azure, security >Reporter: Sivaguru Sankaridurg >Assignee: Sivaguru Sankaridurg > Attachments: HADOOP-14416.001.patch, HADOOP-14416.001.patch, > HADOOP-14416.002.patch, Non-SecureRun-Logs.txt, SecureRunLogs.txt > > > Bug found while launching spark-shell. > Repro-steps : > 1. Create a spark cluster with wasb-acls enabled. > 2. Change spark history log directory configurations to > wasb:///hdp/spark2-events > 3. Launching the spark shell should fail. > The above scenario works fine with clusters that dont have wasb-acl > authorization enabled. > Note : wasb:/// resolves correctly on fs shell. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14422) FTPFileSystem and distcp does not work when ecnountered a complex ftp password with char ':' or '@'
[
https://issues.apache.org/jira/browse/HADOOP-14422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongyuan Li updated HADOOP-14422:
-
Attachment: HADOOP-14422-003.patch
add junit tests
> FTPFileSystem and distcp does not work when ecnountered a complex ftp
> password with char ':' or '@'
> ---
>
> Key: HADOOP-14422
> URL: https://issues.apache.org/jira/browse/HADOOP-14422
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, tools/distcp
>Affects Versions: 3.0.0-alpha2
>Reporter: Hongyuan Li
>Assignee: Hongyuan Li
> Attachments: HADOOP-14422-002.patch, HADOOP-14422-003.patch,
> HADOOP-14422.patch
>
>
> ftp username: test passwd:1@123
> hadoop distcp ftp://test:1@123@node2/home/test/ hdfs://piaopiao/test
> 17/05/15 15:24:56 INFO tools.DistCp: Input Options:
> DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
> ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
> copyStrategy='uniformsize', sourceFileListing=null,
> sourcePaths=[ftp://test:1@123@node2/home/test], targetPath=/test,
> targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
> 17/05/15 15:24:57 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 17/05/15 15:24:59 ERROR tools.DistCp: Exception encountered
> java.io.IOException: Login failed on server - 0.0.0.0, port - 21 as user
> 'null'
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.connect(FTPFileSystem.java:154)
> at
> org.apache.hadoop.fs.ftp.FTPFileSystem.getFileStatus(FTPFileSystem.java:415)
> at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:68)
> at org.apache.hadoop.fs.Globber.glob(Globber.java:263)
> at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1630)
> at
> org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:77)
> at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
> at
> org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:389)
> at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:187)
> at org.apache.hadoop.tools.DistCp.execute(DistCp.java:153)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:126)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:453)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
