[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410435#comment-16410435 ] Hudson commented on HDFS-10524: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13869 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13869/]) HDFS-10524: libhdfs++: Implement chmod and chown. Contributed by (james.clampffer: rev d4c3cfbf47ccc6d033cf9aa16b2eb16809b886ef) * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/namenode_operations.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/bindings/c/hdfs.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/namenode_operations.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_shim.c * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/test_libhdfs_threaded.c * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/include/hdfspp/hdfspp.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.h * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/tests/hdfs_ext_test.cc * (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filesystem.cc > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein >Priority: Major > Attachments: HDFS-10524.HDFS-8707.000.patch, > HDFS-10524.HDFS-8707.001.patch, HDFS-10524.HDFS-8707.002.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341523#comment-15341523 ] Bob Hansen commented on HDFS-10524: --- +1. Thanks, [~anatoli.shein] > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein > Attachments: HDFS-10524.HDFS-8707.000.patch, > HDFS-10524.HDFS-8707.001.patch, HDFS-10524.HDFS-8707.002.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[
https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340322#comment-15340322
]
Hadoop QA commented on HDFS-10524:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 36s
{color} | {color:green} HDFS-8707 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 12s
{color} | {color:green} HDFS-8707 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 15s
{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 7s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 16s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 32s
{color} | {color:green} hadoop-hdfs-native-client in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m 39s
{color} | {color:green} hadoop-hdfs-native-client in the patch passed with JDK
v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 57s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0cf5e66 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811962/HDFS-10524.HDFS-8707.002.patch
|
| JIRA Issue | HDFS-10524 |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 66bc2da188c1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-8707 / 6c264b1 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_91
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 |
| JDK v1.7.0_101 Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15834/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15834/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> libhdfs++: Implement chmod and chown
>
>
> Key: HDFS-10524
> URL: https://issues.apache.org/jira/browse/HDFS-10524
> Project: Hadoop HDFS
> Issue Type: Sub-task
>
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336809#comment-15336809 ] Bob Hansen commented on HDFS-10524: --- Thanks, [~anatoli.shein]. We're getting pretty close. The implementations in hdfs_shim.c of hdfsConnect, hdfsConnectAsUserNewInstance, and hdfsConnectNewInstance are returning invalid pointers. For the shim methods, we should be returning a shim hdfsFs_internal pointer, and your implementations of those are returning a libhdfs++ hdfsFS pointer. hdfsFreeBuilder should free the bld passed-in pointer when it is finished. hdfsDisconnect should try to disconnect the libhdfspp instance even if the libhdfs instance failed to disconnect. It should only return 0 if both returned 0. > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein > Attachments: HDFS-10524.HDFS-8707.000.patch, > HDFS-10524.HDFS-8707.001.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[
https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336139#comment-15336139
]
Hadoop QA commented on HDFS-10524:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 41s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
14s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 16s
{color} | {color:green} HDFS-8707 passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 14s
{color} | {color:green} HDFS-8707 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 18s
{color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s {color} | {color:green} HDFS-8707 passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_91 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 5m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 44s
{color} | {color:green} hadoop-hdfs-native-client in the patch passed with JDK
v1.8.0_91. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 6m 47s
{color} | {color:green} hadoop-hdfs-native-client in the patch passed with JDK
v1.7.0_101. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 31s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0cf5e66 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12811358/HDFS-10524.HDFS-8707.001.patch
|
| JIRA Issue | HDFS-10524 |
| Optional Tests | asflicense compile cc mvnsite javac unit |
| uname | Linux 07f0084c5524 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-8707 / 6c264b1 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_91
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 |
| JDK v1.7.0_101 Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15806/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15806/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> libhdfs++: Implement chmod and chown
>
>
> Key: HDFS-10524
> URL: https://issues.apache.org/jira/browse/HDFS-10524
> Project: Hadoop HDFS
> Issue Type: Sub-task
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332143#comment-15332143 ] Anatoli Shein commented on HDFS-10524: -- [~James C] and [~bobhansen], thank you for your feedback, I will include these changes into this patch. Regarding the permissions question, I actually looked into it before, and found out that posix permissions allow max value of 0, while Hadoop allows only up to 01777 (with just one sticky bit), so this part should be okay. I will just add a separate function in namenode_operations for this check, as James mentioned. > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein > Attachments: HDFS-10524.HDFS-8707.000.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332045#comment-15332045 ] James Clampffer commented on HDFS-10524: bq. I would respectfully disagree about having redundant parameter checking. If it's not a performance or maintenance burden, defence in depth against future stupidity is a Good Thing. Yea, you're right. I'd still like to see an implementation of something like IsValidPermissionMask() that gets used in both places so if we change the bounds checks it can be done in 1 place and doesn't require memorizing the upper bound. I'm sure [~anatoli.shein] knows the details better than myself at this point but IIRC that check needs to get done in some other RPC calls as well. > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein > Attachments: HDFS-10524.HDFS-8707.000.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[ https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331888#comment-15331888 ] Bob Hansen commented on HDFS-10524: --- [~James C]: Good catch on the future/tuple thing. I would respectfully disagree about having redundant parameter checking. If it's not a performance or maintenance burden, defence in depth against future stupidity is a Good Thing. [~anatoli.shein] - Are permissions of 01777 valid for Hadoop? While it's valid in POSIX-land, I think in Hadoop-land, the maximum value is 0777. In NameNodeOperations::SetPermission, whenever we're returning an error that a value is invalid, it is very helpful to the consumer to include the value that made it into the code. Frequently, the value has been mangled somewhere in the consumer or library code, and it can be a big help in debugging. In this case, when we return that the permissions are out of range, we should include the (octal) value of the permissions. A stringstream would be helpful in constructing the error message. > libhdfs++: Implement chmod and chown > > > Key: HDFS-10524 > URL: https://issues.apache.org/jira/browse/HDFS-10524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs-client >Reporter: Anatoli Shein >Assignee: Anatoli Shein > Attachments: HDFS-10524.HDFS-8707.000.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10524) libhdfs++: Implement chmod and chown
[
https://issues.apache.org/jira/browse/HDFS-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330670#comment-15330670
]
James Clampffer commented on HDFS-10524:
Just a couple minor things, everything looks solid functionally.
Do we need to use std::tuple in FileSystemImpl::SetPermission and SetOwner to
make 1-tuples? They'll likely get inlined away but less templates generally
makes things easier to debug.
{code}
auto callstate = std::make_shared>();
...
callstate->set_value(std::make_tuple(s));
...
auto returnstate = future.get();
Status stat = std::get<0>(returnstate);
{code}
It looks like there is some duplicate error checking code.
FileSystemImpl::SetPermissions will check the path and permission mask and then
call NameNodeOperations::SetPermission which does the same. Do we plan on
making the NameNodeOperations object accessable outside of FileSystem(Impl)?
If not it might be worth getting rid of the checks in one of them, or better
yet factoring out the checking code into a function and using that in both
places.
> libhdfs++: Implement chmod and chown
>
>
> Key: HDFS-10524
> URL: https://issues.apache.org/jira/browse/HDFS-10524
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
>Reporter: Anatoli Shein
>Assignee: Anatoli Shein
> Attachments: HDFS-10524.HDFS-8707.000.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
