subject:"\[jira\] \[Updated\] \(HDFS\-12485\) expunge may not remove trash from encryption zone"

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

2017-09-21 Thread Wei-Chiu Chuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HDFS-12485:
---
Attachment: HDFS-12485.001.patch

Update patch rev01.
* Use dfs.ugi.getShortUserName().
* Removed a reset of System.setProperty() in 
TestTrashWithSecureEncryptionZones. --> This is why the test passed for this 
bug.

> expunge may not remove trash from encryption zone
> -
>
> Key: HDFS-12485
> URL: https://issues.apache.org/jira/browse/HDFS-12485
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
> Attachments: HDFS-12485.001.patch
>
>
> This is related to HDFS-12484, but turns out that even if I have super user 
> permission, -expunge may not remove trash either.
> If I log into Linux as root, and then login as the superuser h...@example.com
> {noformat}
> [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
> 17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
> trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> [root@nightly511-1 ~]# hdfs dfs -expunge
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: 
> /user/hdfs/.Trash/170918143916
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> [root@nightly511-1 ~]# hdfs dfs -ls 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> -rw-r--r--   3 hdfs systest  0 2017-09-18 15:21 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> {noformat}
> expunge does not remove trash under /scale, because it does not know I am 
> 'hdfs' user.
> {code:title=DistributedFileSystem#getTrashRoots}
> Path ezTrashRoot = new Path(it.next().getPath(),
> FileSystem.TRASH_PREFIX);
> if (!exists(ezTrashRoot)) {
>   continue;
> }
> if (allUsers) {
>   for (FileStatus candidate : listStatus(ezTrashRoot)) {
> if (exists(candidate.getPath())) {
>   ret.add(candidate);
> }
>   }
> } else {
>   Path userTrash = new Path(ezTrashRoot, System.getProperty(
>   "user.name")); --> bug
>   try {
> ret.add(getFileStatus(userTrash));
>   } catch (FileNotFoundException ignored) {
>   }
> }
> {code}
> It should use UGI for user name, rather than system login user name.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

2017-09-21 Thread Wei-Chiu Chuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HDFS-12485:
---
Status: Patch Available  (was: Open)

> expunge may not remove trash from encryption zone
> -
>
> Key: HDFS-12485
> URL: https://issues.apache.org/jira/browse/HDFS-12485
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha1, 2.8.0
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
> Attachments: HDFS-12485.001.patch
>
>
> This is related to HDFS-12484, but turns out that even if I have super user 
> permission, -expunge may not remove trash either.
> If I log into Linux as root, and then login as the superuser h...@example.com
> {noformat}
> [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
> 17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
> trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> [root@nightly511-1 ~]# hdfs dfs -expunge
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: 
> /user/hdfs/.Trash/170918143916
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> [root@nightly511-1 ~]# hdfs dfs -ls 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> -rw-r--r--   3 hdfs systest  0 2017-09-18 15:21 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> {noformat}
> expunge does not remove trash under /scale, because it does not know I am 
> 'hdfs' user.
> {code:title=DistributedFileSystem#getTrashRoots}
> Path ezTrashRoot = new Path(it.next().getPath(),
> FileSystem.TRASH_PREFIX);
> if (!exists(ezTrashRoot)) {
>   continue;
> }
> if (allUsers) {
>   for (FileStatus candidate : listStatus(ezTrashRoot)) {
> if (exists(candidate.getPath())) {
>   ret.add(candidate);
> }
>   }
> } else {
>   Path userTrash = new Path(ezTrashRoot, System.getProperty(
>   "user.name")); --> bug
>   try {
> ret.add(getFileStatus(userTrash));
>   } catch (FileNotFoundException ignored) {
>   }
> }
> {code}
> It should use UGI for user name, rather than system login user name.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

2017-09-18 Thread Wei-Chiu Chuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HDFS-12485:
---
Description: 
This is related to HDFS-12484, but turns out that even if I have super user 
permission, -expunge may not remove trash either.

If I log into Linux as root, and then login as the superuser h...@example.com
{noformat}
[root@nightly511-1 ~]# hdfs dfs -rm /scale/b
17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
[root@nightly511-1 ~]# hdfs dfs -expunge
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: 
/user/hdfs/.Trash/170918143916
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
[root@nightly511-1 ~]# hdfs dfs -ls hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
-rw-r--r--   3 hdfs systest  0 2017-09-18 15:21 
hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
{noformat}

expunge does not remove trash under /scale, because it does not know I am 
'hdfs' user.

{code:title=DistributedFileSystem#getTrashRoots}
Path ezTrashRoot = new Path(it.next().getPath(),
FileSystem.TRASH_PREFIX);
if (!exists(ezTrashRoot)) {
  continue;
}
if (allUsers) {
  for (FileStatus candidate : listStatus(ezTrashRoot)) {
if (exists(candidate.getPath())) {
  ret.add(candidate);
}
  }
} else {
  Path userTrash = new Path(ezTrashRoot, System.getProperty(
  "user.name")); --> bug
  try {
ret.add(getFileStatus(userTrash));
  } catch (FileNotFoundException ignored) {
  }
}
{code}

It should use UGI for user name, rather than system login user name.

  was:
If I log into Linux as root, and then login as the superuser h...@example.com
{noformat}
[root@nightly511-1 ~]# hdfs dfs -rm /scale/b
17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
[root@nightly511-1 ~]# hdfs dfs -expunge
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: 
/user/hdfs/.Trash/170918143916
17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
[root@nightly511-1 ~]# hdfs dfs -ls hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
-rw-r--r--   3 hdfs systest  0 2017-09-18 15:21 
hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
{noformat}

expunge does not remove trash under /scale, because it does not know I am 
'hdfs' user.

{code:title=DistributedFileSystem#getTrashRoots}
Path ezTrashRoot = new Path(it.next().getPath(),
FileSystem.TRASH_PREFIX);
if (!exists(ezTrashRoot)) {
  continue;
}
if (allUsers) {
  for (FileStatus candidate : listStatus(ezTrashRoot)) {
if (exists(candidate.getPath())) {
  ret.add(candidate);
}
  }
} else {
  Path userTrash = new Path(ezTrashRoot, System.getProperty(
  "user.name")); --> bug
  try {
ret.add(getFileStatus(userTrash));
  } catch (FileNotFoundException ignored) {
  }
}
{code}

It should use UGI for user name, rather than system login user name.


> expunge may not remove trash from encryption zone
> -
>
> Key: HDFS-12485
> URL: https://issues.apache.org/jira/browse/HDFS-12485
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>
> This is related to HDFS-12484, but turns out that even if I have super user 
> permission, -expunge may not remove trash either.
> If I log into Linux as root, and then login as the superuser h...@example.com
> {noformat}
> [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
> 17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
> trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> [root@nightly511-1 ~]# hdfs dfs -expunge
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

2017-09-18 Thread Wei-Chiu Chuang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei-Chiu Chuang updated HDFS-12485:
---
Summary: expunge may not remove trash from encryption zone  (was: expunge 
may not remove trash from non-home directory encryption zone)

> expunge may not remove trash from encryption zone
> -
>
> Key: HDFS-12485
> URL: https://issues.apache.org/jira/browse/HDFS-12485
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>
> If I log into Linux as root, and then login as the superuser h...@example.com
> {noformat}
> [root@nightly511-1 ~]# hdfs dfs -rm /scale/b
> 17/09/18 15:21:32 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/scale/b' to 
> trash at: hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> [root@nightly511-1 ~]# hdfs dfs -expunge
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#deleteCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: Deleted trash checkpoint: 
> /user/hdfs/.Trash/170918143916
> 17/09/18 15:21:59 INFO fs.TrashPolicyDefault: 
> TrashPolicyDefault#createCheckpoint for trashRoot: hdfs://ns1/user/hdfs/.Trash
> [root@nightly511-1 ~]# hdfs dfs -ls 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> -rw-r--r--   3 hdfs systest  0 2017-09-18 15:21 
> hdfs://ns1/scale/.Trash/hdfs/Current/scale/b
> {noformat}
> expunge does not remove trash under /scale, because it does not know I am 
> 'hdfs' user.
> {code:title=DistributedFileSystem#getTrashRoots}
> Path ezTrashRoot = new Path(it.next().getPath(),
> FileSystem.TRASH_PREFIX);
> if (!exists(ezTrashRoot)) {
>   continue;
> }
> if (allUsers) {
>   for (FileStatus candidate : listStatus(ezTrashRoot)) {
> if (exists(candidate.getPath())) {
>   ret.add(candidate);
> }
>   }
> } else {
>   Path userTrash = new Path(ezTrashRoot, System.getProperty(
>   "user.name")); --> bug
>   try {
> ret.add(getFileStatus(userTrash));
>   } catch (FileNotFoundException ignored) {
>   }
> }
> {code}
> It should use UGI for user name, rather than system login user name.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

[jira] [Updated] (HDFS-12485) expunge may not remove trash from encryption zone

4 matches

Site Navigation

Mail list logo

Footer information