[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-16082: Labels: pull-request-available (was: ) > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2.0, 3.1.1 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available > Attachments: HADOOP-16082.001.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HADOOP-16082: - Target Version/s: 3.2.3 (was: 3.2.2) Updated the target version to 3.2.3 for preparing 3.2.2 release. Please let me know if it is blocker for you. Thanks. > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2.0, 3.1.1 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16082.001.patch > > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated HADOOP-16082: -- Target Version/s: 3.3.0, 3.2.1, 3.1.4 (was: 3.3.0, 3.2.1, 3.1.3) > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.2.0, 3.1.1 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16082.001.patch > > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HADOOP-16082: Status: In Progress (was: Patch Available) > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.1.1, 3.2.0 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > Attachments: HADOOP-16082.001.patch > > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[
https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-16082:
Affects Version/s: 3.2.0
Attachment: HADOOP-16082.001.patch
Target Version/s: 3.3.0, 3.2.1, 3.1.3
Status: Patch Available (was: Open)
Uploaded patch rev 001 for trunk. Implemented for HDFS. Added unit test
TestDFSShell#testLsInodeId() for HDFS. To test it,
{code:bash|title=hdfs:// supported}
$ hdfs dfs -ls -i /
Found 2 items
16386 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d1
16388 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d4
$ hdfs dfs -ls -i -R /
16386 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d1
16387 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d1/d2
16390 -rw-r--r-- 2 user1 supergroup117 2019-01-29 02:56 /d1/d2/f3
16388 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d4
16389 drwxr-xr-x - user1 supergroup 0 2019-01-29 02:56 /d4/d5
{code}
Doesn't support file:// at the moment, will always print 0 for fileId / inode
id at the moment:
{code:bash|title=file:// not supported yet}
$ hdfs dfs -ls -i file:///usr/
Found 9 items
0 drwxr-xr-x - root wheel288 2016-09-26 22:11 file:///usr/X11
0 drwxr-xr-x - root wheel288 2016-09-26 22:11 file:///usr/X11R6
0 drwxr-xr-x - root wheel 31104 2019-01-22 12:42 file:///usr/bin
0 drwxr-xr-x - root wheel 9728 2019-01-22 12:42 file:///usr/lib
0 drwxr-xr-x - root wheel 7968 2019-01-22 12:42 file:///usr/libexec
0 drwxr-xr-x - root wheel512 2018-11-06 16:01 file:///usr/local
0 drwxr-xr-x - root wheel 7648 2019-01-22 12:42 file:///usr/sbin
0 drwxr-xr-x - root wheel 1472 2018-10-19 22:12 file:///usr/share
0 drwxr-xr-x - root wheel160 2018-09-20 21:06 file:///usr/standalone
{code}
> FsShell ls: Add option -i to print inode id
> ---
>
> Key: HADOOP-16082
> URL: https://issues.apache.org/jira/browse/HADOOP-16082
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Affects Versions: 3.1.1, 3.2.0
>Reporter: Siyao Meng
>Assignee: Siyao Meng
>Priority: Major
> Attachments: HADOOP-16082.001.patch
>
>
> When debugging the FSImage corruption issue, I often need to know a file's or
> directory's inode id. At this moment, the only way to do that is to use OIV
> tool to dump the FSImage and look up the filename, which is very inefficient.
> Here I propose adding option "-i" in FsShell that prints files' or
> directories' inode id.
> h2. Implementation
> h3. For hdfs:// (HDFS)
> fileId exists in HdfsLocatedFileStatus, which is already returned to
> hdfs-client. We just need to print it in Ls#processPath().
> h3. For file:// (Local FS)
> h4. Linux
> Use java.nio.
> h4. Windows
> Windows has the concept of "File ID" which is similar to inode id. It is
> unique in NTFS and ReFS.
> h3. For other FS
> The fileId entry will be "0" in FileStatus if it is not set. We could either
> ignore or throw an exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HADOOP-16082: Affects Version/s: 3.1.1 > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.1.1 >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16082) FsShell ls: Add option -i to print inode id
[ https://issues.apache.org/jira/browse/HADOOP-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HADOOP-16082: Description: When debugging the FSImage corruption issue, I often need to know a file's or directory's inode id. At this moment, the only way to do that is to use OIV tool to dump the FSImage and look up the filename, which is very inefficient. Here I propose adding option "-i" in FsShell that prints files' or directories' inode id. h2. Implementation h3. For hdfs:// (HDFS) fileId exists in HdfsLocatedFileStatus, which is already returned to hdfs-client. We just need to print it in Ls#processPath(). h3. For file:// (Local FS) h4. Linux Use java.nio. h4. Windows Windows has the concept of "File ID" which is similar to inode id. It is unique in NTFS and ReFS. h3. For other FS The fileId entry will be "0" in FileStatus if it is not set. We could either ignore or throw an exception. was: When debugging the FSImage corruption issue, I often need to know a file's or directory's inode id. At this moment, the only way to do that is to use OIV tool to dump the FSImage and look up the filename, which is very inefficient. Here I propose adding option "-i" in FsShell that prints files' or directories' inode id. h2. Implementation h3. For hdfs:// (HDFS) fileId exists in HdfsLocatedFileStatus, which is already returned to hdfs-client. We just need to print it in Ls#processPath(). h3. For file:// h4. Linux Use java.nio. h4. Windows Windows has the concept of "File ID" which is similar to inode id. It is unique in NTFS and ReFS. h3. For other FS The fileId entry will be "0" in FileStatus if it is not set. We could either ignore or throw an exception. > FsShell ls: Add option -i to print inode id > --- > > Key: HADOOP-16082 > URL: https://issues.apache.org/jira/browse/HADOOP-16082 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Siyao Meng >Assignee: Siyao Meng >Priority: Major > > When debugging the FSImage corruption issue, I often need to know a file's or > directory's inode id. At this moment, the only way to do that is to use OIV > tool to dump the FSImage and look up the filename, which is very inefficient. > Here I propose adding option "-i" in FsShell that prints files' or > directories' inode id. > h2. Implementation > h3. For hdfs:// (HDFS) > fileId exists in HdfsLocatedFileStatus, which is already returned to > hdfs-client. We just need to print it in Ls#processPath(). > h3. For file:// (Local FS) > h4. Linux > Use java.nio. > h4. Windows > Windows has the concept of "File ID" which is similar to inode id. It is > unique in NTFS and ReFS. > h3. For other FS > The fileId entry will be "0" in FileStatus if it is not set. We could either > ignore or throw an exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
