[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273470#comment-16273470
]
Rae Marks edited comment on HDFS-12873 at 11/30/17 10:53 PM:
-
Oops, not very descriptive of me. I meant if you try to get the file status of
a file using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, getFileStatus on {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's info ({{x}}) - it just
throws away {{I/don't/exist}}.
was (Author: raemarks):
Oops, not very descriptive. I meant if you try to get the file status of a file
using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, getFileStatus on {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's info ({{x}}) - it just
throws away {{I/don't/exist}}.
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Rae Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an exception. GetListing on '/.reserved/.inodes/'
> shows '..', while GetListing on '/x/y' does not.
> Mkdirs INotify events were reported with the following paths, in order:
> /x
> /x/y
> /x/y/z
> /x/y/z/..
> /x/y/z/../foo
> I can also chain these dotdot directories and make them as deep as I want.
> Mkdirs works with the following paths appended to the inode path for
> directory y: '/z/../../../foo', '/z/../../../../../',
> '/z/../../../foo/bar/../..' etc, and it constructs all the '..' directories
> as if they weren't special names.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273470#comment-16273470
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 9:38 PM:
--
Oops, not very descriptive. I meant if you try to get the file status of a file
using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, getFileStatus on {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's info ({{x}}) - it just
throws away {{I/don't/exist}}.
was (Author: raemarks):
Oops, not very descriptive. I meant if you try to get the file status of a file
using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's inode number ({{x}}) - it
just throws away {{I/don't/exist}}.
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an exception. GetListing on '/.reserved/.inodes/'
> shows '..', while GetListing on '/x/y' does not.
> Mkdirs INotify events were reported with the following paths, in order:
> /x
> /x/y
> /x/y/z
> /x/y/z/..
> /x/y/z/../foo
> I can also chain these dotdot directories and make them as deep as I want.
> Mkdirs works with the following paths appended to the inode path for
> directory y: '/z/../../../foo', '/z/../../../../../',
> '/z/../../../foo/bar/../..' etc, and it constructs all the '..' directories
> as if they weren't special names.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273470#comment-16273470
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 9:29 PM:
--
Oops, not very descriptive. I meant if you try to get the file status of a file
using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's inode number ({{x}}) - it
just throws away {{I/don't/exist}}.
was (Author: raemarks):
Oops, not very descriptive. I meant if you try to get the file status of a file
using a path containing {{..}} after the 4th index, but I was recalling
something different and was incorrect. Please disregard that comment.
What I mentioned about ignoring anything after the {{..}} in the 4th index is
true though. For example, {{'/.reserved/.inodes//../I/don't/exist}} would return y's parent's inode number (x) - it just
throws away {{I/don't/exist}}.
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an exception. GetListing on '/.reserved/.inodes/'
> shows '..', while GetListing on '/x/y' does not.
> Mkdirs INotify events were reported with the following paths, in order:
> /x
> /x/y
> /x/y/z
> /x/y/z/..
> /x/y/z/../foo
> I can also chain these dotdot directories and make them as deep as I want.
> Mkdirs works with the following paths appended to the inode path for
> directory y: '/z/../../../foo', '/z/../../../../../',
> '/z/../../../foo/bar/../..' etc, and it constructs all the '..' directories
> as if they weren't special names.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273122#comment-16273122
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 6:33 PM:
--
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{code}
# hadoop version
Hadoop 2.8.0
{code}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{code}
>> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
>> auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
>> bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
}
{code}
was (Author: raemarks):
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{code}
# hadoop version
Hadoop 2.8.0
{code}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{code}
>> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
>> auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
}
{code}
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
>
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273122#comment-16273122
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 6:31 PM:
--
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{code}
# hadoop version
Hadoop 2.8.0
{code}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{{ >> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
} }}
was (Author: raemarks):
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{{# hadoop version
Hadoop 2.8.0}}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{{ >> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
} }}
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an exception. GetListing on
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273122#comment-16273122
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 6:31 PM:
--
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{code}
# hadoop version
Hadoop 2.8.0
{code}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{code}
>> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
>> auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
}
{code}
was (Author: raemarks):
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{code}
# hadoop version
Hadoop 2.8.0
{code}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{{ >> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
} }}
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an
[jira] [Comment Edited] (HDFS-12873) Creating a '..' directory is possible using inode paths
[
https://issues.apache.org/jira/browse/HDFS-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273122#comment-16273122
]
Raeanne J Marks edited comment on HDFS-12873 at 11/30/17 6:29 PM:
--
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{{# hadoop version
Hadoop 2.8.0}}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{{>> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
}}}
was (Author: raemarks):
[~shahrs87] By it worked as expected, do you mean it compressed the path to
{{/user/rushabhs/benchmarks/test2}}, or threw an exception? I can confirm I'm
seeing this in {{2.8.0}}:
{{# hadoop version
Hadoop 2.8.0}}
I am using a snakebite-like hadoop client. It does not perform any client-side
path manipulation/validation - it relies on the NameNode to properly
handle/report unsupported path formats. Here is my script:
{{>> client = pydoofus.namenode.v9.Client('172.18.0.2', 8020,
auth={'effective_user': 'hdfs'})
>> print client.mkdirs('/x/y/z', 0777, create_parent=True)
True
>> file_status = client.get_file_info('/x/y')
>> print file_status
FileStatus {
file_type: DIRECTORY
path:
length: 0
permission:
Permission {
mode: 0777
}
owner: hdfs
group: supergroup
modification_time: 1512066397076
access_time: 0
symlink:
replication: 0
block_size: 0
locations: None
file_id: 16580
num_children: 1
}
bad_path = '/.reserved/.inodes/' + str(file_status.file_id) + '/z/../foo'
>> print client.mkdirs(bad_path, 0777, create_parent=True)
True
>> print client.get_listing('/x/y/z')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'..', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16582L, num_children=1)]
remaining: 0
}
>> print client.get_listing('/x/y/z/..')
DirectoryListing {
entries: [FileStatus(file_type='DIRECTORY', path=u'foo', length=0L,
permission=Permission(mode=0777), owner=u'hdfs', group=u'supergroup',
modification_time=1512066397083, access_time=0, symlink='', replication=0,
block_size=0L, locations=None, file_id=16583L, num_children=0)]
remaining: 0
}}}
> Creating a '..' directory is possible using inode paths
> ---
>
> Key: HDFS-12873
> URL: https://issues.apache.org/jira/browse/HDFS-12873
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Affects Versions: 2.8.0
> Environment: Apache NameNode running in a Docker container on a
> Fedora 25 workstation.
>Reporter: Raeanne J Marks
>
> Start with a fresh deployment of HDFS.
> 1. Mkdirs '/x/y/z'
> 2. use GetFileInfo to get y's inode number
> 3. Mkdirs '/.reserved/.inodes//z/../foo'
> Expectation: The path in step 3 is rejected as invalid (exception thrown) OR
> foo would be created under y.
> Observation: This created a directory called '..' under z and 'foo' under
> that '..' directory instead of consolidating the path to '/x/y/foo' or
> throwing an exception. GetListing on
