[
https://issues.apache.org/jira/browse/HDDS-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591475#comment-16591475
]
Elek, Marton commented on HDDS-336:
-----------------------------------
Thank you very much the patch [~GeLiXin]. Overall it looks very good to me.
I have only two concerns:
1. One container could have multiple blocks and one block is defined with
containerid + localid pair. Because ContainerId is not unique the
Map<Long,Long> is not good enough to store the location information. (one
container could have (c1,l1), (c1,l2) blocks an in that case the map would do a
c1->l2 from this information)
It could be Map.Entry<Long, Long> but creating a specific class would be even
better, because it would be more clean what are meanings of the fields.
For example:
{code}
public class OzoneKeyLocation {
private final long containerId;
private final long localId;
private final long length;
private final long offset;
{code}
And List<Map<Long,Long>> could be changed to List<OzoneKeyLocation>.
(length and offsets are not hard requirement buts with getting this information
we can check if we have all the bytes for a specific key )
2. this could be finished in a separated jira, but I would add it to here as
this is the best part: it would be great to print out these information to the
console using the 'ozone oz' cli. As you modified all the required code part,
this is the easy endgame: ClientProtocol.getKeyDetails and OzoneBucket.getKey
could be modified to return with OzoneKeyDetails. And at the end of the
InfoKeyHandler we can print out the containerid->localid mapping.
it could be tested with the docker pseudo cluster:
1. do a full build
2. do a 'docker-compose up -d' in the hadoop-dist/target/ozone/compose/ozone
directory
3. do a 'docker-compose scale datanode=3' (just to have 3 datanodes)
4. wait until you can see 3 HEALTHY nodes on localhost:9876 (host could be
different for mac/windows)
5. use 'docker-compose exec datanode bash'
6. Try out the ozone oz cli:
{code}
ozone oz -createVolume /vol1 -user hadoop --quota 1TB --root
ozone oz -createBucket /vol1/bucket
dd if=/dev/zero of=/tmp/test bs=1024000 count=512
ozone oz -putKey /vol1/bucket/file1 -replicationFactor 3 -file /tmp/test
ozone oz -infoKey /vol1/bucket/file1
{code}
Thank you again the patch, this is a very precious implementation of the
original plan.
> Print out container location information for a specific ozone key
> ------------------------------------------------------------------
>
> Key: HDDS-336
> URL: https://issues.apache.org/jira/browse/HDDS-336
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: SCM
> Reporter: Elek, Marton
> Assignee: LiXin Ge
> Priority: Major
> Labels: newbie
> Fix For: 0.2.1
>
> Attachments: HDDS-336.000.patch, HDDS-336.001.patch,
> HDDS-336.002.patch
>
>
> In the protobuf protocol we have all the containerid/localid(=blockid)
> information for a specific ozone key.
> It would be a big help to print out this information to the command line with
> the ozone cli.
> It requires to improve the REST and RPC interface with additionalOzone
> KeyLocation information.
> It would help a very big help during the test of the current scm behaviour.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
