[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[
https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054822#comment-16054822
]
Anu Engineer commented on HDFS-11997:
-
[~vagarychen] Thanks for your comments. You are absolutely correct. Chunk
Manager never really needs to know which key it is part of. However, some time
ago we did a prototype for Disaster Recovery (DR) using the containers. In that
prototype we explored writing the chunks directly from the datanode to S3
buckets.
If my memory serves me correct, that is when we added this. Since having the
context made it easier to write the chunk to the remote cluster. If we decide
to do that in future this might be useful.
You are absolutely right from a layering perspective, this does not make sense
-- A chunk never needs to know the parent context. This was only for
propagating that information over to another cluster.
Right now, we have no strong use case for this and DR has to be re-developed
anyway (probably will not need this info for that). So this is prototyping
residue that can indeed be removed. Please feel free to remove it. Completely
your call.
Thanks
> ChunkManager functions do not use the argument keyName
> --
>
> Key: HDFS-11997
> URL: https://issues.apache.org/jira/browse/HDFS-11997
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>
> {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}}
> {{deleteChunk}} all take a {{keyName}} argument, which is not being used by
> any of them.
> I think this makes sense because conceptually {{ChunkManager}} should not
> have to know keyName to do anything, probably except for some sort of sanity
> check or logging, which is not there either. We should revisit whether we
> need it here. I think we should remove it to make the Chunk syntax, and the
> function signatures more cleanly abstracted.
> Any comments? [~anu]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[
https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054779#comment-16054779
]
Chen Liang commented on HDFS-11997:
---
I think from the perspective of abstraction, {{ChunkManager}} should work
(read/write/delete a chunk) given just the metadata of the chunk. This is not
causing any issue for now and will more likely never, but I felt having this
field but not being used causes confusions. I simply didn't see any cases where
this field should be used by ChunkManager as part of any of the operations. In
fact, an implementation of chunk manager that relies on key name seems breaking
the abstraction in some way to me...
> ChunkManager functions do not use the argument keyName
> --
>
> Key: HDFS-11997
> URL: https://issues.apache.org/jira/browse/HDFS-11997
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>
> {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}}
> {{deleteChunk}} all take a {{keyName}} argument, which is not being used by
> any of them.
> I think this makes sense because conceptually {{ChunkManager}} should not
> have to know keyName to do anything, probably except for some sort of sanity
> check or logging, which is not there either. We should revisit whether we
> need it here. I think we should remove it to make the Chunk syntax, and the
> function signatures more cleanly abstracted.
> Any comments? [~anu]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[
https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054754#comment-16054754
]
Anu Engineer commented on HDFS-11997:
-
As an API for chunk calls, isn't it better to have these in the API ?.
This particular implementation of containers is not using it. But many others
might find it useful. If it is just part of signature, I don't know if it is
causing any issue.
> ChunkManager functions do not use the argument keyName
> --
>
> Key: HDFS-11997
> URL: https://issues.apache.org/jira/browse/HDFS-11997
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>
> {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}}
> {{deleteChunk}} all take a {{keyName}} argument, which is not being used by
> any of them.
> I think this makes sense because conceptually {{ChunkManager}} should not
> have to know keyName to do anything, probably except for some sort of sanity
> check or logging, which is not there either. We should revisit whether we
> need it here. I think we should remove it to make the Chunk syntax, and the
> function signatures more cleanly abstracted.
> Any comments? [~anu]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
