[jira] [Commented] (HDFS-13395) Ozone: Plugins support in HDSL Datanode Service
[
https://issues.apache.org/jira/browse/HDFS-13395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430330#comment-16430330
]
Shashikant Banerjee commented on HDFS-13395:
+1,
Thanks [~nandakumar131], for the updated patch . I have tested it locally by
running freon tests and it worked fine.
> Ozone: Plugins support in HDSL Datanode Service
> ---
>
> Key: HDFS-13395
> URL: https://issues.apache.org/jira/browse/HDFS-13395
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Nanda kumar
>Priority: Major
> Attachments: HDFS-13395-HDFS-7240.000.patch,
> HDFS-13395-HDFS-7240.001.patch, HDFS-13395-HDFS-7240.002.patch
>
>
> As part of Datanode, we start {{HdslDatanodeService}} if {{ozone}} is
> enabled. We need provision to load plugins like {{Ozone Rest Service}} as
> part of {{HdslDatanodeService}} start.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13342: --- Attachment: HDFS-13342-HDFS-7240.005.patch > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch, > HDFS-13342-HDFS-7240.002.patch, HDFS-13342-HDFS-7240.003.patch, > HDFS-13342-HDFS-7240.004.patch, HDFS-13342-HDFS-7240.005.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427859#comment-16427859 ] Shashikant Banerjee commented on HDFS-13342: Thanks [~elek] for the review comments. As per your review comments and offline discussion, i have updated the patch. > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch, > HDFS-13342-HDFS-7240.002.patch, HDFS-13342-HDFS-7240.003.patch, > HDFS-13342-HDFS-7240.004.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13342: --- Attachment: HDFS-13342-HDFS-7240.004.patch > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch, > HDFS-13342-HDFS-7240.002.patch, HDFS-13342-HDFS-7240.003.patch, > HDFS-13342-HDFS-7240.004.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16428104#comment-16428104 ] Shashikant Banerjee commented on HDFS-13342: Patch v5 addresses the acceptance tests issues. > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch, > HDFS-13342-HDFS-7240.002.patch, HDFS-13342-HDFS-7240.003.patch, > HDFS-13342-HDFS-7240.004.patch, HDFS-13342-HDFS-7240.005.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430361#comment-16430361 ] Shashikant Banerjee commented on HDFS-12794: Rebased patch v12 to latest repo. Need HDFS-13395 for running the Freon tests added. > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12794: --- Attachment: HDFS-12794-HDFS-7240.012.patch > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12794: --- Attachment: (was: HDFS-12794-HDFS-7240.012.patch) > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12794: --- Attachment: HDFS-12794-HDFS-7240.012.patch > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430367#comment-16430367 ] Shashikant Banerjee commented on HDFS-12794: Reuploaded correct patch v12. > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12794: --- Attachment: (was: HDFS-12794-HDFS-7240.012.patch) > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13438) Fix javadoc in FsVolumeList#removeVolume
[ https://issues.apache.org/jira/browse/HDFS-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13438: --- Attachment: (was: HDFS-13438.000.patch) > Fix javadoc in FsVolumeList#removeVolume > > > Key: HDFS-13438 > URL: https://issues.apache.org/jira/browse/HDFS-13438 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13438.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13438) Fix javadoc in FsVolumeList#removeVolume
[ https://issues.apache.org/jira/browse/HDFS-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13438: --- Status: Patch Available (was: Open) > Fix javadoc in FsVolumeList#removeVolume > > > Key: HDFS-13438 > URL: https://issues.apache.org/jira/browse/HDFS-13438 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13438.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13438) Fix javadoc in FsVolumeList#removeVolume
[ https://issues.apache.org/jira/browse/HDFS-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13438: --- Attachment: HDFS-13438.000.patch > Fix javadoc in FsVolumeList#removeVolume > > > Key: HDFS-13438 > URL: https://issues.apache.org/jira/browse/HDFS-13438 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13438.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13438) Fix javadoc in FsVolumeList#removeVolume
[ https://issues.apache.org/jira/browse/HDFS-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436843#comment-16436843 ] Shashikant Banerjee commented on HDFS-13438: Resubmitting the patch to trigger jenkins run. > Fix javadoc in FsVolumeList#removeVolume > > > Key: HDFS-13438 > URL: https://issues.apache.org/jira/browse/HDFS-13438 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13438.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13438) Fix javadoc in FsVolumeList#removeVolume
[ https://issues.apache.org/jira/browse/HDFS-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13438: --- Status: Open (was: Patch Available) > Fix javadoc in FsVolumeList#removeVolume > > > Key: HDFS-13438 > URL: https://issues.apache.org/jira/browse/HDFS-13438 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13438.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13413) ClusterId and DatanodeUuid should be marked mandatory fileds in SCMRegisteredCmdResponseProto
[
https://issues.apache.org/jira/browse/HDFS-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13413:
---
Attachment: (was: HDFS-13413-HDFS-7240.000.patch)
> ClusterId and DatanodeUuid should be marked mandatory fileds in
> SCMRegisteredCmdResponseProto
> -
>
> Key: HDFS-13413
> URL: https://issues.apache.org/jira/browse/HDFS-13413
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Minor
> Fix For: HDFS-7240
>
>
> ClusterId as well as the DatanodeUuid are optional fields in
> {{SCMRegisteredCmdResponseProto}}
> currently. We have to make both clusterId and DatanodeUuid as required field
> and handle it properly. As of now, we don't do anything with the response of
> datanode registration. We should validate the clusterId and also the
> datanodeUuid
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13413) ClusterId and DatanodeUuid should be marked mandatory fileds in SCMRegisteredCmdResponseProto
[
https://issues.apache.org/jira/browse/HDFS-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13413:
---
Attachment: HDFS-13413-HDFS-7240.000.patch
> ClusterId and DatanodeUuid should be marked mandatory fileds in
> SCMRegisteredCmdResponseProto
> -
>
> Key: HDFS-13413
> URL: https://issues.apache.org/jira/browse/HDFS-13413
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Minor
> Fix For: HDFS-7240
>
> Attachments: HDFS-13413-HDFS-7240.000.patch
>
>
> ClusterId as well as the DatanodeUuid are optional fields in
> {{SCMRegisteredCmdResponseProto}}
> currently. We have to make both clusterId and DatanodeUuid as required field
> and handle it properly. As of now, we don't do anything with the response of
> datanode registration. We should validate the clusterId and also the
> datanodeUuid
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13413) ClusterId and DatanodeUuid should be marked mandatory fileds in SCMRegisteredCmdResponseProto
[
https://issues.apache.org/jira/browse/HDFS-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13413:
---
Attachment: HDFS-13413-HDFS-7240.001.patch
> ClusterId and DatanodeUuid should be marked mandatory fileds in
> SCMRegisteredCmdResponseProto
> -
>
> Key: HDFS-13413
> URL: https://issues.apache.org/jira/browse/HDFS-13413
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Minor
> Fix For: HDFS-7240
>
> Attachments: HDFS-13413-HDFS-7240.000.patch,
> HDFS-13413-HDFS-7240.001.patch
>
>
> ClusterId as well as the DatanodeUuid are optional fields in
> {{SCMRegisteredCmdResponseProto}}
> currently. We have to make both clusterId and DatanodeUuid as required field
> and handle it properly. As of now, we don't do anything with the response of
> datanode registration. We should validate the clusterId and also the
> datanodeUuid
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13413) ClusterId and DatanodeUuid should be marked mandatory fileds in SCMRegisteredCmdResponseProto
[
https://issues.apache.org/jira/browse/HDFS-13413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434997#comment-16434997
]
Shashikant Banerjee commented on HDFS-13413:
Thanks [~bharatviswa], for the review. Patch v1 addresses your review comments.
Please have a look.
> ClusterId and DatanodeUuid should be marked mandatory fileds in
> SCMRegisteredCmdResponseProto
> -
>
> Key: HDFS-13413
> URL: https://issues.apache.org/jira/browse/HDFS-13413
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Minor
> Fix For: HDFS-7240
>
> Attachments: HDFS-13413-HDFS-7240.000.patch,
> HDFS-13413-HDFS-7240.001.patch
>
>
> ClusterId as well as the DatanodeUuid are optional fields in
> {{SCMRegisteredCmdResponseProto}}
> currently. We have to make both clusterId and DatanodeUuid as required field
> and handle it properly. As of now, we don't do anything with the response of
> datanode registration. We should validate the clusterId and also the
> datanodeUuid
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13172) Impelement task Manager to handle create and delete multiLevel nodes is SnapshotSkipList
[ https://issues.apache.org/jira/browse/HDFS-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440649#comment-16440649 ] Shashikant Banerjee edited comment on HDFS-13172 at 4/17/18 9:17 AM: - Inline deletion of multilevel nodes in snapshotSkipList is quite fast. Moreover, async deletion of multilevel nodes from snapshotSkipList containing snapshotDiffs leads to complicated scenarios of maintaining quota consistency semantics in HDFS. It won't be done for now. was (Author: shashikant): Inline deletion of multilevel nodes in snapshotSkipList is quite fast. Moreover, async deletion of multilevel nodes from snapshotSkipList containing snapshotDiffs leads to complicated scenarios of maintaining quota consistency semantics in HDFS. It won't be done for now. > Impelement task Manager to handle create and delete multiLevel nodes is > SnapshotSkipList > > > Key: HDFS-13172 > URL: https://issues.apache.org/jira/browse/HDFS-13172 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > > This Jira proposes to add a Taskmanager in SnapshotManager. The TaskManager > will maintain a TaskQueue. Each task in the task queue will handle creation > of multiple levels of a node or or balancing of the list in case of deletion > of a node in the SnapshotSkipList. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13464) Fix javadoc in FsVolumeList#handleVolumeFailures
[ https://issues.apache.org/jira/browse/HDFS-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13464: --- Status: Patch Available (was: Open) > Fix javadoc in FsVolumeList#handleVolumeFailures > > > Key: HDFS-13464 > URL: https://issues.apache.org/jira/browse/HDFS-13464 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13464.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13464) Fix javadoc in FsVolumeList#handleVolumeFailures
[ https://issues.apache.org/jira/browse/HDFS-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13464: --- Attachment: HDFS-13464.000.patch > Fix javadoc in FsVolumeList#handleVolumeFailures > > > Key: HDFS-13464 > URL: https://issues.apache.org/jira/browse/HDFS-13464 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13464.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13464) Fix javadoc in FsVolumeList#handleVolumeFailures
[ https://issues.apache.org/jira/browse/HDFS-13464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440607#comment-16440607 ] Shashikant Banerjee commented on HDFS-13464: Patch vo fixes the javadoc for FsVolumeList#handleVolumeFailures. > Fix javadoc in FsVolumeList#handleVolumeFailures > > > Key: HDFS-13464 > URL: https://issues.apache.org/jira/browse/HDFS-13464 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13464.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13464) Fix javadoc in FsVolumeList#handleVolumeFailures
Shashikant Banerjee created HDFS-13464: -- Summary: Fix javadoc in FsVolumeList#handleVolumeFailures Key: HDFS-13464 URL: https://issues.apache.org/jira/browse/HDFS-13464 Project: Hadoop HDFS Issue Type: Bug Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-13172) Impelement task Manager to handle create and delete multiLevel nodes is SnapshotSkipList
[ https://issues.apache.org/jira/browse/HDFS-13172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee resolved HDFS-13172. Resolution: Won't Do Inline deletion of multilevel nodes in snapshotSkipList is quite fast. Moreover, async deletion of multilevel nodes from snapshotSkipList containing snapshotDiffs leads to complicated scenarios of maintaining quota consistency semantics in HDFS. It won't be done for now. > Impelement task Manager to handle create and delete multiLevel nodes is > SnapshotSkipList > > > Key: HDFS-13172 > URL: https://issues.apache.org/jira/browse/HDFS-13172 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > > This Jira proposes to add a Taskmanager in SnapshotManager. The TaskManager > will maintain a TaskQueue. Each task in the task queue will handle creation > of multiple levels of a node or or balancing of the list in case of deletion > of a node in the SnapshotSkipList. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13463) Fix javadoc in FsDatasetImpl#checkAndUpdate
Shashikant Banerjee created HDFS-13463: -- Summary: Fix javadoc in FsDatasetImpl#checkAndUpdate Key: HDFS-13463 URL: https://issues.apache.org/jira/browse/HDFS-13463 Project: Hadoop HDFS Issue Type: Bug Components: datanode Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13463) Fix javadoc in FsDatasetImpl#checkAndUpdate
[ https://issues.apache.org/jira/browse/HDFS-13463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13463: --- Status: Patch Available (was: Open) > Fix javadoc in FsDatasetImpl#checkAndUpdate > --- > > Key: HDFS-13463 > URL: https://issues.apache.org/jira/browse/HDFS-13463 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13463.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13463) Fix javadoc in FsDatasetImpl#checkAndUpdate
[ https://issues.apache.org/jira/browse/HDFS-13463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440628#comment-16440628 ] Shashikant Banerjee commented on HDFS-13463: Patch vo addresss the javadoc issue in FsDatasetImpl#checkAndUpdate. > Fix javadoc in FsDatasetImpl#checkAndUpdate > --- > > Key: HDFS-13463 > URL: https://issues.apache.org/jira/browse/HDFS-13463 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13463.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13463) Fix javadoc in FsDatasetImpl#checkAndUpdate
[ https://issues.apache.org/jira/browse/HDFS-13463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13463: --- Attachment: HDFS-13463.000.patch > Fix javadoc in FsDatasetImpl#checkAndUpdate > --- > > Key: HDFS-13463 > URL: https://issues.apache.org/jira/browse/HDFS-13463 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Minor > Attachments: HDFS-13463.000.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Status: Open (was: Patch Available)
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Attachment: (was: HDFS-13454-HDFS-7240.000.patch)
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Status: Patch Available (was: Open)
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13454-HDFS-7240.000.patch
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Attachment: HDFS-13454-HDFS-7240.000.patch
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13454-HDFS-7240.000.patch
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440715#comment-16440715
]
Shashikant Banerjee commented on HDFS-13454:
patch v0 fixes the test logic in TestKeySpaceManager#testDeleteKey.
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13454-HDFS-7240.000.patch
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Status: Patch Available (was: Open)
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13454-HDFS-7240.000.patch
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
[
https://issues.apache.org/jira/browse/HDFS-13454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13454:
---
Attachment: HDFS-13454-HDFS-7240.000.patch
> Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
> ---
>
> Key: HDFS-13454
> URL: https://issues.apache.org/jira/browse/HDFS-13454
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13454-HDFS-7240.000.patch
>
>
> The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The
> test validates the keyArgs instead of blockId to make sure the key gets
> deleted from SCM. Also, after the first exception validation , the subsequent
> statements in the junit never gets executed here.
> {code:java}
> keys.add(keyArgs.getResourceName());
> exception.expect(IOException.class);
> exception.expectMessage("Specified block key does not exist");
> cluster.getStorageContainerManager().getBlockLocations(keys);
> // Delete the key again to test deleting non-existing key.
> // These will never get executed.
> exception.expect(IOException.class);
> exception.expectMessage("KEY_NOT_FOUND");
> storageHandler.deleteKey(keyArgs);
> Assert.assertEquals(1 + numKeyDeleteFails,
> ksmMetrics.getNumKeyDeletesFails());{code}
> The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path
[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451990#comment-16451990 ] Shashikant Banerjee commented on HDFS-13489: Thanks [~hpatro], for updating the patch. Patch v2 looks good to me. Please address the checkStyle issues . > Get base snapshotable path if exists for a given path > - > > Key: HDFS-13489 > URL: https://issues.apache.org/jira/browse/HDFS-13489 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs >Reporter: Harkrishn Patro >Assignee: Harkrishn Patro >Priority: Major > Attachments: HDFS-13489.001.patch, HDFS-13489.002.patch > > > Currently, hdfs only lists the snapshotable paths in the filesystem. This > feature would add the functionality of figuring out if a given path is > snapshotable or not. If yes, it would return the base snapshotable path. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path
[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447677#comment-16447677 ] Shashikant Banerjee commented on HDFS-13489: Thanks [~hpatro], for filing and working on this Jira. I just had a quick look at this. A given path can resolve to a directory , file etc. I think in the current patch, if the path resolves to anything other than a directory e, its not handled.Is it a hard requirement here that the given path provided has to resolve to a directory? > Get base snapshotable path if exists for a given path > - > > Key: HDFS-13489 > URL: https://issues.apache.org/jira/browse/HDFS-13489 > Project: Hadoop HDFS > Issue Type: Task > Components: hdfs >Reporter: Harkrishn Patro >Assignee: Harkrishn Patro >Priority: Major > Attachments: HDFS-13489.001.patch > > > Currently, hdfs only lists the snapshotable paths in the filesystem. This > feature would add the functionality of figuring out if a given path is > snapshotable or not. If yes, it would return the base snapshotable path. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13454) Ozone : Fix the test logic in TestKeySpaceManager#testDeleteKey
Shashikant Banerjee created HDFS-13454:
--
Summary: Ozone : Fix the test logic in
TestKeySpaceManager#testDeleteKey
Key: HDFS-13454
URL: https://issues.apache.org/jira/browse/HDFS-13454
Project: Hadoop HDFS
Issue Type: Sub-task
Components: ozone
Reporter: Shashikant Banerjee
Assignee: Shashikant Banerjee
Fix For: HDFS-7240
The test logic in TestKeySpaceManager#testDeleteKey seems to be wrong. The test
validates the keyArgs instead of blockId to make sure the key gets deleted from
SCM. Also, after the first exception validation , the subsequent statements in
the junit never gets executed here.
{code:java}
keys.add(keyArgs.getResourceName());
exception.expect(IOException.class);
exception.expectMessage("Specified block key does not exist");
cluster.getStorageContainerManager().getBlockLocations(keys);
// Delete the key again to test deleting non-existing key.
// These will never get executed.
exception.expect(IOException.class);
exception.expectMessage("KEY_NOT_FOUND");
storageHandler.deleteKey(keyArgs);
Assert.assertEquals(1 + numKeyDeleteFails,
ksmMetrics.getNumKeyDeletesFails());{code}
The test needs to be modified to address all these.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13361) Ozone: Remove commands from command queue when the datanode is declared dead
Shashikant Banerjee created HDFS-13361: -- Summary: Ozone: Remove commands from command queue when the datanode is declared dead Key: HDFS-13361 URL: https://issues.apache.org/jira/browse/HDFS-13361 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone Affects Versions: HDFS-7240 Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee Fix For: HDFS-7240 SCM can queue commands for Datanodes to pickup. However, a dead datanode may never pick up the commands.The command queue needs to be cleaned for the datanode once its declared dead. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
Shashikant Banerjee created HDFS-13368: -- Summary: Ozone:TestEndPoint tests are failing consistently Key: HDFS-13368 URL: https://issues.apache.org/jira/browse/HDFS-13368 Project: Hadoop HDFS Issue Type: Bug Components: ozone Affects Versions: HDFS-7240 Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee Fix For: HDFS-7240 With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file made required fiields. These parameters are not set in TestEndPoint which lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[
https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418647#comment-16418647
]
Shashikant Banerjee edited comment on HDFS-13320 at 3/29/18 9:19 AM:
-
Thanks [~elek], for having a look at this. Strangely, it shows no conflict for
me.
{code:java}
HW15685:hadoop sbanerjee$ git status
On branch HDFS-7240
Your branch is up-to-date with 'origin/HDFS-7240'.
nothing to commit, working tree clean
HW15685:hadoop sbanerjee$ git apply ~/Desktop/HDFS-13320-HDFS-7240.002.patch
HW15685:hadoop sbanerjee${code}
was (Author: shashikant):
Thanks [~elek], for having a look at this. Strangely, it shows no conflict for
me.
{code:java}
HW15685:hadoop sbanerjee$ git status
On branch HDFS-7240
Your branch is up-to-date with 'origin/HDFS-7240'.
nothing to commit, working tree clean
HW15685:hadoop sbanerjee$ git apply ~/Desktop/HDFS-13320-HDFS-7240.002.patch
HW15685:hadoop sbanerjee${code}
> Ozone:Support for MicrobenchMarking Tool
>
>
> Key: HDFS-13320
> URL: https://issues.apache.org/jira/browse/HDFS-13320
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13320-HDFS-7240.001.patch,
> HDFS-13320-HDFS-7240.002.patch
>
>
> This Jira proposes to add a micro benchmarking tool called Genesis which
> executes a set of HDSL/Ozone benchmarks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[
https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418647#comment-16418647
]
Shashikant Banerjee commented on HDFS-13320:
Thanks [~elek], for having a look at this. Strangely, it shows no conflict for
me.
{code:java}
HW15685:hadoop sbanerjee$ git status
On branch HDFS-7240
Your branch is up-to-date with 'origin/HDFS-7240'.
nothing to commit, working tree clean
HW15685:hadoop sbanerjee$ git apply ~/Desktop/HDFS-13320-HDFS-7240.002.patch
HW15685:hadoop sbanerjee${code}
> Ozone:Support for MicrobenchMarking Tool
>
>
> Key: HDFS-13320
> URL: https://issues.apache.org/jira/browse/HDFS-13320
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13320-HDFS-7240.001.patch,
> HDFS-13320-HDFS-7240.002.patch
>
>
> This Jira proposes to add a micro benchmarking tool called Genesis which
> executes a set of HDSL/Ozone benchmarks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418738#comment-16418738 ] Shashikant Banerjee commented on HDFS-13320: Thanks [~elek], for reviewing this. Patch v3 fixes the compilation issues. Please have a look. > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch, HDFS-13320-HDFS-7240.003.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Attachment: (was: HDFS-13368.000.patch) > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368-HDFS-7240.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Attachment: HDFS-13368-HDFS-7240.000.patch > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368-HDFS-7240.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Attachment: (was: HDFS-13368-HDFS-7240.000.patch) > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368-HDFS-7240.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13348) Ozone: Update IP and hostname in Datanode from SCM's response to the register call
[
https://issues.apache.org/jira/browse/HDFS-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13348:
---
Attachment: HDFS-13348-HDFS-7240.000.patch
> Ozone: Update IP and hostname in Datanode from SCM's response to the register
> call
> --
>
> Key: HDFS-13348
> URL: https://issues.apache.org/jira/browse/HDFS-13348
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13348-HDFS-7240.000.patch
>
>
> Whenever a Datanode registers with SCM, the SCM resolves the IP address and
> hostname of the Datanode form the RPC call. This IP address and hostname
> should be sent back to Datanode in the response to register call and the
> Datanode has to update the values from the response to its
> {{DatanodeDetails}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Attachment: HDFS-13368.000.patch > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Status: Patch Available (was: Open) > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13348) Ozone: Update IP and hostname in Datanode from SCM's response to the register call
[
https://issues.apache.org/jira/browse/HDFS-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418778#comment-16418778
]
Shashikant Banerjee commented on HDFS-13348:
Patch v0 updates the hostName and DataNode IP in the datanode registration
response. It is dependent on HDFS-13368. Not submitting it for now.
> Ozone: Update IP and hostname in Datanode from SCM's response to the register
> call
> --
>
> Key: HDFS-13348
> URL: https://issues.apache.org/jira/browse/HDFS-13348
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13348-HDFS-7240.000.patch
>
>
> Whenever a Datanode registers with SCM, the SCM resolves the IP address and
> hostname of the Datanode form the RPC call. This IP address and hostname
> should be sent back to Datanode in the response to register call and the
> Datanode has to update the values from the response to its
> {{DatanodeDetails}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Attachment: HDFS-13320-HDFS-7240.003.patch > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch, HDFS-13320-HDFS-7240.003.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418766#comment-16418766 ] Shashikant Banerjee commented on HDFS-13368: Renamed the patch to correct format. > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368-HDFS-7240.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[ https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13368: --- Attachment: HDFS-13368-HDFS-7240.000.patch > Ozone:TestEndPoint tests are failing consistently > - > > Key: HDFS-13368 > URL: https://issues.apache.org/jira/browse/HDFS-13368 > Project: Hadoop HDFS > Issue Type: Bug > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13368-HDFS-7240.000.patch, > HDFS-13368-HDFS-7240.000.patch > > > With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file > made required fiields. These parameters are not set in TestEndPoint which > lead these to fail consistently. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13324) Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails
[ https://issues.apache.org/jira/browse/HDFS-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422234#comment-16422234 ] Shashikant Banerjee commented on HDFS-13324: patch v1 depends on HDFS-13301 . Not submitting it for now. > Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails > -- > > Key: HDFS-13324 > URL: https://issues.apache.org/jira/browse/HDFS-13324 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Shashikant Banerjee >Priority: Major > Labels: newbie > Attachments: HDFS-13324-HDFS-7240.000.patch > > > We have removed the dependency of DatanodeID in HDSL/Ozone and there is no > need for InfoPort and InfoSecurePort. It is now safe to remove InfoPort and > InfoSecurePort from DatanodeDetails. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13324) Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails
[ https://issues.apache.org/jira/browse/HDFS-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13324: --- Attachment: HDFS-13324-HDFS-7240.000.patch > Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails > -- > > Key: HDFS-13324 > URL: https://issues.apache.org/jira/browse/HDFS-13324 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Shashikant Banerjee >Priority: Major > Labels: newbie > Attachments: HDFS-13324-HDFS-7240.000.patch > > > We have removed the dependency of DatanodeID in HDSL/Ozone and there is no > need for InfoPort and InfoSecurePort. It is now safe to remove InfoPort and > InfoSecurePort from DatanodeDetails. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13325) Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
[
https://issues.apache.org/jira/browse/HDFS-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422202#comment-16422202
]
Shashikant Banerjee commented on HDFS-13325:
patch v1 renames ObjectStoreRestPlugin to OzoneDatanodeService and removes
DataNodeServicePlugin.
> Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
> ---
>
> Key: HDFS-13325
> URL: https://issues.apache.org/jira/browse/HDFS-13325
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13325-HDFS-7240.000.patch
>
>
> Based on this comment, we can rename {{ObjectStoreRestPlugin}} to
> {{OzoneDatanodeService}}. So that the plugin name will be consistant with
> {{HdslDatanodeService}}. We can also remove {{DataNodeServicePlugin}} and
> directly use {{ServicePlugin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13325) Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
[
https://issues.apache.org/jira/browse/HDFS-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13325:
---
Status: Patch Available (was: Open)
> Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
> ---
>
> Key: HDFS-13325
> URL: https://issues.apache.org/jira/browse/HDFS-13325
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13325-HDFS-7240.000.patch
>
>
> Based on this comment, we can rename {{ObjectStoreRestPlugin}} to
> {{OzoneDatanodeService}}. So that the plugin name will be consistant with
> {{HdslDatanodeService}}. We can also remove {{DataNodeServicePlugin}} and
> directly use {{ServicePlugin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13301) Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto
[
https://issues.apache.org/jira/browse/HDFS-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422214#comment-16422214
]
Shashikant Banerjee commented on HDFS-13301:
Patch v1 removes the HDSL/Ozone related changes from hdfs proto and DatanodeID.
> Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and
> DatanodeIDProto
> -
>
> Key: HDFS-13301
> URL: https://issues.apache.org/jira/browse/HDFS-13301
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13301-HDFS-7240.000.patch
>
>
> HDFS-13300 decouples DatanodeID from HDSL/Ozone, it's now safe to remove
> {{containerPort}}, {{ratisPort}} and {{ozoneRestPort}} from {{DatanodeID}}
> and {{DatanodeIDProto}}. This jira is to track the removal of Ozone related
> fields from {{DatanodeID}} and {{DatanodeIDProto}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13301) Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto
[
https://issues.apache.org/jira/browse/HDFS-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13301:
---
Status: Patch Available (was: Open)
> Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and
> DatanodeIDProto
> -
>
> Key: HDFS-13301
> URL: https://issues.apache.org/jira/browse/HDFS-13301
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13301-HDFS-7240.000.patch
>
>
> HDFS-13300 decouples DatanodeID from HDSL/Ozone, it's now safe to remove
> {{containerPort}}, {{ratisPort}} and {{ozoneRestPort}} from {{DatanodeID}}
> and {{DatanodeIDProto}}. This jira is to track the removal of Ozone related
> fields from {{DatanodeID}} and {{DatanodeIDProto}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13301) Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto
[
https://issues.apache.org/jira/browse/HDFS-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13301:
---
Attachment: HDFS-13301-HDFS-7240.000.patch
> Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and
> DatanodeIDProto
> -
>
> Key: HDFS-13301
> URL: https://issues.apache.org/jira/browse/HDFS-13301
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13301-HDFS-7240.000.patch
>
>
> HDFS-13300 decouples DatanodeID from HDSL/Ozone, it's now safe to remove
> {{containerPort}}, {{ratisPort}} and {{ozoneRestPort}} from {{DatanodeID}}
> and {{DatanodeIDProto}}. This jira is to track the removal of Ozone related
> fields from {{DatanodeID}} and {{DatanodeIDProto}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422070#comment-16422070 ] Shashikant Banerjee commented on HDFS-12794: Rebased to the latest repo in patch v12. > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12794) Ozone: Parallelize ChunkOutputSream Writes to container
[ https://issues.apache.org/jira/browse/HDFS-12794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-12794: --- Attachment: HDFS-12794-HDFS-7240.012.patch > Ozone: Parallelize ChunkOutputSream Writes to container > --- > > Key: HDFS-12794 > URL: https://issues.apache.org/jira/browse/HDFS-12794 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-12794-HDFS-7240.001.patch, > HDFS-12794-HDFS-7240.002.patch, HDFS-12794-HDFS-7240.003.patch, > HDFS-12794-HDFS-7240.004.patch, HDFS-12794-HDFS-7240.005.patch, > HDFS-12794-HDFS-7240.006.patch, HDFS-12794-HDFS-7240.007.patch, > HDFS-12794-HDFS-7240.008.patch, HDFS-12794-HDFS-7240.009.patch, > HDFS-12794-HDFS-7240.010.patch, HDFS-12794-HDFS-7240.011.patch, > HDFS-12794-HDFS-7240.012.patch > > > The chunkOutPutStream Write are sync in nature .Once one chunk of data gets > written, the next chunk write is blocked until the previous chunk is written > to the container. > The ChunkOutputWrite Stream writes should be made async and Close on the > OutputStream should ensure flushing of all dirty buffers to the container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13325) Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
[
https://issues.apache.org/jira/browse/HDFS-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee reassigned HDFS-13325:
--
Assignee: Shashikant Banerjee
> Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
> ---
>
> Key: HDFS-13325
> URL: https://issues.apache.org/jira/browse/HDFS-13325
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
>
> Based on this comment, we can rename {{ObjectStoreRestPlugin}} to
> {{OzoneDatanodeService}}. So that the plugin name will be consistant with
> {{HdslDatanodeService}}. We can also remove {{DataNodeServicePlugin}} and
> directly use {{ServicePlugin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13325) Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
[
https://issues.apache.org/jira/browse/HDFS-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13325:
---
Attachment: HDFS-13325-HDFS-7240.000.patch
> Ozone: Rename ObjectStoreRestPlugin to OzoneDatanodeService
> ---
>
> Key: HDFS-13325
> URL: https://issues.apache.org/jira/browse/HDFS-13325
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13325-HDFS-7240.000.patch
>
>
> Based on this comment, we can rename {{ObjectStoreRestPlugin}} to
> {{OzoneDatanodeService}}. So that the plugin name will be consistant with
> {{HdslDatanodeService}}. We can also remove {{DataNodeServicePlugin}} and
> directly use {{ServicePlugin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13324) Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails
[ https://issues.apache.org/jira/browse/HDFS-13324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee reassigned HDFS-13324: -- Assignee: Shashikant Banerjee > Ozone: Remove InfoPort and InfoSecurePort from DatanodeDetails > -- > > Key: HDFS-13324 > URL: https://issues.apache.org/jira/browse/HDFS-13324 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nanda kumar >Assignee: Shashikant Banerjee >Priority: Major > Labels: newbie > > We have removed the dependency of DatanodeID in HDSL/Ozone and there is no > need for InfoPort and InfoSecurePort. It is now safe to remove InfoPort and > InfoSecurePort from DatanodeDetails. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13301) Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto
[
https://issues.apache.org/jira/browse/HDFS-13301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee reassigned HDFS-13301:
--
Assignee: Shashikant Banerjee
> Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and
> DatanodeIDProto
> -
>
> Key: HDFS-13301
> URL: https://issues.apache.org/jira/browse/HDFS-13301
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: newbie
>
> HDFS-13300 decouples DatanodeID from HDSL/Ozone, it's now safe to remove
> {{containerPort}}, {{ratisPort}} and {{ozoneRestPort}} from {{DatanodeID}}
> and {{DatanodeIDProto}}. This jira is to track the removal of Ozone related
> fields from {{DatanodeID}} and {{DatanodeIDProto}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423470#comment-16423470 ] Shashikant Banerjee commented on HDFS-13320: Thanks [~anu], for having a look at it. The issue does not seem to be relate with the patch. The hadoop_classpath is not getting properly set. I am not able to run KSM , SCM etc. HW15685:hadoop sbanerjee$ ./hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/bin/oz ksm WARNING: /Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/logs does not exist. Creating. Error: Could not find or load main class org.apache.hadoop.ozone.ksm.KeySpaceManager HW15685:hadoop sbanerjee$ ./hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/bin/oz scm Error: Could not find or load main class org.apache.hadoop.ozone.scm.StorageContainerManager This is related to HDFS-13383. > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch, HDFS-13320-HDFS-7240.003.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13383) Ozone: start-ozone.sh fail to start datanode because of incomplete classpaths
[
https://issues.apache.org/jira/browse/HDFS-13383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423498#comment-16423498
]
Shashikant Banerjee commented on HDFS-13383:
Thanks [~msingh], for working on this. I just applied the patch and ran oz
script to start KSM ,SCM etc... The issue still seems to exist there.
HW15685:hadoop sbanerjee$ ./hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/bin/oz
genesis -h
Error: Could not find or load main class org.apache.hadoop.ozone.genesis.Genesis
HW15685:hadoop sbanerjee$ ./hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/bin/oz scm
WARNING: /Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/logs
does not exist. Creating.
Error: Could not find or load main class
org.apache.hadoop.ozone.scm.StorageContainerManager
> Ozone: start-ozone.sh fail to start datanode because of incomplete classpaths
> -
>
> Key: HDFS-13383
> URL: https://issues.apache.org/jira/browse/HDFS-13383
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13383-HDFS-7240.001.patch
>
>
> start-ozone.sh calls start-dfs.sh to start the NN and DN in a ozone cluster.
> Starting of datanode fails because of incomplete classpaths as datanode is
> unable to load all the plugins.
> Setting the class path to the following values does resolve the issue:
> {code}
> export
> HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/ozone/*:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/hdsl/*:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/ozone/lib/*:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/hdsl/lib/*:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/cblock/*:/opt/hadoop/hadoop-3.2.0-SNAPSHOT/share/hadoop/cblock/lib/*
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13079) Provide a config to start namenode in safemode state upto a certain transaction id
[
https://issues.apache.org/jira/browse/HDFS-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee reassigned HDFS-13079:
--
Assignee: Shashikant Banerjee (was: Mukul Kumar Singh)
> Provide a config to start namenode in safemode state upto a certain
> transaction id
> --
>
> Key: HDFS-13079
> URL: https://issues.apache.org/jira/browse/HDFS-13079
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Mukul Kumar Singh
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13079.001.patch
>
>
> In some cases it necessary to rollback the Namenode back to a certain
> transaction id. This is especially needed when the user issues a {{rm -Rf
> -skipTrash}} by mistake.
> Rolling back to a transaction id helps in taking a peek at the filesystem at
> a particular instant. This jira proposes to provide a configuration variable
> using which the namenode can be started upto a certain transaction id. The
> filesystem will be in a readonly safemode which cannot be overridden
> manually. It will only be overridden by removing the config value from the
> config file. Please also note that this will not cause any changes in the
> filesystem state, the filesystem will be in safemode state and no changes to
> the filesystem state will be allowed.
> Please note that in case a checkpoint has already happened and the requested
> transaction id has been subsumed in an FSImage, then the namenode will be
> started with the next nearest transaction id. Further FSImage files and edits
> will be ignored.
> If the checkpoint hasn't happen then the namenode will be started with the
> exact transaction id.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[
https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13368:
---
Attachment: HDFS-13368-HDFS-7240.001.patch
> Ozone:TestEndPoint tests are failing consistently
> -
>
> Key: HDFS-13368
> URL: https://issues.apache.org/jira/browse/HDFS-13368
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13368-HDFS-7240.000.patch,
> HDFS-13368-HDFS-7240.001.patch
>
>
> With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file
> made required fiields. These parameters are not set in TestEndPoint which
> lead these to fail consistently.
> TestEndPoint#testRegisterToInvalidEndpoint
> {code:java}
> com.google.protobuf.UninitializedMessageException: Message missing required
> fields: ipAddress, hostName
> at
> com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
> at
> org.apache.hadoop.hdsl.protocol.proto.HdslProtos$DatanodeDetailsProto$Builder.build(HdslProtos.java:1756)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.registerTaskHelper(TestEndPoint.java:236)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.testRegisterToInvalidEndpoint(TestEndPoint.java:257)
> {code}
> TestEndPoint#testHeartbeatTaskToInvalidNode
> {code:java}
> 2018-03-29 18:14:54,140 WARN impl.RaftServerProxy: FAILED new RaftServerProxy
> attempt #1/5: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer, sleep 500ms and then retry.
> java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer
> at
> org.apache.ratis.server.storage.RaftLogWorker.(RaftLogWorker.java:104)
> at
> org.apache.ratis.server.storage.SegmentedRaftLog.(SegmentedRaftLog.java:113)
> at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:151)
> at org.apache.ratis.server.impl.ServerState.(ServerState.java:101){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[
https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13368:
---
Description:
With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file
made required fiields. These parameters are not set in TestEndPoint which lead
these to fail consistently.
TestEndPoint#testRegisterToInvalidEndpoint
{code:java}
com.google.protobuf.UninitializedMessageException: Message missing required
fields: ipAddress, hostName
at
com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
at
org.apache.hadoop.hdsl.protocol.proto.HdslProtos$DatanodeDetailsProto$Builder.build(HdslProtos.java:1756)
at
org.apache.hadoop.ozone.container.common.TestEndPoint.registerTaskHelper(TestEndPoint.java:236)
at
org.apache.hadoop.ozone.container.common.TestEndPoint.testRegisterToInvalidEndpoint(TestEndPoint.java:257)
{code}
TestEndPoint#testHeartbeatTaskToInvalidNode
{code:java}
2018-03-29 18:14:54,140 WARN impl.RaftServerProxy: FAILED new RaftServerProxy
attempt #1/5: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
com/codahale/metrics/Timer, sleep 500ms and then retry.
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
com/codahale/metrics/Timer
at org.apache.ratis.server.storage.RaftLogWorker.(RaftLogWorker.java:104)
at
org.apache.ratis.server.storage.SegmentedRaftLog.(SegmentedRaftLog.java:113)
at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:151)
at org.apache.ratis.server.impl.ServerState.(ServerState.java:101){code}
was:With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto
file made required fiields. These parameters are not set in TestEndPoint which
lead these to fail consistently.
> Ozone:TestEndPoint tests are failing consistently
> -
>
> Key: HDFS-13368
> URL: https://issues.apache.org/jira/browse/HDFS-13368
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13368-HDFS-7240.000.patch
>
>
> With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file
> made required fiields. These parameters are not set in TestEndPoint which
> lead these to fail consistently.
> TestEndPoint#testRegisterToInvalidEndpoint
> {code:java}
> com.google.protobuf.UninitializedMessageException: Message missing required
> fields: ipAddress, hostName
> at
> com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
> at
> org.apache.hadoop.hdsl.protocol.proto.HdslProtos$DatanodeDetailsProto$Builder.build(HdslProtos.java:1756)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.registerTaskHelper(TestEndPoint.java:236)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.testRegisterToInvalidEndpoint(TestEndPoint.java:257)
> {code}
> TestEndPoint#testHeartbeatTaskToInvalidNode
> {code:java}
> 2018-03-29 18:14:54,140 WARN impl.RaftServerProxy: FAILED new RaftServerProxy
> attempt #1/5: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer, sleep 500ms and then retry.
> java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer
> at
> org.apache.ratis.server.storage.RaftLogWorker.(RaftLogWorker.java:104)
> at
> org.apache.ratis.server.storage.SegmentedRaftLog.(SegmentedRaftLog.java:113)
> at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:151)
> at org.apache.ratis.server.impl.ServerState.(ServerState.java:101){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13368) Ozone:TestEndPoint tests are failing consistently
[
https://issues.apache.org/jira/browse/HDFS-13368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418937#comment-16418937
]
Shashikant Banerjee commented on HDFS-13368:
Patch v1 adds the dependency of metrics-core lib in scm server pom which is
required for test runs.
> Ozone:TestEndPoint tests are failing consistently
> -
>
> Key: HDFS-13368
> URL: https://issues.apache.org/jira/browse/HDFS-13368
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: HDFS-7240
>
> Attachments: HDFS-13368-HDFS-7240.000.patch,
> HDFS-13368-HDFS-7240.001.patch
>
>
> With HDFS-13300, the hostName and IpAdress in the DatanodeDetails.proto file
> made required fiields. These parameters are not set in TestEndPoint which
> lead these to fail consistently.
> TestEndPoint#testRegisterToInvalidEndpoint
> {code:java}
> com.google.protobuf.UninitializedMessageException: Message missing required
> fields: ipAddress, hostName
> at
> com.google.protobuf.AbstractMessage$Builder.newUninitializedMessageException(AbstractMessage.java:770)
> at
> org.apache.hadoop.hdsl.protocol.proto.HdslProtos$DatanodeDetailsProto$Builder.build(HdslProtos.java:1756)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.registerTaskHelper(TestEndPoint.java:236)
> at
> org.apache.hadoop.ozone.container.common.TestEndPoint.testRegisterToInvalidEndpoint(TestEndPoint.java:257)
> {code}
> TestEndPoint#testHeartbeatTaskToInvalidNode
> {code:java}
> 2018-03-29 18:14:54,140 WARN impl.RaftServerProxy: FAILED new RaftServerProxy
> attempt #1/5: java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer, sleep 500ms and then retry.
> java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
> com/codahale/metrics/Timer
> at
> org.apache.ratis.server.storage.RaftLogWorker.(RaftLogWorker.java:104)
> at
> org.apache.ratis.server.storage.SegmentedRaftLog.(SegmentedRaftLog.java:113)
> at org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:151)
> at org.apache.ratis.server.impl.ServerState.(ServerState.java:101){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13348) Ozone: Update IP and hostname in Datanode from SCM's response to the register call
[
https://issues.apache.org/jira/browse/HDFS-13348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shashikant Banerjee updated HDFS-13348:
---
Status: Patch Available (was: Open)
> Ozone: Update IP and hostname in Datanode from SCM's response to the register
> call
> --
>
> Key: HDFS-13348
> URL: https://issues.apache.org/jira/browse/HDFS-13348
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Nanda kumar
>Assignee: Shashikant Banerjee
>Priority: Major
> Attachments: HDFS-13348-HDFS-7240.000.patch
>
>
> Whenever a Datanode registers with SCM, the SCM resolves the IP address and
> hostname of the Datanode form the RPC call. This IP address and hostname
> should be sent back to Datanode in the response to register call and the
> Datanode has to update the values from the response to its
> {{DatanodeDetails}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411364#comment-16411364 ] Shashikant Banerjee edited comment on HDFS-13342 at 3/29/18 3:45 PM: - Patch v1 rename oz script to ozone and the fixes the classnames for running freon and getOzoneConf in the script . The patch also removes the references of hdfs from SCM and KSM help message.This patch is dependent on HDFS-13320. was (Author: shashikant): Patch v1 rename oz script to ozone and the fixes the classnames for running freon and getOzoneConf in the script . The patch also removes the references of hdfs from SCM and KSM help message. > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13342: --- Description: The Ozone (oz script) has wrong classnames for freon etc, As a result of which freon cannot be started from command line. This Jira proposes to fix all these. The oz script will be renamed to Ozone as well. (was: The Ozone (oz script) has wrong classnames for freon etc, As a result of which freon cannot be started from command line. This Jira proposes to fix all these. The oz script will be renamed to Ozone as well. This patch is dependent on HDFS-13320.) > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407713#comment-16407713 ] Shashikant Banerjee commented on HDFS-13320: Patch v1 implements benchmarking for Rocks DB, Container State Maps and MetadatStore Reads/Writes. > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
Shashikant Banerjee created HDFS-13320: -- Summary: Ozone:Support for MicrobenchMarking Tool Key: HDFS-13320 URL: https://issues.apache.org/jira/browse/HDFS-13320 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee This Jira proposes to add a micro benchmarking tool called Genesis which executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407713#comment-16407713 ] Shashikant Banerjee edited comment on HDFS-13320 at 3/21/18 10:13 AM: -- Patch v1 implements benchmarking for Rocks DB, Container State Maps and MetadataStore Reads/Writes. was (Author: shashikant): Patch v1 implements benchmarking for Rocks DB, Container State Maps and MetadatStore Reads/Writes. > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Attachment: HDFS-13320-HDFS-7240.001.patch > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Status: Patch Available (was: Open) > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Attachment: (was: HDFS-13320-HDFS-7240.002.patch) > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Attachment: HDFS-13320-HDFS-7240.002.patch > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13320: --- Attachment: HDFS-13320-HDFS-7240.002.patch > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13320) Ozone:Support for MicrobenchMarking Tool
[ https://issues.apache.org/jira/browse/HDFS-13320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1648#comment-1648 ] Shashikant Banerjee commented on HDFS-13320: Thanks [~anu], for the review comments. The org.openjdk.jmh:jmh-generator-annprocess was missing in the classpath while executing the genesis tool and hence the error. Addressed this in the v2 patch. W15685:hadoop sbanerjee$ ./hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/bin/oz genesis -h # JMH version: 1.19 # VM version: JDK 1.8.0_144, VM 25.144-b01 # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/bin/java # VM options: -Dproc_genesis -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm= -Djava.security.krb5.kdc= -Djava.security.krb5.conf= -Dyarn.log.dir=/Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/logs -Dyarn.log.file=hadoop.log -Dyarn.home.dir=/Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT -Dyarn.root.logger=INFO,console -Dhadoop.log.dir=/Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/Users/sbanerjee/hadoop/hadoop-dist/target/hadoop-3.2.0-SNAPSHOT -Dhadoop.id.str=sbanerjee -Dhadoop.root.logger=INFO,console -Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender # Warmup: 5 iterations, 1 s each # Measurement: 20 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.apache.hadoop.ozone.genesis.BenchMarkContainerStateMap.createContainerBenchMark # Run progress: 0.00% complete, ETA 00:06:15 # Fork: 1 of 1 # Warmup Iteration 1: 10821.298 ops/s # Warmup Iteration 2: 12306.852 ops/s # Warmup Iteration 3: 11333.630 ops/s > Ozone:Support for MicrobenchMarking Tool > > > Key: HDFS-13320 > URL: https://issues.apache.org/jira/browse/HDFS-13320 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13320-HDFS-7240.001.patch, > HDFS-13320-HDFS-7240.002.patch > > > This Jira proposes to add a micro benchmarking tool called Genesis which > executes a set of HDSL/Ozone benchmarks. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411364#comment-16411364 ] Shashikant Banerjee commented on HDFS-13342: Patch v1 rename oz script to ozone and the fixes the classnames for running freon and getOzoneConf in the script . The patch also removes the references of hdfs from SCM and KSM help message. > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. This patch is > dependent on HDFS-13320. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13342) Ozone: Fix the class names in Ozone Script
Shashikant Banerjee created HDFS-13342: -- Summary: Ozone: Fix the class names in Ozone Script Key: HDFS-13342 URL: https://issues.apache.org/jira/browse/HDFS-13342 Project: Hadoop HDFS Issue Type: Bug Components: HDFS-7240 Affects Versions: HDFS-7240 Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee The Ozone (oz script) has wrong classnames for freon etc, As a result of which freon cannot be started from command line. This Jira proposes to fix all these. The oz script will be renamed to Ozone as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13342: --- Description: The Ozone (oz script) has wrong classnames for freon etc, As a result of which freon cannot be started from command line. This Jira proposes to fix all these. The oz script will be renamed to Ozone as well. This patch is dependent on HDFS-13320. (was: The Ozone (oz script) has wrong classnames for freon etc, As a result of which freon cannot be started from command line. This Jira proposes to fix all these. The oz script will be renamed to Ozone as well.) > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. This patch is > dependent on HDFS-13320. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13342) Ozone: Fix the class names in Ozone Script
[ https://issues.apache.org/jira/browse/HDFS-13342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13342: --- Attachment: HDFS-13342-HDFS-7240.001.patch > Ozone: Fix the class names in Ozone Script > -- > > Key: HDFS-13342 > URL: https://issues.apache.org/jira/browse/HDFS-13342 > Project: Hadoop HDFS > Issue Type: Bug > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13342-HDFS-7240.001.patch > > > The Ozone (oz script) has wrong classnames for freon etc, As a result of > which freon cannot be started from command line. This Jira proposes to fix > all these. The oz script will be renamed to Ozone as well. This patch is > dependent on HDFS-13320. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13223) Reduce DiffListBySkipList memory usage
[ https://issues.apache.org/jira/browse/HDFS-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13223: --- Attachment: HDFS-13223.001.patch > Reduce DiffListBySkipList memory usage > -- > > Key: HDFS-13223 > URL: https://issues.apache.org/jira/browse/HDFS-13223 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Tsz Wo Nicholas Sze >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13223.001.patch > > > There are several ways to reduce memory footprint of DiffListBySkipList. > - Move maxSkipLevels and skipInterval to DirectoryDiffListFactory. > - Use an array for skipDiffList instead of List. > - Do not store the level 0 element in skipDiffList. > - Do not create new ChildrenDiff for the same value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13223) Reduce DiffListBySkipList memory usage
[ https://issues.apache.org/jira/browse/HDFS-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13223: --- Status: Patch Available (was: Open) > Reduce DiffListBySkipList memory usage > -- > > Key: HDFS-13223 > URL: https://issues.apache.org/jira/browse/HDFS-13223 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Tsz Wo Nicholas Sze >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13223.001.patch > > > There are several ways to reduce memory footprint of DiffListBySkipList. > - Move maxSkipLevels and skipInterval to DirectoryDiffListFactory. > - Use an array for skipDiffList instead of List. > - Do not store the level 0 element in skipDiffList. > - Do not create new ChildrenDiff for the same value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13223) Reduce DiffListBySkipList memory usage
[ https://issues.apache.org/jira/browse/HDFS-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385745#comment-16385745 ] Shashikant Banerjee commented on HDFS-13223: Thanks [~szetszwo], for filing this Jira and your suggestions for improving the memory footprint of DiffListBySkipList. Patch v1 addresses the ideas suggested. Please have a look. > Reduce DiffListBySkipList memory usage > -- > > Key: HDFS-13223 > URL: https://issues.apache.org/jira/browse/HDFS-13223 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Tsz Wo Nicholas Sze >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13223.001.patch > > > There are several ways to reduce memory footprint of DiffListBySkipList. > - Move maxSkipLevels and skipInterval to DirectoryDiffListFactory. > - Use an array for skipDiffList instead of List. > - Do not store the level 0 element in skipDiffList. > - Do not create new ChildrenDiff for the same value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13223) Reduce DiffListBySkipList memory usage
[ https://issues.apache.org/jira/browse/HDFS-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13223: --- Attachment: HDFS-13223.002.patch > Reduce DiffListBySkipList memory usage > -- > > Key: HDFS-13223 > URL: https://issues.apache.org/jira/browse/HDFS-13223 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Tsz Wo Nicholas Sze >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13223.001.patch, HDFS-13223.002.patch > > > There are several ways to reduce memory footprint of DiffListBySkipList. > - Move maxSkipLevels and skipInterval to DirectoryDiffListFactory. > - Use an array for skipDiffList instead of List. > - Do not store the level 0 element in skipDiffList. > - Do not create new ChildrenDiff for the same value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13223) Reduce DiffListBySkipList memory usage
[ https://issues.apache.org/jira/browse/HDFS-13223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385998#comment-16385998 ] Shashikant Banerjee commented on HDFS-13223: patch v2 fixes the checkstyle issues. > Reduce DiffListBySkipList memory usage > -- > > Key: HDFS-13223 > URL: https://issues.apache.org/jira/browse/HDFS-13223 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Tsz Wo Nicholas Sze >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13223.001.patch, HDFS-13223.002.patch > > > There are several ways to reduce memory footprint of DiffListBySkipList. > - Move maxSkipLevels and skipInterval to DirectoryDiffListFactory. > - Use an array for skipDiffList instead of List. > - Do not store the level 0 element in skipDiffList. > - Do not create new ChildrenDiff for the same value. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13227) Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList
[ https://issues.apache.org/jira/browse/HDFS-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385992#comment-16385992 ] Shashikant Banerjee commented on HDFS-13227: Patch v1 introduces an method called DirectoryWithSnapshotFeature#DirectoryDiffList#getDiffListBetweenSnapshots to generate minimal list of diffs needed to combine the overall cumulative diff between two snapshots. DirectoryWithSnapshotFeature#getChildrenList and DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots have been modified accordingly. > Add a method to calculate cumulative diff over multiple snapshots in > DirectoryDiffList > --- > > Key: HDFS-13227 > URL: https://issues.apache.org/jira/browse/HDFS-13227 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13227.001.patch > > > This Jira proposes to add an API in DirectoryWithSnapshotFeature#f > DirectoryDiffList which will return minimal list of diffs needed to combine > to get the cumulative diff between two given snapshots. The same method will > be made use while constructing the childrenList for a directory. > DirectoryWithSnapshotFeature#getChildrenList and > DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the > same method to get the cumulative diff. When snapshotSkipList, with minimal > set of diffs to combine in order to get the cumulative diff, the overall > computation will be faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13227) Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList
[ https://issues.apache.org/jira/browse/HDFS-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13227: --- Description: This Jira proposes to add an API in DirectoryWithSnapshotFeature#f DirectoryDiffList which will return minimal list of diffs needed to combine to get the cumulative diff between two given snapshots. The same method will be made use while constructing the childrenList for a directory. DirectoryWithSnapshotFeature#getChildrenList and DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the same method to get the cumulative diff. When snapshotSkipList, with minimal set of diffs to combine in order to get the cumulative diff, the overall computation will be faster. (was: This Jira proposes to add an API in DirectoryWithSnapshotFeature#f DirectoryDiffList which will return minimal list of diffs needed to combine to get the cumulative diff between two given snapshots. The same method will be made use while constructing the childrenList for a directory. DirectoryWithSnapshotFeature#getChildrenList and DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the same method to get the cumulative diff. When snapshotSkipList, with minimal set of diffs to combine in order to get the cumulative diff, the overall computation will be faster.) > Add a method to calculate cumulative diff over multiple snapshots in > DirectoryDiffList > --- > > Key: HDFS-13227 > URL: https://issues.apache.org/jira/browse/HDFS-13227 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > > This Jira proposes to add an API in DirectoryWithSnapshotFeature#f > DirectoryDiffList which will return minimal list of diffs needed to combine > to get the cumulative diff between two given snapshots. The same method will > be made use while constructing the childrenList for a directory. > DirectoryWithSnapshotFeature#getChildrenList and > DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the > same method to get the cumulative diff. When snapshotSkipList, with minimal > set of diffs to combine in order to get the cumulative diff, the overall > computation will be faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13227) Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList
Shashikant Banerjee created HDFS-13227: -- Summary: Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList Key: HDFS-13227 URL: https://issues.apache.org/jira/browse/HDFS-13227 Project: Hadoop HDFS Issue Type: Improvement Components: snapshots Reporter: Shashikant Banerjee Assignee: Shashikant Banerjee This Jira proposes to add an API in DirectoryWithSnapshotFeature#f DirectoryDiffList which will return minimal list of diffs needed to combine to get the cumulative diff between two given snapshots. The same method will be made use while constructing the childrenList for a directory. DirectoryWithSnapshotFeature#getChildrenList and DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the same method to get the cumulative diff. When snapshotSkipList, with minimal set of diffs to combine in order to get the cumulative diff, the overall computation will be faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13227) Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList
[ https://issues.apache.org/jira/browse/HDFS-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13227: --- Attachment: HDFS-13227.001.patch > Add a method to calculate cumulative diff over multiple snapshots in > DirectoryDiffList > --- > > Key: HDFS-13227 > URL: https://issues.apache.org/jira/browse/HDFS-13227 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13227.001.patch > > > This Jira proposes to add an API in DirectoryWithSnapshotFeature#f > DirectoryDiffList which will return minimal list of diffs needed to combine > to get the cumulative diff between two given snapshots. The same method will > be made use while constructing the childrenList for a directory. > DirectoryWithSnapshotFeature#getChildrenList and > DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the > same method to get the cumulative diff. When snapshotSkipList, with minimal > set of diffs to combine in order to get the cumulative diff, the overall > computation will be faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13227) Add a method to calculate cumulative diff over multiple snapshots in DirectoryDiffList
[ https://issues.apache.org/jira/browse/HDFS-13227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-13227: --- Status: Patch Available (was: Open) > Add a method to calculate cumulative diff over multiple snapshots in > DirectoryDiffList > --- > > Key: HDFS-13227 > URL: https://issues.apache.org/jira/browse/HDFS-13227 > Project: Hadoop HDFS > Issue Type: Improvement > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDFS-13227.001.patch > > > This Jira proposes to add an API in DirectoryWithSnapshotFeature#f > DirectoryDiffList which will return minimal list of diffs needed to combine > to get the cumulative diff between two given snapshots. The same method will > be made use while constructing the childrenList for a directory. > DirectoryWithSnapshotFeature#getChildrenList and > DirectoryWithSnapshotFeature#computeDiffBetweenSnapshots will make use of the > same method to get the cumulative diff. When snapshotSkipList, with minimal > set of diffs to combine in order to get the cumulative diff, the overall > computation will be faster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
