[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade
[
https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528984#comment-16528984
]
Yuanbo Liu commented on HDFS-10664:
---
[~djp] I find it's a bit complicated to reproduce it in unit test because
LayoutVersion is not an option in Hadoop. I remembered that I tested it from
2.6.2 to 2.7.2. My unit test case doesn't work because the layout version is
always the latest one. I will write another test case to cover it. Thanks again
for your review.
> layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION
> file after cluster upgrade
> ---
>
> Key: HDFS-10664
> URL: https://issues.apache.org/jira/browse/HDFS-10664
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, hdfs
>Affects Versions: 2.7.1
>Reporter: Amit Anand
>Assignee: Yuanbo Liu
>Priority: Major
> Attachments: HDFS-10664.001.patch
>
>
> After a cluster is upgraded I see a mismatch in {{layoutVersion}} between NN
> VERSION file and JN VERSION file.
> Here is what I see:
> Before cluster upgrade:
> ==
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-60
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> After cluster upgrade:
> =
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-63
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> Since {{Namenode}} is what creates {{Journalnode}} {{VERSION}} file during
> {{initializeSharedEdits}}, it should also update the file with correct
> information after the cluster is upgraded and {{hdfs dfsadmin
> -finalizeUpgrade}} has been executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade
[
https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528949#comment-16528949
]
Yuanbo Liu commented on HDFS-10664:
---
[~djp] Thanks for your comments, I will test it again and if this is not an
issue, I will close it.
> layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION
> file after cluster upgrade
> ---
>
> Key: HDFS-10664
> URL: https://issues.apache.org/jira/browse/HDFS-10664
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, hdfs
>Affects Versions: 2.7.1
>Reporter: Amit Anand
>Assignee: Yuanbo Liu
>Priority: Major
> Attachments: HDFS-10664.001.patch
>
>
> After a cluster is upgraded I see a mismatch in {{layoutVersion}} between NN
> VERSION file and JN VERSION file.
> Here is what I see:
> Before cluster upgrade:
> ==
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-60
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> After cluster upgrade:
> =
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-63
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> Since {{Namenode}} is what creates {{Journalnode}} {{VERSION}} file during
> {{initializeSharedEdits}}, it should also update the file with correct
> information after the cluster is upgraded and {{hdfs dfsadmin
> -finalizeUpgrade}} has been executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade
[
https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanbo Liu updated HDFS-10664:
--
Comment: was deleted
(was: [~djp] Thanks for your comments, will rebase my patch shortly)
> layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION
> file after cluster upgrade
> ---
>
> Key: HDFS-10664
> URL: https://issues.apache.org/jira/browse/HDFS-10664
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, hdfs
>Affects Versions: 2.7.1
>Reporter: Amit Anand
>Assignee: Yuanbo Liu
>Priority: Major
> Attachments: HDFS-10664.001.patch
>
>
> After a cluster is upgraded I see a mismatch in {{layoutVersion}} between NN
> VERSION file and JN VERSION file.
> Here is what I see:
> Before cluster upgrade:
> ==
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-60
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> After cluster upgrade:
> =
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-63
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> Since {{Namenode}} is what creates {{Journalnode}} {{VERSION}} file during
> {{initializeSharedEdits}}, it should also update the file with correct
> information after the cluster is upgraded and {{hdfs dfsadmin
> -finalizeUpgrade}} has been executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10664) layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION file after cluster upgrade
[
https://issues.apache.org/jira/browse/HDFS-10664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528948#comment-16528948
]
Yuanbo Liu commented on HDFS-10664:
---
[~djp] Thanks for your comments, will rebase my patch shortly
> layoutVersion mismatch between Namenode VERSION file and Journalnode VERSION
> file after cluster upgrade
> ---
>
> Key: HDFS-10664
> URL: https://issues.apache.org/jira/browse/HDFS-10664
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ha, hdfs
>Affects Versions: 2.7.1
>Reporter: Amit Anand
>Assignee: Yuanbo Liu
>Priority: Major
> Attachments: HDFS-10664.001.patch
>
>
> After a cluster is upgraded I see a mismatch in {{layoutVersion}} between NN
> VERSION file and JN VERSION file.
> Here is what I see:
> Before cluster upgrade:
> ==
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-60
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> After cluster upgrade:
> =
> {code}
> ## Version file from NN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=NAME_NODE
> blockpoolID=BP-786201894-10.0.100.11-1466026941507
> layoutVersion=-63
> {code}
> {code}
> ## Version file from JN current directory
> namespaceID=109645726
> clusterID=CID-edcb62c5-bc1f-49f5-addb-37827340b5de
> cTime=0
> storageType=JOURNAL_NODE
> layoutVersion=-60
> {code}
> Since {{Namenode}} is what creates {{Journalnode}} {{VERSION}} file during
> {{initializeSharedEdits}}, it should also update the file with correct
> information after the cluster is upgraded and {{hdfs dfsadmin
> -finalizeUpgrade}} has been executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-187) Command status publisher for datanode
[
https://issues.apache.org/jira/browse/HDDS-187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528902#comment-16528902
]
genericqa commented on HDDS-187:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 19s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
16s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
35s{color} | {color:red} hadoop-hdds in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 0m 35s{color} |
{color:red} hadoop-hdds in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 35s{color}
| {color:red} hadoop-hdds in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
19s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
18s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
17s{color} | {color:red} hadoop-hdds_server-scm generated 3 new + 0 unchanged -
0 fixed = 3 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
7s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 21s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 18s{color}
| {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m
[jira] [Commented] (HDDS-187) Command status publisher for datanode
[ https://issues.apache.org/jira/browse/HDDS-187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528885#comment-16528885 ] Ajay Kumar commented on HDDS-187: - patch v3 updates commandHandlers to change status of commands in finally block. > Command status publisher for datanode > - > > Key: HDDS-187 > URL: https://issues.apache.org/jira/browse/HDDS-187 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.2.1 >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-187.00.patch, HDDS-187.01.patch, HDDS-187.02.patch, > HDDS-187.03.patch > > > Currently SCM sends set of commands for DataNode. DataNode executes them via > CommandHandler. This jira intends to create a Command status publisher which > will return status of these commands back to the SCM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-187) Command status publisher for datanode
[ https://issues.apache.org/jira/browse/HDDS-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDDS-187: Attachment: HDDS-187.03.patch > Command status publisher for datanode > - > > Key: HDDS-187 > URL: https://issues.apache.org/jira/browse/HDDS-187 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.2.1 >Reporter: Ajay Kumar >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-187.00.patch, HDDS-187.01.patch, HDDS-187.02.patch, > HDDS-187.03.patch > > > Currently SCM sends set of commands for DataNode. DataNode executes them via > CommandHandler. This jira intends to create a Command status publisher which > will return status of these commands back to the SCM. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-199) Implement ReplicationManager to replicate ClosedContainers
[
https://issues.apache.org/jira/browse/HDDS-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528863#comment-16528863
]
genericqa commented on HDDS-199:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
39s{color} | {color:red} hadoop-hdds in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 40s{color}
| {color:red} hadoop-hdds in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
18s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 28s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} server-scm in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
3s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
30s{color} | {color:green} framework in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} container-service in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 20s{color}
| {color:red} server-scm in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
26s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd |
| JIRA Issue | HDDS-199 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12929826/HDDS-199.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux 73d68c85d654 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018
[jira] [Commented] (HDDS-199) Implement ReplicationManager to replicate ClosedContainers
[ https://issues.apache.org/jira/browse/HDDS-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528831#comment-16528831 ] Elek, Marton commented on HDDS-199: --- First version is uploaded. Notable changes: * Existing datanodes added to the ContainerPlacementPolicy * Default Polling frequency is modified in LeaseManager (when activeLeases empty, it starts to sleep until the and of the word) * PriorityQueue is modified to use BlockingQueue and using blocking call (take) > Implement ReplicationManager to replicate ClosedContainers > -- > > Key: HDDS-199 > URL: https://issues.apache.org/jira/browse/HDDS-199 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-199.001.patch > > > HDDS/Ozone supports Open and Closed containers. In case of specific > conditions (container is full, node is failed) the container will be closed > and will be replicated in a different way. The replication of Open containers > are handled with Ratis and PipelineManger. > The ReplicationManager should handle the replication of the ClosedContainers. > The replication information will be sent as an event > (UnderReplicated/OverReplicated). > The Replication manager will collect all of the events in a priority queue > (to replicate first the containers where more replica is missing) calculate > the destination datanode (first with a very simple algorithm, later with > calculating scatter-width) and send the Copy/Delete container to the datanode > (CommandQueue). > A CopyCommandWatcher/DeleteCommandWatcher are also included to retry the > copy/delete in case of failure. This is an in-memory structure (based on > HDDS-195) which can requeue the underreplicated/overreplicated events to the > prioirity queue unless the confirmation of the copy/delete command is arrived. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-199) Implement ReplicationManager to replicate ClosedContainers
[ https://issues.apache.org/jira/browse/HDDS-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDDS-199: -- Status: Patch Available (was: Open) > Implement ReplicationManager to replicate ClosedContainers > -- > > Key: HDDS-199 > URL: https://issues.apache.org/jira/browse/HDDS-199 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-199.001.patch > > > HDDS/Ozone supports Open and Closed containers. In case of specific > conditions (container is full, node is failed) the container will be closed > and will be replicated in a different way. The replication of Open containers > are handled with Ratis and PipelineManger. > The ReplicationManager should handle the replication of the ClosedContainers. > The replication information will be sent as an event > (UnderReplicated/OverReplicated). > The Replication manager will collect all of the events in a priority queue > (to replicate first the containers where more replica is missing) calculate > the destination datanode (first with a very simple algorithm, later with > calculating scatter-width) and send the Copy/Delete container to the datanode > (CommandQueue). > A CopyCommandWatcher/DeleteCommandWatcher are also included to retry the > copy/delete in case of failure. This is an in-memory structure (based on > HDDS-195) which can requeue the underreplicated/overreplicated events to the > prioirity queue unless the confirmation of the copy/delete command is arrived. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-199) Implement ReplicationManager to replicate ClosedContainers
[ https://issues.apache.org/jira/browse/HDDS-199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDDS-199: -- Attachment: HDDS-199.001.patch > Implement ReplicationManager to replicate ClosedContainers > -- > > Key: HDDS-199 > URL: https://issues.apache.org/jira/browse/HDDS-199 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-199.001.patch > > > HDDS/Ozone supports Open and Closed containers. In case of specific > conditions (container is full, node is failed) the container will be closed > and will be replicated in a different way. The replication of Open containers > are handled with Ratis and PipelineManger. > The ReplicationManager should handle the replication of the ClosedContainers. > The replication information will be sent as an event > (UnderReplicated/OverReplicated). > The Replication manager will collect all of the events in a priority queue > (to replicate first the containers where more replica is missing) calculate > the destination datanode (first with a very simple algorithm, later with > calculating scatter-width) and send the Copy/Delete container to the datanode > (CommandQueue). > A CopyCommandWatcher/DeleteCommandWatcher are also included to retry the > copy/delete in case of failure. This is an in-memory structure (based on > HDDS-195) which can requeue the underreplicated/overreplicated events to the > prioirity queue unless the confirmation of the copy/delete command is arrived. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-167) Rename KeySpaceManager to OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528591#comment-16528591 ] Nanda kumar commented on HDDS-167: -- Thanks [~arpitagarwal] for fix OzoneManager UI, it is working now. +1, pending jenkins. > Rename KeySpaceManager to OzoneManager > -- > > Key: HDDS-167 > URL: https://issues.apache.org/jira/browse/HDDS-167 > Project: Hadoop Distributed Data Store > Issue Type: Task > Components: Ozone Manager >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-167.01.patch, HDDS-167.02.patch, HDDS-167.03.patch, > HDDS-167.04.patch, HDDS-167.05.patch, HDDS-167.06.patch, HDDS-167.07.patch, > HDDS-167.08.patch > > > The Ozone KeySpaceManager daemon was renamed to OzoneManager. There's some > more changes needed to complete the rename everywhere e.g. > - command-line > - documentation > - unit tests > - Acceptance tests -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-167) Rename KeySpaceManager to OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-167: - Attachment: (was: HDDS-167.08.patch) > Rename KeySpaceManager to OzoneManager > -- > > Key: HDDS-167 > URL: https://issues.apache.org/jira/browse/HDDS-167 > Project: Hadoop Distributed Data Store > Issue Type: Task > Components: Ozone Manager >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-167.01.patch, HDDS-167.02.patch, HDDS-167.03.patch, > HDDS-167.04.patch, HDDS-167.05.patch, HDDS-167.06.patch, HDDS-167.07.patch, > HDDS-167.08.patch > > > The Ozone KeySpaceManager daemon was renamed to OzoneManager. There's some > more changes needed to complete the rename everywhere e.g. > - command-line > - documentation > - unit tests > - Acceptance tests -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-167) Rename KeySpaceManager to OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-167: - Attachment: HDDS-167.08.patch > Rename KeySpaceManager to OzoneManager > -- > > Key: HDDS-167 > URL: https://issues.apache.org/jira/browse/HDDS-167 > Project: Hadoop Distributed Data Store > Issue Type: Task > Components: Ozone Manager >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-167.01.patch, HDDS-167.02.patch, HDDS-167.03.patch, > HDDS-167.04.patch, HDDS-167.05.patch, HDDS-167.06.patch, HDDS-167.07.patch, > HDDS-167.08.patch > > > The Ozone KeySpaceManager daemon was renamed to OzoneManager. There's some > more changes needed to complete the rename everywhere e.g. > - command-line > - documentation > - unit tests > - Acceptance tests -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-167) Rename KeySpaceManager to OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-167: - Attachment: HDDS-167.08.patch > Rename KeySpaceManager to OzoneManager > -- > > Key: HDDS-167 > URL: https://issues.apache.org/jira/browse/HDDS-167 > Project: Hadoop Distributed Data Store > Issue Type: Task > Components: Ozone Manager >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-167.01.patch, HDDS-167.02.patch, HDDS-167.03.patch, > HDDS-167.04.patch, HDDS-167.05.patch, HDDS-167.06.patch, HDDS-167.07.patch, > HDDS-167.08.patch > > > The Ozone KeySpaceManager daemon was renamed to OzoneManager. There's some > more changes needed to complete the rename everywhere e.g. > - command-line > - documentation > - unit tests > - Acceptance tests -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-167) Rename KeySpaceManager to OzoneManager
[ https://issues.apache.org/jira/browse/HDDS-167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528569#comment-16528569 ] Nanda kumar commented on HDDS-167: -- Rebased patch on top of latest changes. > Rename KeySpaceManager to OzoneManager > -- > > Key: HDDS-167 > URL: https://issues.apache.org/jira/browse/HDDS-167 > Project: Hadoop Distributed Data Store > Issue Type: Task > Components: Ozone Manager >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-167.01.patch, HDDS-167.02.patch, HDDS-167.03.patch, > HDDS-167.04.patch, HDDS-167.05.patch, HDDS-167.06.patch, HDDS-167.07.patch, > HDDS-167.08.patch > > > The Ozone KeySpaceManager daemon was renamed to OzoneManager. There's some > more changes needed to complete the rename everywhere e.g. > - command-line > - documentation > - unit tests > - Acceptance tests -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
