It means that may be some replicas will be stay in under replica state?
2013/4/3 Azuryy Yu <[email protected]> > bq. then namenode start to copy block replicates on DN-2 to another DN, > supposed DN-2. > > sorry for typo. > > Correct for it: > then namenode start to copy block replicates on DN-1 to another DN, > supposed DN-2. > > > On Wed, Apr 3, 2013 at 9:51 AM, Azuryy Yu <[email protected]> wrote: > >> It's different. >> If you just want to stop DN-1 a short time, just kill the DataNode >> process on DN-1. then do what you want. during this time, Namenode cannot >> receive the heart beat from DN-1, then namenode start to copy block >> replicates on DN-2 to another DN, supposed DN-2. >> >> But when you start DN-1 again, Namenode receive the DN-1 registration, >> then namenode stop to copy the DN-1's block replicates even if NN doesn't >> finish coping. >> >> Am I explain clearly? >> >> >> >> On Wed, Apr 3, 2013 at 9:43 AM, Henry Junyoung Kim <[email protected] >> > wrote: >> >>> @Harsh >>> >>> What's the reasons to make big gaps for removing nodes between >>> decommission and just down nodes? >>> In my understanding, both are necessary to copy un-replicated blocks to >>> another alive nodes. >>> If main costs of them are this one, total elapsed time couldn't be big >>> different. >>> >>> Could you share some articles or documents to understand about >>> decommissioning procedures? >>> - explaining is always thanks ;) >>> >>> >>> 2013. 4. 2., 오후 5:37, Harsh J <[email protected]> 작성: >>> >>> > Yes, you can do the downtime work in steps of 2 DNs at a time, >>> > especially since you mentioned the total work would be only ~30mins at >>> > most. >>> > >>> > On Tue, Apr 2, 2013 at 1:46 PM, Henry Junyoung Kim >>> > <[email protected]> wrote: >>> >> the rest of nodes to be alive has enough size to store. >>> >> >>> >> for this one that you've mentioned. >>> >>> its easier to do so in a rolling manner without need of a >>> >>> decommission. >>> >> >>> >> to check my understanding, just shutting down 2 of them and then 2 >>> more and then 2 more without decommissions. >>> >> >>> >> is this correct? >>> >> >>> >> >>> >> 2013. 4. 2., 오후 4:54, Harsh J <[email protected]> 작성: >>> >> >>> >>> Note though that its only possible to decommission 7 nodes at the >>> same >>> >>> time and expect it to finish iff the remaining 8 nodes have adequate >>> >>> free space for the excess replicas. >>> >>> >>> >>> If you're just going to take them down for a short while (few mins >>> >>> each), its easier to do so in a rolling manner without need of a >>> >>> decommission. You can take upto two down at a time on a replication >>> >>> average of 3 or 3+, and put it back in later without too much data >>> >>> movement impact. >>> >>> >>> >>> On Tue, Apr 2, 2013 at 1:06 PM, Yanbo Liang <[email protected]> >>> wrote: >>> >>>> It's reasonable to decommission 7 nodes at the same time. >>> >>>> But may be it also takes long time to finish it. >>> >>>> Because all the replicas in these 7 nodes need to be copied to >>> remaining 8 >>> >>>> nodes. >>> >>>> The size of transfer from these nodes to the remaining nodes is >>> equal. >>> >>>> >>> >>>> >>> >>>> 2013/4/2 Henry Junyoung Kim <[email protected]> >>> >>>>> >>> >>>>> :) >>> >>>>> >>> >>>>> currently, I have 15 data nodes. >>> >>>>> for some tests, I am trying to decommission until 8 nodes. >>> >>>>> >>> >>>>> Now, the total dfs used size is 52 TB which is including all >>> replicated >>> >>>>> blocks. >>> >>>>> from 15 to 8, total spent time is almost 4 days long. ;( >>> >>>>> >>> >>>>> someone mentioned that I don't need to decommission node by node. >>> >>>>> for this case, is there no problems if I decommissioned 7 nodes at >>> the >>> >>>>> same time? >>> >>>>> >>> >>>>> >>> >>>>> 2013. 4. 2., 오후 12:14, Azuryy Yu <[email protected]> 작성: >>> >>>>> >>> >>>>> I can translate it to native English: how many nodes you want to >>> >>>>> decommission? >>> >>>>> >>> >>>>> >>> >>>>> On Tue, Apr 2, 2013 at 11:01 AM, Yanbo Liang <[email protected]> >>> wrote: >>> >>>>>> >>> >>>>>> You want to decommission how many nodes? >>> >>>>>> >>> >>>>>> >>> >>>>>> 2013/4/2 Henry JunYoung KIM <[email protected]> >>> >>>>>>> >>> >>>>>>> 15 for datanodes and 3 for replication factor. >>> >>>>>>> >>> >>>>>>> 2013. 4. 1., 오후 3:23, varun kumar <[email protected]> 작성: >>> >>>>>>> >>> >>>>>>>> How many nodes do you have and replication factor for it. >>> >>>>>>> >>> >>>>>> >>> >>>>> >>> >>>>> >>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Harsh J >>> >> >>> > >>> > >>> > >>> > -- >>> > Harsh J >>> >>> >> >
