bq. The region exits are forcing the data to start transferring to other regions,
Can you elaborate a bit more on the above ? Did you mean some regions were transferred to other region server(s) ? Can you determined the effective RegionSplitPolicy ? Snippets of server logs would help illustrate the situation. Thanks On Tue, Nov 10, 2015 at 8:09 PM, Abraham Tom <[email protected]> wrote: > I am on CDH 5.4.8 hbase 1.0 > > I am migrating to a new cluster and the data is sourced from CDH 5.3 hbase > 0.98.6 > > I took snapshots from the old cluster exported them to the new cluster - no > errors > > cloned the table, flush, major_compact and balanced the regions > > the configs are practically identical to the old cluster > > > > hbck and fsck all check out with no inconsistencies found > > > > yet when we try to load data via thrift the regions start splitting at an > outrageous pace. > > Eventually the regions become offline starting a snowball affect. The > region exits are forcing the data to start transferring to other regions, > with that creates large GC activity and multiple regions in transition > before thrift can no longer execute any writes. This usually happens after > a few records. How we put is that we perform rowkey scans before puts to > ensure where we last left off. > > > > Thrift eventually stops writing due to regions being offline and no longer > serving. The regions eventually stop splitting and all the GC eventually > complete. Once things settle back down, I am able to perform hbck and fsck > and it again comes back with no inconsistencies > > > > I need ideas as to what is causing this > > > -- > Abraham Tom > Email: [email protected] > Phone: 415-515-3621 >
