Here's today's notes for future reference: https://docs.google.com/document/d/1jXM5Ujvf-zhcyw_5kiQVx6g-HeKe-YGnFS_1-qFXomI/edit?usp=sharing 10/16/2019
Attendee: Weichiu, Cynthia, Craig, Stephen, Akira, David Stephen introduced upgrade domain, which was developed at Twitter. Cloudera is going to support this feature in the next release. The feature was developed a few years back and quite complete, so Cloudera is just adding UI and verification/guardrails to support this feature. Akira is interested in decommission and maintenance mode. Decomm is slow at Y! Japan. Akira’s interested in maintenance mode too, but they are on 2.6.x so can’t try yet. Stephen introduced the decommissioning improvement project. Decommissioning in practice has a few weird behavior and tend to be slow. HDFS-14814 a new decommissioning monitor. It reduces NameNode lock holding time, and spread replication load across DataNodes. It also gives priority to dead nodes than decommissioning nodes. But it’s hard to simulate its performance. It will have to run on a real large cluster to prove it works. Looking for community members to pick it up and introduce it in some large clusters to try out. HDFS-14861 instead of letting the block to go to the end of replication queue, iterator is reset periodically. EC is not considered yet. Next week we will have the Hadoop storage community sync for the APAC time (PDT 10pm Wednesday, CST 1pm Thursday). Looking for topics. Best, Weichiu
