chenhao7253886 commented on a change in pull request #336: First commit of new
tablet repair framework
URL: https://github.com/apache/incubator-doris/pull/336#discussion_r235583891
##########
File path: fe/src/main/java/org/apache/doris/clone/TabletFactory.java
##########
@@ -0,0 +1,655 @@
+package org.apache.doris.clone;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Sets;
+
+import org.apache.doris.catalog.Catalog;
+import org.apache.doris.catalog.Database;
+import org.apache.doris.catalog.MaterializedIndex;
+import org.apache.doris.catalog.OlapTable;
+import org.apache.doris.catalog.Partition;
+import org.apache.doris.catalog.Replica;
+import org.apache.doris.catalog.Replica.ReplicaState;
+import org.apache.doris.catalog.Tablet;
+import org.apache.doris.catalog.Tablet.TabletStatus;
+import org.apache.doris.catalog.TabletInvertedIndex;
+import org.apache.doris.common.Pair;
+import org.apache.doris.common.util.Daemon;
+import org.apache.doris.persist.ReplicaPersistInfo;
+import org.apache.doris.system.Backend;
+import org.apache.doris.system.SystemInfoService;
+import org.apache.doris.task.AgentTaskQueue;
+import org.apache.doris.task.CloneTask;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.PriorityBlockingQueue;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/*
+ * TabletFactory saved the tablets produced by TabletScanner and try to repair
them.
+ * It also try to balance the cluster load if there is no tablet need to be
repaired.
+ *
+ * We are expecting to an efficient way to recovery the entire cluster and
make it balanced.
+ * Case 1:
+ * A Backend is down. All tablets which has replica on this BE should be
repaired as soon as possible.
+ *
+ * Case 1.1:
+ * As Backend is down, some tables should be repaired in high priority. So
the repair task should be able
+ * to preempted.
+ *
+ * Case 2:
+ * A new Backend is added to the cluster. Replicas should be transfer to that
host to balance the cluster load.
+ * 1 sec
+ */
+public class TabletFactory extends Daemon {
+ private static final Logger LOG =
LogManager.getLogger(TabletFactory.class);
+
+ private PriorityBlockingQueue<TabletInfo> q = new
PriorityBlockingQueue<>();;
+ private Set<Long> runningTabletIds = Sets.newHashSet();;
Review comment:
Variable name should't be single letter in most conditions, to describle the
function.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
