v3nkatesh commented on a change in pull request #685: default implementation
for HBase index qps allocator
URL: https://github.com/apache/incubator-hudi/pull/685#discussion_r286636478
##########
File path:
hoodie-client/src/main/java/com/uber/hoodie/config/HoodieHBaseIndexConfig.java
##########
@@ -69,6 +76,34 @@
*/
public static final float DEFAULT_HBASE_QPS_FRACTION = 0.5f;
+ /**
+ * Property to decide if HBASE_QPS_FRACTION_PROP is dynamically calculated
based on volume
+ */
+ public static final String HOODIE_INDEX_COMPUTE_QPS_DYNAMICALLY =
"hoodie.index.hbase.dynamic_qps";
+ public static final boolean DEFAULT_HOODIE_INDEX_COMPUTE_QPS_DYNAMICALLY =
false;
+ /**
+ * Min and Max for HBASE_QPS_FRACTION_PROP to stabilize skewed volume
workloads
+ */
+ public static final String HBASE_MIN_QPS_FRACTION_PROP =
"hoodie.index.hbase.min.qps.fraction";
+ public static final String DEFAULT_HBASE_MIN_QPS_FRACTION_PROP = "0.002";
+
+ public static final String HBASE_MAX_QPS_FRACTION_PROP =
"hoodie.index.hbase.max.qps.fraction";
+ public static final String DEFAULT_HBASE_MAX_QPS_FRACTION_PROP = "0.06";
+ /**
+ * Hoodie index desired puts operation time in seconds
+ */
+ public static final String HOODIE_INDEX_DESIRED_PUTS_TIME =
"hoodie.index.hbase.desired_puts_time";
+ public static final int DEFAULT_HOODIE_INDEX_DESIRED_PUTS_TIME = 600;
+ public static final String HBASE_SLEEP_MS_PUT_BATCH_PROP =
"hoodie.index.hbase.sleep.ms.for.put.batch";
+ public static final String HBASE_SLEEP_MS_GET_BATCH_PROP =
"hoodie.index.hbase.sleep.ms.for.get.batch";
+ public static final String HOODIE_INDEX_HBASE_ZK_SESSION_TIMEOUT_MS =
"hoodie.index.hbase.zk.session_timeout_ms";
+ public static final int DEFAULT_ZK_SESSION_TIMEOUT_MS = 60 * 1000;
+ public static final String HOODIE_INDEX_HBASE_ZK_CONNECTION_TIMEOUT_MS =
+ "hoodie.index.hbase.zk.connection_timeout_ms";
+ public static final int DEFAULT_ZK_CONNECTION_TIMEOUT_MS = 15 * 1000;
+ public static final String HBASE_ZK_ZNODEPARENT_QPS_ROOT =
"hoodie.index.hbase.zknode.qps_root_path";
+ public static final String DEFAULT_HBASE_ZK_ZNODEPARENT_QPS_ROOT =
"/hoover/QPS_ROOT";
Review comment:
updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
