SourabhBadhya commented on code in PR #4091:
URL: https://github.com/apache/hive/pull/4091#discussion_r1167991222
##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -3273,11 +3273,11 @@ public static enum ConfVars {
HIVE_COMPACTOR_ABORTEDTXN_THRESHOLD("hive.compactor.abortedtxn.threshold",
1000,
"Number of aborted transactions involving a given table or partition
that will trigger\n" +
- "a major compaction."),
+ "a major compaction / cleanup of aborted directories."),
Review Comment:
This config is used for controlling the number of aborted txns after which
the cleanup is triggered (either cleanup / compaction) depending on what is
expected.
##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -3273,11 +3273,11 @@ public static enum ConfVars {
HIVE_COMPACTOR_ABORTEDTXN_THRESHOLD("hive.compactor.abortedtxn.threshold",
1000,
"Number of aborted transactions involving a given table or partition
that will trigger\n" +
- "a major compaction."),
+ "a major compaction / cleanup of aborted directories."),
HIVE_COMPACTOR_ABORTEDTXN_TIME_THRESHOLD("hive.compactor.aborted.txn.time.threshold",
"12h",
new TimeValidator(TimeUnit.HOURS),
- "Age of table/partition's oldest aborted transaction when compaction
will be triggered. " +
+ "Age of table/partition's oldest aborted transaction when compaction /
cleanup of aborted directories will be triggered. " +
Review Comment:
This config is used for controlling the time threshold of aborted txns after
which the cleanup is triggered (either cleanup / compaction) depending on what
is expected.
##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactorBase.java:
##########
@@ -89,6 +89,7 @@ public void setup() throws Exception {
hiveConf.setBoolVar(HiveConf.ConfVars.HIVEOPTIMIZEMETADATAQUERIES, false);
MetastoreConf.setBoolVar(hiveConf,
MetastoreConf.ConfVars.COMPACTOR_INITIATOR_ON, true);
MetastoreConf.setBoolVar(hiveConf,
MetastoreConf.ConfVars.COMPACTOR_CLEANER_ON, true);
+ MetastoreConf.setBoolVar(hiveConf,
MetastoreConf.ConfVars.COMPACTOR_CLEAN_ABORTS_USING_CLEANER, false);
Review Comment:
The plan is to keep this config till some point until we know that abort
cleanup is stable. If there are any issues with the given handler, we can use
the compaction cycle. I will create a task once its determined that this
feature is stable and we can remove this feature flag then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
