pvary commented on a change in pull request #2612:
URL: https://github.com/apache/hive/pull/2612#discussion_r699243555
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
##########
@@ -93,6 +93,11 @@ public CompactionTxnHandler() {
try {
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
stmt = dbConn.createStatement();
+ // De-duplicate
Review comment:
Shouldn't we check that the corresponding transaction is not aborted, or
open?
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
##########
@@ -93,6 +93,11 @@ public CompactionTxnHandler() {
try {
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
stmt = dbConn.createStatement();
+ // De-duplicate
+ stmt.execute("DELETE FROM \"COMPLETED_TXN_COMPONENTS\" WHERE
CTC_WRITEID\" NOT IN ("+
Review comment:
Materialized view invalidation is dependent on `CTC_UPDATE_DELETE`
values, so I think we should keep different values if there are any. See
`TxnHandler.getMaterializationInvalidationInfo`. Maybe @kasakrisz knows more
about this.
Also I think moving this query to `AcidHouseKeeperService` would be good, as
this is housekeeping.
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
##########
@@ -93,6 +93,11 @@ public CompactionTxnHandler() {
try {
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
stmt = dbConn.createStatement();
+ // De-duplicate
Review comment:
Got it. Thanks!
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java
##########
@@ -93,6 +93,11 @@ public CompactionTxnHandler() {
try {
dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
stmt = dbConn.createStatement();
+ // De-duplicate
+ stmt.execute("DELETE FROM \"COMPLETED_TXN_COMPONENTS\" WHERE
CTC_WRITEID\" NOT IN ("+
Review comment:
Should we just advise the user to decrease the frequency of the
`AcidHouseKeeperService`? Or use a different config?
IMHO it would be better than putting the code to a different place, just
because of the frequency is lower there...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
