kangkaisen opened a new issue #2672: Remove Transaction need a very long time URL: https://github.com/apache/incubator-doris/issues/2672 **Describe the bug** ``` 2019-12-31 12:00:04,581 INFO 83 [GlobalTransactionMgr.removeOldTransactions():704] remove old transactions begin 2019-12-31 12:00:04,693 INFO 83 [GlobalTransactionMgr.removeOldTransactions():779] transactionsToDelete size is 1001 2019-12-31 12:00:05,235 INFO 83 [GlobalTransactionMgr.removeOldTransactions():784] remove old transactions done 2019-12-31 12:28:19,489 INFO 83 [GlobalTransactionMgr.removeOldTransactions():704] remove old transactions begin 2019-12-31 12:28:19,529 INFO 83 [GlobalTransactionMgr.removeOldTransactions():779] transactionsToDelete size is 1001 2019-12-31 12:28:20,039 INFO 83 [GlobalTransactionMgr.removeOldTransactions():784] remove old transactions done ```  In Doris 0.10. The removeOldTransactions will run a long time: **More than ten minutes** which will lead to all realtime load job failed, which is a serious case. In Doris 0.10. I tmp fix this issue by make each `removeOldTransactions` only remove 1000 or 2000 transaction. We could also use `future` and `Interrupt` to ensure each `removeOldTransactions` only run certain time, but I think **the essential solution is improve the remove transaction performance.** Currently, we could reduce decrease `transaction_clean_interval_second` value, and make remove transaction interval as short as possible and each remove transaction cycle as soon as possible
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
