hf200012 opened a new issue #5635: URL: https://github.com/apache/incubator-doris/issues/5635
This problem did not occur in version 0.13.15 The specific phenomenon is as follows ： Stream load task error, report: failed to call frontend service The number of txns quickly reached the upper limit：errCode = 2, detailMessage = current running txns on db 11001 is 150, larger than limit 150 View backends heartbeat loss [https://github.com/hf200012/images/blob/master/doris/20210412145426.png](url) Restart BE, the log shows that the heartbeat of FE has not been received To check transactions, it shows that many tasks are not over ： [https://github.com/hf200012/images/blob/master/doris/20210412145444.png](url) **Finally, only restart FE, and then return to normal** Fe master keeps refreshing the following logs ： 2021-04-12 11:45:43,053 WARN (thrift-server-pool-297|673) [ReportHandler.putToQueue():182] the report queue size exceeds the limit: 120. current: 121 2021-04-12 11:45:43,060 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD 2021-04-12 11:45:43,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902] 2021-04-12 11:45:43,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip 2021-04-12 11:45:43,119 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605972] is expired, remove it from transaction manager 2021-04-12 11:45:43,165 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605973] is expired, remove it from transaction manager 2021-04-12 11:45:43,211 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605974] is expired, remove it from transaction manager 2021-04-12 11:45:43,257 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605979] is expired, remove it from transaction manager 2021-04-12 11:45:43,302 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605981] is expired, remove it from transaction manager 2021-04-12 11:45:43,348 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605984] is expired, remove it from transaction manager 2021-04-12 11:45:43,400 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605985] is expired, remove it from transaction manager 2021-04-12 11:45:43,451 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17605986] is expired, remove it from transaction manager 2021-04-12 11:45:43,502 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606000] is expired, remove it from transaction manager 2021-04-12 11:45:43,554 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606001] is expired, remove it from transaction manager 2021-04-12 11:45:43,605 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606007] is expired, remove it from transaction manager 2021-04-12 11:45:43,663 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606008] is expired, remove it from transaction manager 2021-04-12 11:45:43,720 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606009] is expired, remove it from transaction manager 2021-04-12 11:45:43,771 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606017] is expired, remove it from transaction manager 2021-04-12 11:45:43,823 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606018] is expired, remove it from transaction manager 2021-04-12 11:45:43,874 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606031] is expired, remove it from transaction manager 2021-04-12 11:45:43,926 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606033] is expired, remove it from transaction manager 2021-04-12 11:45:43,977 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606034] is expired, remove it from transaction manager 2021-04-12 11:45:44,029 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606040] is expired, remove it from transaction manager 2021-04-12 11:45:44,061 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD 2021-04-12 11:45:44,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902] 2021-04-12 11:45:44,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip 2021-04-12 11:45:44,080 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606041] is expired, remove it from transaction manager 2021-04-12 11:45:44,132 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606045] is expired, remove it from transaction manager 2021-04-12 11:45:44,183 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606046] is expired, remove it from transaction manager 2021-04-12 11:45:44,235 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606047] is expired, remove it from transaction manager 2021-04-12 11:45:44,286 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606074] is expired, remove it from transaction manager 2021-04-12 11:45:44,338 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606075] is expired, remove it from transaction manager 2021-04-12 11:45:44,389 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606081] is expired, remove it from transaction manager 2021-04-12 11:45:44,441 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606082] is expired, remove it from transaction manager 2021-04-12 11:45:44,492 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606083] is expired, remove it from transaction manager 2021-04-12 11:45:44,544 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606089] is expired, remove it from transaction manager 2021-04-12 11:45:44,595 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606092] is expired, remove it from transaction manager 2021-04-12 11:45:44,647 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606107] is expired, remove it from transaction manager 2021-04-12 11:45:44,698 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606109] is expired, remove it from transaction manager 2021-04-12 11:45:44,750 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606111] is expired, remove it from transaction manager 2021-04-12 11:45:44,801 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606119] is expired, remove it from transaction manager 2021-04-12 11:45:44,853 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606120] is expired, remove it from transaction manager 2021-04-12 11:45:44,904 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606124] is expired, remove it from transaction manager 2021-04-12 11:45:44,956 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606125] is expired, remove it from transaction manager 2021-04-12 11:45:45,007 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606126] is expired, remove it from transaction manager 2021-04-12 11:45:45,022 WARN (thrift-server-pool-852|14940) [ReportHandler.putToQueue():182] the report queue size exceeds the limit: 120. current: 121 2021-04-12 11:45:45,062 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():105] get number of low load paths: 10, with medium: HDD 2021-04-12 11:45:45,063 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():186] select alternative tablets for cluster: default_cluster, medium: HDD, num: 2, detail: [5092902, 5092902] 2021-04-12 11:45:45,063 INFO (tablet scheduler|39) [BeLoadRebalancer.selectAlternativeTabletsForCluster():84] cluster is balance: default_cluster with medium: SSD. skip 2021-04-12 11:45:45,066 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606135] is expired, remove it from transaction manager 2021-04-12 11:45:45,123 INFO (txnCleaner|69) [DatabaseTransactionMgr.removeExpiredTxns():1101] transaction [17606136] is expired, remove it from transaction manager -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
