eric9204 commented on issue #6011: URL: https://github.com/apache/hudi/issues/6011#issuecomment-1200966001
> @eric9204 #6253 Try this PR. @yuzhaojing The problem has not been solved，the first compaction was successful, the second one failed yet,the error has changed. 2022-08-01 17:07:56,683 INFO org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView [] - Sending request : (http://10.19.29.170:59932/v1/hoodie/vi ew/compactions/pending/?basepath=%2Ftmp%2Ftest_houhang%2Fhudi%2Fhudi_table0801c&lastinstantts=20220801170448141&timelinehash=0a64ba8ceeb4ec20ad9743d990d256dcb fdca1122f61aff27eb96a3939003b87&numinstants=4) 1334 2022-08-01 17:07:57,215 ERROR org.apache.hudi.common.table.view.PriorityBasedFileSystemView [] - Got error running preferred function. Trying secondary 1335 org.apache.hudi.exception.HoodieRemoteException: Connect to 10.19.29.170:59932 [/10.19.29.170] failed: Connection refused (Connection refused) 1336 at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getPendingCompactionOperations(RemoteHoodieTableFileSystemView.java:434) ~[hudi-f link1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1337 at org.apache.hudi.common.table.view.PriorityBasedFileSystemView.execute(PriorityBasedFileSystemView.java:68) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT. jar:0.12.0-SNAPSHOT] 1338 at org.apache.hudi.common.table.view.PriorityBasedFileSystemView.getPendingCompactionOperations(PriorityBasedFileSystemView.java:224) ~[hudi-flink1.14 -bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1339 at org.apache.hudi.table.action.clean.CleanPlanner.<init>(CleanPlanner.java:96) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1340 at org.apache.hudi.table.action.clean.CleanPlanActionExecutor.requestClean(CleanPlanActionExecutor.java:97) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.ja r:0.12.0-SNAPSHOT] 1341 at org.apache.hudi.table.action.clean.CleanPlanActionExecutor.requestClean(CleanPlanActionExecutor.java:141) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.j ar:0.12.0-SNAPSHOT] 1342 at org.apache.hudi.table.action.clean.CleanPlanActionExecutor.execute(CleanPlanActionExecutor.java:166) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0. 12.0-SNAPSHOT] 1343 at org.apache.hudi.table.HoodieFlinkCopyOnWriteTable.scheduleCleaning(HoodieFlinkCopyOnWriteTable.java:316) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.ja r:0.12.0-SNAPSHOT] 1344 at org.apache.hudi.client.BaseHoodieWriteClient.scheduleTableServiceInternal(BaseHoodieWriteClient.java:1355) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT. jar:0.12.0-SNAPSHOT] 1345 at org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:867) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1346 at org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:840) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1347 at org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:894) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1348 at org.apache.hudi.client.BaseHoodieWriteClient.clean(BaseHoodieWriteClient.java:884) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1349 at org.apache.hudi.sink.compact.CompactionCommitSink.doCommit(CompactionCommitSink.java:168) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSH OT] 1350 at org.apache.hudi.sink.compact.CompactionCommitSink.commitIfNecessary(CompactionCommitSink.java:143) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12 .0-SNAPSHOT] 1351 at org.apache.hudi.sink.compact.CompactionCommitSink.invoke(CompactionCommitSink.java:106) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT ] 1352 at org.apache.hudi.sink.compact.CompactionCommitSink.invoke(CompactionCommitSink.java:57) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1353 at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:54) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1354 at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233) ~[flink-dist_2.12-1.14. 4.jar:1.14.4] 1355 at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134) ~[flink-dist_2.12-1.14 .4.jar:1.14.4] 1356 at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105) ~[flink-dist_2.12-1.14.4.jar :1.14.4] 1357 at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1358 at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:496) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1359 at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1360 at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:809) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1361 at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:761) ~[flink-dist_2.12-1.14.4.jar:1.14.4] 1362 at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) [flink-dist_2.12-1.14.4.jar:1.14.4] 1363 at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) [flink-dist_2.12-1.14.4.jar:1.14.4] 1364 at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) [flink-dist_2.12-1.14.4.jar:1.14.4] 1365 at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) [flink-dist_2.12-1.14.4.jar:1.14.4] 1366 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_271] 1367 Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 10.19.29.170:59932 [/10.19.29.170] failed: Connection refused (Connection refused) 1368 at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) ~[hudi-flink1.14-bundle-0.12.0- SNAPSHOT.jar:0.12.0-SNAPSHOT] 1369 at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[hudi-flink1.14-bundle-0.12.0-SN APSHOT.jar:0.12.0-SNAPSHOT] 1370 at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1371 at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1372 at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1373 at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1374 at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1375 at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1376 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1377 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1378 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1379 at org.apache.http.client.fluent.Request.execute(Request.java:151) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1380 at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:174) ~[hudi-flink1.14-bundle- 0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1381 at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getPendingCompactionOperations(RemoteHoodieTableFileSystemView.java:430) ~[hudi-f link1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1382 ... 30 more 1383 Caused by: java.net.ConnectException: Connection refused (Connection refused) 1384 at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_271] 1385 at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476) ~[?:1.8.0_271] 1386 at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218) ~[?:1.8.0_271] 1387 at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200) ~[?:1.8.0_271] 1388 at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394) ~[?:1.8.0_271] 1389 at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_271] 1390 at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHO T.jar:0.12.0-SNAPSHOT] 1391 at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ~[hudi-flink1.14-bundle-0.12.0- SNAPSHOT.jar:0.12.0-SNAPSHOT] 1392 at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[hudi-flink1.14-bundle-0.12.0-SN APSHOT.jar:0.12.0-SNAPSHOT] 1393 at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1394 at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1395 at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1396 at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1397 at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1398 at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1399 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1400 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1401 at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1402 at org.apache.http.client.fluent.Request.execute(Request.java:151) ~[hudi-flink1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1403 at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:174) ~[hudi-flink1.14-bundle- 0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1404 at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getPendingCompactionOperations(RemoteHoodieTableFileSystemView.java:430) ~[hudi-f link1.14-bundle-0.12.0-SNAPSHOT.jar:0.12.0-SNAPSHOT] 1405 ... 30 more 1406 2022-08-01 17:07:57,224 INFO org.apache.hudi.table.action.clean.CleanPlanner [] - No earliest commit to retain. No need to scan partitions !! 1407 2022-08-01 17:07:57,225 INFO org.apache.hudi.table.action.clean.CleanPlanner [] - Nothing to clean here. It is already clean 1408 2022-08-01 17:07:57,233 INFO org.apache.hudi.common.table.timeline.HoodieActiveTimeline [] - Loaded instants upto : Option{val=[==>20220801170510574__delta commit__INFLIGHT]} -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
