ehurheap commented on issue #9903: URL: https://github.com/apache/hudi/issues/9903#issuecomment-2125644288
I also ran into this problem when running: `compaction validate --instant 20240516172801913` The validation itself appears to complete ok because I see this output: ``` 24/05/22 17:58:05 INFO DAGScheduler: Job 0 finished: collect at HoodieSparkEngineContext.java:103, took 297.868011 s Result of Validation Operation : 24/05/22 17:58:05 INFO MultipartUploadOutputStream: close closed:false s3://bucketname-redacted/tmp/b0fcc58c-da3a-4a74-90f5-4a63bca5ada8.ser 24/05/22 17:58:05 INFO BlockManagerInfo: Removed broadcast_0_piece0 on ip-10-18-146-49.heap:44129 in memory (size: 598.3 KiB, free: 2.2 GiB) 24/05/22 17:58:05 INFO MultipartUploadOutputStream: close closed:true s3://bucketname-redacted/tmp/b0fcc58c-da3a-4a74-90f5-4a63bca5ada8.ser 24/05/22 17:58:05 INFO Javalin: Stopping Javalin ... 24/05/22 17:58:05 INFO Javalin: Javalin has stopped 24/05/22 17:58:05 INFO SparkUI: Stopped Spark web UI at http://ip-10-18-146-49.heap:4040 24/05/22 17:58:05 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! ``` then it continues shutting down spark but then: ``` 24/05/22 17:58:05 INFO SparkContext: Successfully stopped SparkContext 24/05/22 17:58:05 INFO ShutdownHookManager: Shutdown hook called 24/05/22 17:58:05 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-ab2d5508-e1e0-4f35-af09-d75a368a84b6 24/05/22 17:58:05 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-51ff5e59-d464-4f7a-8ed5-dbd4906ea294 Command failed java.lang.IllegalArgumentException: Wrong FS: s3:/tmp/b0fcc58c-da3a-4a74-90f5-4a63bca5ada8.ser, expected: s3://bucketname-redacted OperationResult{operation=CompactionOperation{baseInstantTime=.. etc the details of the compaction here} ``` hudi version 0.13.1 hudi-cli version 1.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
