wForget commented on PR #6942: URL: https://github.com/apache/incubator-gluten/pull/6942#issuecomment-2301556790
@zhztheplayer @PHILO-HE Could you please help to see if the [failed GA](https://opencicd.kyligence.com/job/gluten/job/gluten-ci/11653/) is a relevant failure? ``` 16:32:19 - test cache file command *** FAILED *** 16:32:19 org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4) (gluten-gluten-ci-11653-llqgx-65q8j-rm2bs executor driver): org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Open file(hdfs://127.0.0.1:8020/tpch-data/lineitem/part-00000-d08071cb-0dfa-42dc-9198-83cb334ccda3-c000.snappy.parquet) failed. IOError: Code: 107. DB::ErrnoException: Cannot open file /tmp/gluten_hdfs_cache//4654/6d2/6d25666d180cc924b04f3234d772995f/0: , errno: 2, strerror: No such file or directory: Cache info: Buffer path: tpch-data/lineitem/part-00000-d08071cb-0dfa-42dc-9198-83cb334ccda3-c000.snappy.parquet, hash key: 6d25666d180cc924b04f3234d772995f, file_offset_of_buffer_end: 19164575, read_until_position: 19230111, internal buffer end: None, read_type: CACHED, last caller: 215fdc3e-15d9-4926-b9ad-9201b2f74587:57933 , file segment info: File segment: [0, 19230110], key: 6d25666d180cc924b04f3234d772995f, state: DOWNLOADED, downloaded size: 19230111, reserved size: 19230111, downloader id: None, current write offset: 19230111, caller id: 215fdc3e-15d9-4926-b9ad-9201b2f74587:57933, kind: Regular, unbound: 0. (FILE_DOESNT_EXIST) (version 24.9.1.1): While executing SubstraitFileSource 16:32:19 0. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bc09edb 16:32:19 1. ../ClickHouse/src/Common/Exception.h:110: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x00000000064a3f2c 16:32:19 2. ../ClickHouse/src/Common/Exception.h:128: DB::Exception::Exception<String const&, String>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<String>::type>, String const&, String&&) @ 0x00000000064c036b 16:32:19 3. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Storages/SubstraitSource/ParquetFormatFile.cpp:146: local_engine::ParquetFormatFile::collectRequiredRowGroups(DB::ReadBuffer*, int&) const @ 0x000000000c30ee5f 16:32:19 4. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Storages/SubstraitSource/ParquetFormatFile.cpp:70: local_engine::ParquetFormatFile::createInputFormat(DB::Block const&) @ 0x000000000c30d3e6 16:32:19 5. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Storages/SubstraitSource/SubstraitFileSource.cpp:345: local_engine::NormalFileReader::NormalFileReader(std::shared_ptr<local_engine::FormatFile> const&, std::shared_ptr<DB::Context const> const&, DB::Block const&, DB::Block const&) @ 0x000000000c3059f2 16:32:19 6. ../ClickHouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714: local_engine::SubstraitFileSource::generate() @ 0x000000000c303049 16:32:19 7. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/ISource.cpp:139: DB::ISource::tryGenerate() @ 0x000000001013ac5b 16:32:19 8. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/ISource.cpp:108: DB::ISource::work() @ 0x000000001013a967 16:32:19 9. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/Executors/ExecutionThreadContext.cpp:47: DB::ExecutionThreadContext::executeTask() @ 0x00000000101550c9 16:32:19 10. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/Executors/PipelineExecutor.cpp:279: DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x0000000010148570 16:32:19 11. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/Executors/PipelineExecutor.cpp:153: DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x0000000010147f48 16:32:19 12. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/Executors/PullingPipelineExecutor.cpp:54: DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000001015ba77 16:32:19 13. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Processors/Executors/PullingPipelineExecutor.cpp:65: DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x000000001015bc99 16:32:19 14. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Parser/SerializedPlanParser.cpp:1613: local_engine::LocalExecutor::hasNext() @ 0x000000000c030e45 16:32:19 15. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/local_engine_jni.cpp:277: Java_org_apache_gluten_vectorized_BatchIterator_nativeHasNext @ 0x0000000006487e77 16:32:19 16:32:19 0. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000bc09edb 16:32:19 1. ../ClickHouse/src/Common/Exception.h:109: DB::Exception::createRuntime(int, String&) @ 0x00000000064a042c 16:32:19 2. ../local-engine/jni/jni_common.h:98: unsigned char local_engine::safeCallBooleanMethod<>(JNIEnv_*, _jobject*, _jmethodID*) @ 0x00000000064a0f5f 16:32:19 3. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Shuffle/NativeSplitter.cpp:161: local_engine::NativeSplitter::hasNext() @ 0x000000000c12c1e5 16:32:19 4. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/local_engine_jni.cpp:1147: Java_org_apache_gluten_vectorized_BlockSplitIterator_nativeHasNext @ 0x000000000649ab37 16:32:19 16:32:19 at org.apache.gluten.vectorized.BlockSplitIterator.nativeHasNext(Native Method) 16:32:19 at org.apache.gluten.vectorized.BlockSplitIterator.hasNext(BlockSplitIterator.java:55) 16:32:19 at org.apache.spark.sql.execution.utils.CHExecUtil$$anon$2.hasNext(CHExecUtil.scala:165) 16:32:19 at org.apache.gluten.vectorized.CloseablePartitionedBlockIterator.hasNext(CloseablePartitionedBlockIterator.scala:33) 16:32:19 at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140) 16:32:19 at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59) 16:32:19 at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99) 16:32:19 at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52) 16:32:19 at org.apache.spark.scheduler.Task.run(Task.scala:136) 16:32:19 at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) 16:32:19 at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) 16:32:19 at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) 16:32:19 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 16:32:19 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 16:32:19 at java.lang.Thread.run(Thread.java:748) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
