ccat3z opened a new issue, #9846: URL: https://github.com/apache/incubator-gluten/issues/9846
### Backend VL (Velox) ### Bug description If reclaim happened in TableScan preload's IO thread during `TableScan::getOuput()` waiting async preload, reclaimTask will wait `TableScan::getOuput()` until timeout. ``` org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError Error Source: RUNTIME Error Code: INVALID_STATE Reason: Operator::getOutput failed for [operator: TableScan, plan node ID: 0]: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError Error Source: RUNTIME Error Code: INVALID_STATE Reason: Memory reclaim failed to wait for task Gluten_Stage_2_TID_12269_VTID_1 to pause after 1h 0m 0s with max timeout 1h 0m 0s Retriable: False Context: Split Hive: ... File: /tmp/build/src/dev/meituan/src/velox/velox/exec/Task.cpp Line: 3161 Stack trace: # 0 _ZN8facebook5velox7process10StackTraceC1Ei # 1 _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_ # 2 _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_ # 3 _ZN8facebook5velox4exec4Task15MemoryReclaimer11reclaimTaskERKSt10shared_ptrIS2_EmmRNS0_6memory15MemoryReclaimer5StatsE # 4 _ZN8facebook5velox4exec4Task15MemoryReclaimer7reclaimEPNS0_6memory10MemoryPoolEmmRNS4_15MemoryReclaimer5StatsE # 5 _ZN8facebook5velox6memory15MemoryReclaimer7reclaimEPNS1_10MemoryPoolEmmRNS2_5StatsE # 6 _ZN6gluten20ListenableArbitrator14shrinkCapacityEmbb # 7 _ZN6gluten24WholeStageResultIterator14spillFixedSizeEl # 8 Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeSpill # 9 0x00007fd0235a2a34 ``` Related to https://github.com/apache/incubator-gluten/issues/7800 and https://github.com/apache/incubator-gluten/pull/7799. ### Gluten version _No response_ ### Spark version None ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
