beliefer opened a new issue, #11203: URL: https://github.com/apache/incubator-gluten/issues/11203
### Backend VL (Velox) ### Bug description In our environment, some Hive table with parquet format, but the partitioned formats are different. Some is the same as the table metadata, some is the other format. https://github.com/apache/incubator-gluten/pull/11113 already supports mixed input format for partitioned Hive table. The bug is if some input format for partition is unsupported by Velox, we can see the error. ``` Caused by: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError Error Source: RUNTIME Error Code: INVALID_STATE Reason: (unknown vs. unknown) Retriable: False Expression: baseReaderOpts_.fileFormat() != dwio::common::FileFormat::UNKNOWN Context: Split [Hive: hdfs://fenghuang/data/gamein/ma84_sdc/ods/ods_ma84_raw_hfee/20251020/part-00000-ee5e2912-8723-4289-9da3-6a6ee328d9fe-c000 16777216 - 2421822] Task Gluten_Stage_361_TID_523 5_VTID_460 Additional Context: Operator: TableScan[0] 0 Operator: ValueStream[0] 0 Function: createReader File: /home/hadoop/gluten/ep/build-velox/build/velox_ep/velox/connectors/hive/SplitReader.cpp Line: 225 Stack trace: # 0 _ZN8facebook5velox7process10StackTraceC1Ei # 1 _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_ # 2 _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEEvRKNS1_18VeloxCheckFailArgsET0_ # 3 _ZN8facebook5velox9connector4hive11SplitReader12createReaderEv # 4 _ZN8facebook5velox9connector4hive11SplitReader12prepareSplitESt10shared_ptrINS0_6common14MetadataFilterEERNS0_4dwio6common17RuntimeStatisticsE # 5 _ZN8facebook5velox9connector4hive14HiveDataSource8addSplitESt10shared_ptrINS1_14ConnectorSplitEE # 6 _ZN8facebook5velox4exec9TableScan9getOutputEv # 7 _ZZN8facebook5velox4exec6Driver11runInternalERSt10shared_ptrIS2_ERS3_INS1_13BlockingStateEERS3_INS0_9RowVectorEEENKUlvE3_clEv # 8 _ZN8facebook5velox4exec6Driver11runInternalERSt10shared_ptrIS2_ERS3_INS1_13BlockingStateEERS3_INS0_9RowVectorEE # 9 _ZN8facebook5velox4exec6Driver4nextEPN5folly10SemiFutureINS3_4UnitEEE # 10 _ZN8facebook5velox4exec4Task4nextEPN5folly10SemiFutureINS3_4UnitEEE # 11 _ZN6gluten24WholeStageResultIterator4nextEv # 12 Java_org_apache_gluten_vectorized_ColumnarBatchOutIterator_nativeHasNext # 13 0x00007f5115578328 at org.apache.gluten.vectorized.ColumnarBatchOutIterator.nativeHasNext(Native Method) at org.apache.gluten.vectorized.ColumnarBatchOutIterator.hasNext0(ColumnarBatchOutIterator.java:57) at org.apache.gluten.iterator.ClosableIterator.hasNext(ClosableIterator.java:39) ... 41 more ``` ### Gluten version main branch ### Spark version None ### Spark configurations _No response_ ### System information _No response_ ### Relevant logs ```bash ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
