acvictor commented on PR #11618: URL: https://github.com/apache/incubator-gluten/pull/11618#issuecomment-3991449699
> @acvictor Thanks for the fix. The code looks good. However in the log, it seems there are still some fallback on scan reported, is this expected? https://github.com/apache/incubator-gluten/actions/runs/22039361874/job/63677680172?pr=11618#step:8:8427 @zhouyuan this is expcted. Baseline ``` 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=34], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=34], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260302 15:03:48.568771 27742 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2464, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: Scan parquet spark_catalog.default.dynparttest2, due to: - Unsupported decimal partition column in native scan. 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: ColumnarToRow, due to: - Unsupported decimal partition column in native scan. 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: Scan parquet spark_catalog.default.dynparttest2[QueryId=36], due to: - Unsupported decimal partition column in native scan. 26/03/02 15:03:48 WARN GlutenFallbackReporter: Validation failed for plan: ColumnarToRow[QueryId=36], due to: - Unsupported decimal partition column in native scan. - Incorrect decimal casting for partition read ``` This PR ``` 26/02/15 17:07:03 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=34], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:03 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=34], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:03.596613 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:03 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=40], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:03 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=40], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:04.033113 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:04 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=46], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:04 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=46], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:04.451627 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:04 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=52], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:04 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=52], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:04.846966 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=58], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=58], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:05.233858 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=59], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Project[QueryId=59], due to: - Validation failed with exception from: ProjectExecTransformer, reason: CheckOverflowInTableInsert is used in ANSI mode, but Gluten does not support ANSI mode. E20260215 17:07:05.426112 27433 Exceptions.h:53] Line: /work/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2455, Function:terminate, Expression: Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=61], due to: [FallbackByBackendSettings] Validation failed on node Exchange 26/02/15 17:07:05 WARN GlutenFallbackReporter: Validation failed for plan: Exchange[QueryId=62], due to: [FallbackByBackendSettings] Validation failed on node Exchange - Incorrect decimal casting for partition read ``` The Exchange/Project fallbacks with CheckOverflowInTableInsert are pre-existing on the INSERT path and the baseline also has this. This PR has more instances because I extended the test to go from 1 INSERT to 6 INSERTs to cover additional decimal scenarios. The logs do show an improvement from the baseline, because Scan parquet spark_catalog.default.dynparttest2 was previously falling back with "Unsupported decimal partition column in native scan." but in this PR, that scan fallback is eliminated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
