This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 149e15589b65 [SPARK-55128][INFRA][FOLLOWUP] Restore SQL tests by pin
'pandas==2.3.3' for maven daily test
149e15589b65 is described below
commit 149e15589b65139f79453ba4325ab8a44bfae28e
Author: yangjie01 <[email protected]>
AuthorDate: Fri Jan 23 16:07:36 2026 +0800
[SPARK-55128][INFRA][FOLLOWUP] Restore SQL tests by pin 'pandas==2.3.3' for
maven daily test
### What changes were proposed in this pull request?
Similar to https://github.com/apache/spark/pull/53910, this pr pins the
pandas version to 2.3.3.
### Why are the changes needed?
To restore SQL tests for maven daily test.
- https://github.com/apache/spark/actions/runs/21249870076/job/61148348328
```
- udf/postgreSQL/udf-case.sql - Scalar Pandas UDF *** FAILED ***
udf/postgreSQL/udf-case.sql - Scalar Pandas UDF
Python: 3.11 Pandas: 3.0.0 PyArrow: 23.0.0
Expected Some("struct<Two:string,i:int,f:double,i:int,j:int>"), but got
Some("struct<>") Schema did not match for query #30
SELECT '' AS `Two`, *
FROM CASE_TBL a, CASE2_TBL b
WHERE udf(COALESCE(f,b.i) = 2): -- !query
SELECT '' AS `Two`, *
FROM CASE_TBL a, CASE2_TBL b
WHERE udf(COALESCE(f,b.i) = 2)
-- !query schema
struct<>
-- !query output
org.apache.spark.SparkRuntimeException
{
"errorClass" : "CAST_INVALID_INPUT",
"sqlState" : "22018",
"messageParameters" : {
"ansiConfig" : "\"spark.sql.ansi.enabled\"",
"expression" : "'nan'",
"sourceType" : "\"STRING\"",
"targetType" : "\"BOOLEAN\""
},
"queryContext" : [ {
"objectType" : "",
"objectName" : "",
"startIndex" : 62,
"stopIndex" : 85,
"fragment" : "udf(COALESCE(f,b.i) = 2)"
} ]
} (SQLQueryTestSuite.scala:681)
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
monitor maven daily test after pr merged
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53933 from LuciferYang/SPARK-55128-FOLLOWUP.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit 3f1c9a37a53cdc45eef173b343c0526138927c3d)
Signed-off-by: yangjie01 <[email protected]>
---
.github/workflows/maven_test.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.github/workflows/maven_test.yml b/.github/workflows/maven_test.yml
index 74bd818a522d..7828c4bdc6ed 100644
--- a/.github/workflows/maven_test.yml
+++ b/.github/workflows/maven_test.yml
@@ -176,7 +176,7 @@ jobs:
- name: Install Python packages (Python 3.11)
if: contains(matrix.modules, 'resource-managers#yarn') ||
(contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect')
run: |
- python3.11 -m pip install 'numpy>=1.22' pyarrow pandas pyyaml scipy
unittest-xml-reporting 'grpcio==1.76.0' 'grpcio-status==1.76.0'
'protobuf==6.33.0' 'zstandard==0.25.0'
+ python3.11 -m pip install 'numpy>=1.22' pyarrow 'pandas==2.3.3'
pyyaml scipy unittest-xml-reporting 'grpcio==1.76.0' 'grpcio-status==1.76.0'
'protobuf==6.33.0' 'zstandard==0.25.0'
python3.11 -m pip list
# Run the tests using script command.
# BSD's script command doesn't support -c option, and the usage is
different from Linux's one.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
