Yifeng-Wang commented on PR #9107:
URL:
https://github.com/apache/incubator-gluten/pull/9107#issuecomment-3219032842
**Summary**
##### **What has been done:**
**Ⅰ. Rebased previous proposal onto current main branch progress**
Rebased code onto current main branch. This PR still extractS logics for
`case i: StaticInvoke` branch into a separated
`replaceStaticInvokeWithExpressionTransformer`, but now an extra
`validateandTransform` is introduced to do the backend validation. The logic
remain the same which ensures a Transformer is instantiated only when the
underlying backend supports this expression.
##### **Ⅱ. Tests for the new round**
- **Gluten Test Suite**
run `mvn clean install -Pbackends-velox -Pspark-3.5
-DwildcardSuites="org.apache.gluten.functions.ScalarFunctionsValidateSuite"`
(-Pspark-3.4 for Spark v3.4) .
**Validation**:
Got BUILD SUCCESS and checked surefire test reports
"TEST-org.apache.gluten.functions.ScalarFunctionsValidateSuite*" all pass.
- **Spark + Gluten + Velox tests**
Test for all current supported StaticInvoke Expressions.
```SQL
-- url_decode
create table url_tbl(id string) stored as parquet;
insert into url_tbl values ('https%3A%2F%2Fspark.apache.org');
select url_decode(id) from url_tbl; -- https://spark.apache.org
-- url_encode
create table url_tbl2(id string) stored as parquet;
insert into url_tbl2 values ('https://spark.apache.org');
select url_encode(id) from url_tbl2; -- https%3A%2F%2Fspark.apache.org
-- luhn_check, min Spark>=3.5
select luhn_check(id) from url_tbl;
-- base64 and unbase64
select unbase64(base64(id)) from url_tbl;
-- CharVarcharCodegenUtils functions (this PR)
create table srcvarchar(id string) stored as parquet;
insert into srcvarchar values ('ab');
create table tgtvarchar(id varchar(3)) stored as parquet;
create table tgtchar(id char(3)) stored as parquet;
-- varcharTypeWriteSideCheck
insert into tgtvarchar select id from srcvarchar;
-- charTypeWriteSideCheck
insert into tgtchar select id from srcvarchar;
-- readSidePadding
SELECT id FROM tgtchar;
```
- **Validation:**
Verify Transformers within SQL execution plans without fallbacks and results
correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
