wForget commented on PR #5591:
URL: https://github.com/apache/kyuubi/pull/5591#issuecomment-1811711951
> This may lose the ordering of the query data, e.g. `order by xx limit 100`
I did a simple test and the results were as expected. (Test Env: Kyuubi
1.8.0 + Spark 3.5.0)
```
create table wangzhen_test_20231115_t1(id bigint, name string) stored as
parquet;
insert into wangzhen_test_20231115_t1 values (1, 'a');
insert into wangzhen_test_20231115_t1 values (2, 'b');
insert into wangzhen_test_20231115_t1 values (3, 'c');
set kyuubi.operation.language=scala;
val df = spark.sql("select * from wangzhen_test_20231115_t1 order by id
limit 2");
df.write.format("parquet").save("hfds://XXX/result.parquet");
spark.sql("set kyuubi.operation.language=sql");
select * from `parquet`.`hfds://XXX/result.parquet`;
```
```
== Physical Plan ==
Execute InsertIntoHadoopFsRelationCommand (5)
+- WriteFiles (4)
+- TakeOrderedAndProject (3)
+- * ColumnarToRow (2)
+- Scan parquet spark_catalog.XXX.wangzhen_test_20231115_t1 (1)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
