kecookier opened a new issue, #6846:
URL: https://github.com/apache/incubator-gluten/issues/6846
### Backend
VL (Velox)
### Bug description
```
24/08/14 20:34:21 ERROR TaskResources: Task 1 failed by error:
java.lang.UnsupportedOperationException
at
org.apache.gluten.vectorized.ColumnarBatchSerializerInstance.serializeStream(ColumnarBatchSerializer.scala:244)
at
org.apache.spark.storage.DiskBlockObjectWriter.open(DiskBlockObjectWriter.scala:157)
at
org.apache.spark.storage.DiskBlockObjectWriter.write(DiskBlockObjectWriter.scala:279)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:172)
at
org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:102)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
at org.apache.spark.scheduler.Task.run(Task.scala:134)
at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:479)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1426)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:482)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
```
SQL:
select b_deal_id ,max(price) ,max(market_price) from tableA where
partition_date='2024-08-03' AND b_deal_id = 114360 group by b_deal_id;
### Spark version
None
### Spark configurations
```
spark.plugins=org.apache.gluten.GlutenPlugin
spark.gluten.sql.columnar.backend.lib=velox
spark.memory.offHeap.enabled=true
spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager
spark.adjust.gluten.memory.enabled=true
spark.gluten.on.heap.memory.ratio=0.25
spark.gluten.off.heap.memory.ratio=0.75
spark.gluten.sql.columnar.shuffle.writeEOS=false
spark.gluten.sql.columnar.backend.velox.enableUserExceptionStacktrace=false
spark.gluten.sql.columnar.collectFallbackReason=true
spark.gluten.sql.complexType.scan.fallback.enabled=false
spark.gluten.sql.columnar.backend.velox.glogSeverityLevel=0
#spark.gluten.sql.columnar.backend.velox.glogVerboseLevel=0
spark.gluten.sql.debug=true
```
### System information
Velox System Info v0.0.2
Commit: 37211f7868f1dd150134dcc771fe0712ecc3c457
CMake Version: 3.22.1
System: Linux-3.10.0-862.el7.x86_64
Arch: x86_64
CPU Name: Model name: Intel(R) Xeon(R) Platinum 8255C CPU @
2.50GHz
C++ Compiler: /usr/lib64/ccache/c++
C++ Compiler Version: 9.3.0
C Compiler: /usr/lib64/ccache/cc
C Compiler Version: 9.3.0
CMake Prefix Path: /usr/local;/usr
### Relevant logs
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
