Hi everyone,
Recently Velox has deprecated threshold-based spill for
Aggragation/Join/Orderby and related configs are removed. Gluten also did a
cleanup in this
commit<https://github.com/apache/incubator-gluten/commit/f51b7ab230a19fd9ddeab7eb53996c308c38b46a>.
So please note that DON'T use following options in your configurations:
spark.gluten.sql.columnar.backend.velox.aggregationSpillMemoryThreshold
spark.gluten.sql.columnar.backend.velox.joinSpillMemoryThreshold
spark.gluten.sql.columnar.backend.velox.orderBySpillMemoryThreshold
Thanks,
Yan