yikf commented on issue #8043: URL: https://github.com/apache/incubator-gluten/issues/8043#issuecomment-2499822005
@FelixYBW thank you for initiating this matter, and at the same time, I noticed some configuration related to table write. Please correct me if missing something. | Spark Config | Respected by Gluten | Gluten Config | Transparent to Gluten | | ----------- | ----------- | ----------- | ----------- | | spark.sql.parquet.compression.codec | Y | N/A | Y | the Parquet configuration options in the table options are also respected by Gluten, as follows: | Options | Respected by Gluten | Gluten Config | Transparent to Gluten | | ----------- | ----------- | ----------- | ----------- | | compression | Y | N/A | Y | | parquet.compression | Y | N/A | Y | | parquet.block.size | Y | spark.gluten.sql.columnar.parquet.write.blockSize | Y | | parquet.block.rows | Y | spark.gluten.sql.native.parquet.write.blockRows | Y | | parquet.gzip.windowSize | Y | N/A | Y | In addition to the above configuration, any other Spark configuration related to table write will not be respected by Gluten at the moment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
