[
https://issues.apache.org/jira/browse/SPARK-36773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-36773:
---------------------------------
Affects Version/s: (was: 3.3)
3.3.0
> Fix the uts to check the parquet compression
> --------------------------------------------
>
> Key: SPARK-36773
> URL: https://issues.apache.org/jira/browse/SPARK-36773
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: jingpan xiong
> Priority: Major
>
> In my own test env. I import a wrong jar version about zstd-jni-1.4.4-3.jar
> with the parquet version 1.12.0, when I pass the full uts and release a spark
> version to my env , using the parquet table with compression zstd, it throw a
> "NoClassDefFoundError" Exception. And I check the parquet source code, find
> the right zstd version for the 1.12.0 is 1.4.9-1. Once I upgrade the version
> about zstd-jni, it work well.
> !https://clouddevops.huawei.com/vision-file-storage/api/file/download/upload-v2/2021/8/14/xwx915262/ed1735e06a684bf19cf1d1a68a130b6a/image.png!
> But I think we need check for all plugin compression algorithm can work well
> when we upgrade the version about those compression jars and parquet version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
