This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/orc.git
The following commit(s) were added to refs/heads/main by this push:
new 3f2a0b388 ORC-1593: Set `orc.compression.zstd.level` to 3 by default
3f2a0b388 is described below
commit 3f2a0b388c6fd603f64233421f8dfd497a7a4153
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jan 17 09:00:33 2024 -0800
ORC-1593: Set `orc.compression.zstd.level` to 3 by default
### What changes were proposed in this pull request?
This PR aims to set `orc.compression.zstd.level` to 3 by default.
### Why are the changes needed?
To prevent a regression from ORC 1.9.x
**ORC 1.9**
```
data/generated//taxi:
total 2196176
drwxr-xr-x 5 dongjoon staff 160B Jan 17 08:02 .
drwxr-xr-x 5 dongjoon staff 160B Jan 17 08:07 ..
-rw-r--r-- 1 dongjoon staff 299M Jan 17 08:03 orc.zstd
```
**ORC 2.0**
```
-rw-r--r-- 1 dongjoon staff 334M Jan 17 07:56 orc.zstd (level 1)
-rw-r--r-- 1 dongjoon staff 299M Jan 17 08:16 orc.zstd (level 3)
-rw-r--r-- 1 dongjoon staff 302M Jan 17 08:21 orc.zstd (level 4)
-rw-r--r-- 1 dongjoon staff 300M Jan 17 08:27 orc.zstd (level 5)
```
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #1760 from dongjoon-hyun/ORC-1593.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
java/core/src/java/org/apache/orc/OrcConf.java | 2 +-
java/core/src/java/org/apache/orc/impl/ZstdCodec.java | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/java/core/src/java/org/apache/orc/OrcConf.java
b/java/core/src/java/org/apache/orc/OrcConf.java
index d4bebe2cd..fa7bc9bf3 100644
--- a/java/core/src/java/org/apache/orc/OrcConf.java
+++ b/java/core/src/java/org/apache/orc/OrcConf.java
@@ -73,7 +73,7 @@ public enum OrcConf {
"This changes the compression level of higher level compression\n" +
"codec (like ZLIB)."),
COMPRESSION_ZSTD_LEVEL("orc.compression.zstd.level",
- "hive.exec.orc.compression.zstd.level", 1,
+ "hive.exec.orc.compression.zstd.level", 3,
"Define the compression level to use with ZStandard codec "
+ "while writing data. The valid range is 1~22"),
COMPRESSION_ZSTD_WINDOWLOG("orc.compression.zstd.windowlog",
diff --git a/java/core/src/java/org/apache/orc/impl/ZstdCodec.java
b/java/core/src/java/org/apache/orc/impl/ZstdCodec.java
index cdbf1f3fd..8b5cd24e1 100644
--- a/java/core/src/java/org/apache/orc/impl/ZstdCodec.java
+++ b/java/core/src/java/org/apache/orc/impl/ZstdCodec.java
@@ -34,7 +34,7 @@ public class ZstdCodec implements CompressionCodec {
}
public ZstdCodec() {
- this(1, 0);
+ this(3, 0);
}
public ZstdOptions getZstdOptions() {
@@ -148,7 +148,7 @@ public class ZstdCodec implements CompressionCodec {
}
private static final ZstdOptions DEFAULT_OPTIONS =
- new ZstdOptions(1, 0);
+ new ZstdOptions(3, 0);
@Override
public Options getDefaultOptions() {