Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-31 Thread via GitHub


blackmwk commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3014829155


##
crates/iceberg/src/puffin/writer.rs:
##
@@ -251,7 +251,8 @@ mod tests {
 async fn test_write_zstd_compressed_metric_data() {
 let temp_dir = TempDir::new().unwrap();
 let blobs = vec![blob_0(), blob_1()];
-let blobs_with_compression = blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd);
+let blobs_with_compression =
+blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd(None));

Review Comment:
   I was thinking not changing testing behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-31 Thread via GitHub


blackmwk merged PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-30 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3012497125


##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;

Review Comment:
   moved to a private constant used in default construction.



##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;
 encoder.include_checksum(true)?;
 encoder.set_pledged_src_size(Some(bytes.len().try_into()?))?;
 std::io::copy(&mut &bytes[..], &mut encoder)?;
 Ok(encoder.finish()?)
 }
-CompressionCodec::Gzip => {
-let mut encoder = GzEncoder::new(Vec::new(), 
Compression::default());
+CompressionCodec::Gzip(level) => {
+let compression =
+level.map_or_else(Compression::default, |l| 
Compression::new(l.min(9) as u32));

Review Comment:
   done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-30 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3012424232


##
crates/iceberg/src/puffin/writer.rs:
##
@@ -251,7 +251,8 @@ mod tests {
 async fn test_write_zstd_compressed_metric_data() {
 let temp_dir = TempDir::new().unwrap();
 let blobs = vec![blob_0(), blob_1()];
-let blobs_with_compression = blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd);
+let blobs_with_compression =
+blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd(None));

Review Comment:
   I'm sorry could you elaborate on why you think was incorrect, it appears to 
test only code and the actual compress call would use the default value?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-30 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3012397248


##
crates/iceberg/src/spec/table_metadata.rs:
##
@@ -3618,7 +3618,7 @@ mod tests {
 let original_metadata: TableMetadata = 
get_test_table_metadata("TableMetadataV2Valid.json");
 let json = serde_json::to_string(&original_metadata).unwrap();
 
-let compressed = CompressionCodec::Gzip
+let compressed = CompressionCodec::Gzip(None)

Review Comment:
   9 I think is max decompression level, 6 (is the default 
https://docs.rs/flate2/latest/src/flate2/lib.rs.html#251), by assigning None 
here I think we end up with 6.  I'm in the process of changing the code to have 
a default constructor without the None.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-29 Thread via GitHub


blackmwk commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3007318913


##
crates/iceberg/src/compression.rs:
##
@@ -17,28 +17,74 @@
 
 //! Compression codec support for data compression and decompression.
 
+use std::fmt;
 use std::io::{Read, Write};
 
 use flate2::Compression;
 use flate2::read::GzDecoder;
 use flate2::write::GzEncoder;
-use serde::{Deserialize, Serialize};
+use serde::{Deserialize, Deserializer, Serialize, Serializer};
 
 use crate::{Error, ErrorKind, Result};
 
 /// Data compression formats
-#[derive(Debug, PartialEq, Eq, Clone, Copy, Default, Serialize, Deserialize)]
-#[serde(rename_all = "lowercase")]
+#[derive(Debug, PartialEq, Eq, Clone, Copy, Default)]
 pub enum CompressionCodec {
 #[default]
 /// No compression
 None,
 /// LZ4 single compression frame with content size present
 Lz4,
-/// Zstandard single compression frame with content size present
-Zstd,
-/// Gzip compression
-Gzip,
+/// Zstandard single compression frame with content size present. Optional 
level 0–22,
+/// where 0 means default compression level (not no compression, unlike 
Gzip).
+Zstd(Option),

Review Comment:
   If there is no `no compression` option, why this has to be Option?



##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;
 encoder.include_checksum(true)?;
 encoder.set_pledged_src_size(Some(bytes.len().try_into()?))?;
 std::io::copy(&mut &bytes[..], &mut encoder)?;
 Ok(encoder.finish()?)
 }
-CompressionCodec::Gzip => {
-let mut encoder = GzEncoder::new(Vec::new(), 
Compression::default());
+CompressionCodec::Gzip(level) => {
+let compression =
+level.map_or_else(Compression::default, |l| 
Compression::new(l.min(9) as u32));

Review Comment:
   Ditto.



##
crates/iceberg/src/spec/table_metadata.rs:
##
@@ -3618,7 +3618,7 @@ mod tests {
 let original_metadata: TableMetadata = 
get_test_table_metadata("TableMetadataV2Valid.json");
 let json = serde_json::to_string(&original_metadata).unwrap();
 
-let compressed = CompressionCodec::Gzip
+let compressed = CompressionCodec::Gzip(None)

Review Comment:
   I think this is incorrect? It should be 9?



##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;

Review Comment:
   Not related to this pr, but I don't think we should use a magic number here. 
We should create a constant.



##
crates/iceberg/src/puffin/writer.rs:
##
@@ -251,7 +251,8 @@ mod tests {
 async fn test_write_zstd_compressed_metric_data() {
 let temp_dir = TempDir::new().unwrap();
 let blobs = vec![blob_0(), blob_1()];
-let blobs_with_compression = blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd);
+let blobs_with_compression =
+blobs_with_compression(blobs.clone(), 
CompressionCodec::Zstd(None));

Review Comment:
   This is incorrect, similar to the one with gzip.



##
crates/iceberg/src/spec/table_metadata.rs:
##
@@ -3618,7 +3618,7 @@ mod tests {
 let original_metadata: TableMetadata = 
get_test_table_metadata("TableMetadataV2Valid.json");
 let json = serde_json::to_string(&original_metadata).unwrap();
 
-let compressed = CompressionCodec::Gzip
+let compressed = CompressionCodec::Gzip(None)

Review Comment:
   If 9 is default compression level, you could create a method 
`CompressionCodec::gzip_default` for it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubs

Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003655446


##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;

Review Comment:
   actually, thats not clear and this keeps existing behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003512357


##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +116,25 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;

Review Comment:
   we can just use 0 here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003510433


##
crates/iceberg/src/spec/table_properties.rs:
##
@@ -85,13 +85,15 @@ pub(crate) fn parse_metadata_file_compression(
 
 // Validate that only None and Gzip are used for metadata
 match codec {
-CompressionCodec::None | CompressionCodec::Gzip => Ok(codec),
-CompressionCodec::Lz4 | CompressionCodec::Zstd => Err(Error::new(
-ErrorKind::DataInvalid,
-format!(
-"Invalid metadata compression codec: {value}. Only 'none' and 
'gzip' are supported for metadata files."
-),
-)),
+CompressionCodec::None | CompressionCodec::Gzip(_) => Ok(codec),
+CompressionCodec::Lz4 | CompressionCodec::Zstd(_) | 
CompressionCodec::Snappy => {

Review Comment:
   done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003509871


##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +115,24 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;
 encoder.include_checksum(true)?;
 encoder.set_pledged_src_size(Some(bytes.len().try_into()?))?;
 std::io::copy(&mut &bytes[..], &mut encoder)?;
 Ok(encoder.finish()?)
 }
-CompressionCodec::Gzip => {
-let mut encoder = GzEncoder::new(Vec::new(), 
Compression::default());
+CompressionCodec::Gzip(level) => {
+let compression = Compression::new(level.unwrap_or(6).min(9) 
as u32);

Review Comment:
   done.



##
crates/iceberg/src/compression.rs:
##
@@ -17,28 +17,73 @@
 
 //! Compression codec support for data compression and decompression.
 
+use std::fmt;
 use std::io::{Read, Write};
 
 use flate2::Compression;
 use flate2::read::GzDecoder;
 use flate2::write::GzEncoder;
-use serde::{Deserialize, Serialize};
+use serde::{Deserialize, Deserializer, Serialize, Serializer};
 
 use crate::{Error, ErrorKind, Result};
 
 /// Data compression formats
-#[derive(Debug, PartialEq, Eq, Clone, Copy, Default, Serialize, Deserialize)]
-#[serde(rename_all = "lowercase")]
+#[derive(Debug, PartialEq, Eq, Clone, Copy, Default)]
 pub enum CompressionCodec {
 #[default]
 /// No compression
 None,
 /// LZ4 single compression frame with content size present
 Lz4,
-/// Zstandard single compression frame with content size present
-Zstd,
-/// Gzip compression
-Gzip,
+/// Zstandard single compression frame with content size present. Optional 
level 0–22.

Review Comment:
   done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003403942


##
crates/iceberg/src/compression.rs:
##
@@ -49,13 +94,17 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 decompression is not supported currently",
 )),
-CompressionCodec::Zstd => 
Ok(zstd::stream::decode_all(&bytes[..])?),
-CompressionCodec::Gzip => {
+CompressionCodec::Zstd(_) => 
Ok(zstd::stream::decode_all(&bytes[..])?),
+CompressionCodec::Gzip(_) => {
 let mut decoder = GzDecoder::new(&bytes[..]);
 let mut decompressed = Vec::new();
 decoder.read_to_end(&mut decompressed)?;
 Ok(decompressed)
 }
+CompressionCodec::Snappy => Err(Error::new(
+ErrorKind::FeatureUnsupported,
+"Snappy decompression is not supported currently",
+)),

Review Comment:
   As far as I know the only place Snappy is used is within Avro, and Avro has 
its own code for doing the compression.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


CTTY commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003484196


##
crates/iceberg/src/compression.rs:
##
@@ -49,13 +94,17 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 decompression is not supported currently",
 )),
-CompressionCodec::Zstd => 
Ok(zstd::stream::decode_all(&bytes[..])?),
-CompressionCodec::Gzip => {
+CompressionCodec::Zstd(_) => 
Ok(zstd::stream::decode_all(&bytes[..])?),
+CompressionCodec::Gzip(_) => {
 let mut decoder = GzDecoder::new(&bytes[..]);
 let mut decompressed = Vec::new();
 decoder.read_to_end(&mut decompressed)?;
 Ok(decompressed)
 }
+CompressionCodec::Snappy => Err(Error::new(
+ErrorKind::FeatureUnsupported,
+"Snappy decompression is not supported currently",
+)),

Review Comment:
   This makes sense



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


emkornfield commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003403942


##
crates/iceberg/src/compression.rs:
##
@@ -49,13 +94,17 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 decompression is not supported currently",
 )),
-CompressionCodec::Zstd => 
Ok(zstd::stream::decode_all(&bytes[..])?),
-CompressionCodec::Gzip => {
+CompressionCodec::Zstd(_) => 
Ok(zstd::stream::decode_all(&bytes[..])?),
+CompressionCodec::Gzip(_) => {
 let mut decoder = GzDecoder::new(&bytes[..]);
 let mut decompressed = Vec::new();
 decoder.read_to_end(&mut decompressed)?;
 Ok(decompressed)
 }
+CompressionCodec::Snappy => Err(Error::new(
+ErrorKind::FeatureUnsupported,
+"Snappy decompression is not supported currently",
+)),

Review Comment:
   As far as I now the only place Snappy is used is within Avro, and Avro has 
its own code for doing the compression.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-27 Thread via GitHub


CTTY commented on code in PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#discussion_r3003339106


##
crates/iceberg/src/compression.rs:
##
@@ -17,28 +17,73 @@
 
 //! Compression codec support for data compression and decompression.
 
+use std::fmt;
 use std::io::{Read, Write};
 
 use flate2::Compression;
 use flate2::read::GzDecoder;
 use flate2::write::GzEncoder;
-use serde::{Deserialize, Serialize};
+use serde::{Deserialize, Deserializer, Serialize, Serializer};
 
 use crate::{Error, ErrorKind, Result};
 
 /// Data compression formats
-#[derive(Debug, PartialEq, Eq, Clone, Copy, Default, Serialize, Deserialize)]
-#[serde(rename_all = "lowercase")]
+#[derive(Debug, PartialEq, Eq, Clone, Copy, Default)]
 pub enum CompressionCodec {
 #[default]
 /// No compression
 None,
 /// LZ4 single compression frame with content size present
 Lz4,
-/// Zstandard single compression frame with content size present
-Zstd,
-/// Gzip compression
-Gzip,
+/// Zstandard single compression frame with content size present. Optional 
level 0–22.

Review Comment:
   nit: We should mention that level 0 for zstd means default compression level 
but not no compression like Gzip



##
crates/iceberg/src/spec/table_properties.rs:
##
@@ -85,13 +85,15 @@ pub(crate) fn parse_metadata_file_compression(
 
 // Validate that only None and Gzip are used for metadata
 match codec {
-CompressionCodec::None | CompressionCodec::Gzip => Ok(codec),
-CompressionCodec::Lz4 | CompressionCodec::Zstd => Err(Error::new(
-ErrorKind::DataInvalid,
-format!(
-"Invalid metadata compression codec: {value}. Only 'none' and 
'gzip' are supported for metadata files."
-),
-)),
+CompressionCodec::None | CompressionCodec::Gzip(_) => Ok(codec),
+CompressionCodec::Lz4 | CompressionCodec::Zstd(_) | 
CompressionCodec::Snappy => {

Review Comment:
   nit: since we only support `None` and `Gzip`, this can be `_ =>` to fail all 
other cases



##
crates/iceberg/src/compression.rs:
##
@@ -66,19 +115,24 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 compression is not supported currently",
 )),
-CompressionCodec::Zstd => {
+CompressionCodec::Zstd(level) => {
 let writer = Vecnew();
-let mut encoder = zstd::stream::Encoder::new(writer, 3)?;
+let mut encoder = zstd::stream::Encoder::new(writer, 
level.unwrap_or(3) as i32)?;
 encoder.include_checksum(true)?;
 encoder.set_pledged_src_size(Some(bytes.len().try_into()?))?;
 std::io::copy(&mut &bytes[..], &mut encoder)?;
 Ok(encoder.finish()?)
 }
-CompressionCodec::Gzip => {
-let mut encoder = GzEncoder::new(Vec::new(), 
Compression::default());
+CompressionCodec::Gzip(level) => {
+let compression = Compression::new(level.unwrap_or(6).min(9) 
as u32);

Review Comment:
   We can just call `Compression::default()` when `level` is `None`



##
crates/iceberg/src/compression.rs:
##
@@ -49,13 +94,17 @@ impl CompressionCodec {
 ErrorKind::FeatureUnsupported,
 "LZ4 decompression is not supported currently",
 )),
-CompressionCodec::Zstd => 
Ok(zstd::stream::decode_all(&bytes[..])?),
-CompressionCodec::Gzip => {
+CompressionCodec::Zstd(_) => 
Ok(zstd::stream::decode_all(&bytes[..])?),
+CompressionCodec::Gzip(_) => {
 let mut decoder = GzDecoder::new(&bytes[..]);
 let mut decompressed = Vec::new();
 decoder.read_to_end(&mut decompressed)?;
 Ok(decompressed)
 }
+CompressionCodec::Snappy => Err(Error::new(
+ErrorKind::FeatureUnsupported,
+"Snappy decompression is not supported currently",
+)),

Review Comment:
   Do we plan to address this in a follow up PR? If so, could you create a 
tracking issue?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]



Re: [PR] feat!: Enhance compression codec enum. [iceberg-rust]

2026-03-26 Thread via GitHub


emkornfield commented on PR #2288:
URL: https://github.com/apache/iceberg-rust/pull/2288#issuecomment-4136567155

   @liurenjie1024 @blackmwk would appreciate your thoughts on this approach.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]