pritampan opened a new issue, #18054:
URL: https://github.com/apache/iceberg/issues/18054
### Apache Iceberg version
main (development)
Also reproduced on 1.5.2 / 1.11.0-line code: `Parquet.WriteBuilder.Context`
still uses `PropertyUtil.propertyAsInt`.
### Query engine
Spark
### Please describe the bug 🐞
Setting `write.parquet.row-group-size-bytes` (or
`write.delete.parquet.row-group-size-bytes`) to a value larger than
`Integer.MAX_VALUE` (~2 GiB) fails when the Parquet writer is built, before any
data is written.
```sql
ALTER TABLE t SET TBLPROPERTIES (
'write.parquet.row-group-size-bytes' = '4294967296'
);
INSERT INTO t VALUES (1);
```
**Expected:** the writer accepts the property. Iceberg's `ParquetWriter` and
parquet-mr `withRowGroupSize(long)` already take `long`.
**Actual:**
```text
java.lang.NumberFormatException: For input string: "4294967296"
at java.lang.Integer.parseInt(...)
at org.apache.iceberg.util.PropertyUtil.propertyAsInt(...)
at org.apache.iceberg.parquet.Parquet$WriteBuilder$Context.dataContext(...)
```
Root cause: `Parquet.WriteBuilder.Context` stores row-group size as `int`
and reads it with `propertyAsInt`. A 4 GiB target is a valid flush threshold
for large files; the property cannot currently express it.
Default `PARQUET_ROW_GROUP_SIZE_BYTES_DEFAULT` (128 MiB) can stay `int`. No
public API change is required.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
