badalprasadsingh opened a new pull request, #1223: URL: https://github.com/apache/iceberg-go/pull/1223
## Problem Iceberg-Go's `iceberg.TruncateTransform.Transformer` for `StringType` slices the input by raw UTF-8 byte length instead of Unicode code points. For any non-ASCII input — CJK, emoji, accented Latin — this produces output that may not be valid UTF-8. Following the [Iceberg spec](https://iceberg.apache.org/spec/#truncate-transform-details) defining string truncate as truncation to *N Unicode characters*. Fixes #1222 ## Solution Iterate the input by Unicode code points using `unicode/utf8`. Keep binary truncation byte-based. ## Files changed | File | Description | |---|---| | `transforms.go` | Bug Fix | | `transforms_test.go` | Regression Test | ## Testing - ingested data into a partitioned iceberg table in Glue Catalog - queried through Trino -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
