tisonkun opened a new pull request, #4733: URL: https://github.com/apache/iggy/pull/4733
## Which issue does this PR address? Closes #4732. ## Rationale Cargo feature unification can enable `serde_json/arbitrary_precision` through any dependency. Delta writes then reject ordinary numeric columns, while PostgreSQL source payloads encode numbers as objects containing `$serde_json::private::Number`. ## What changed? Delta decodes JSON bytes into Arrow batches and writes them with `RecordBatchWriter`, keeping serde_json's private number representation out of Arrow's generic Serde serializer. PostgreSQL source payloads use serde_json's native serializer. A Delta regression test explicitly enables `arbitrary_precision` and checks integer, floating-point, and nullable columns across multiple rows. ## Local Execution - `cargo test -p iggy_connector_delta_sink -p iggy_connector_postgres_source`: 135 tests passed. - Rebuilt the server, connector runtime, Delta sink, and PostgreSQL source with `--features serde_json/arbitrary_precision`. - `IGGY_TEST_CLUSTER_NODES=1 cargo nextest run -p integration --features serde_json/arbitrary_precision -E 'test(connectors::delta::) | test(connectors::postgres::postgres_source::)' --no-fail-fast`: 28 tests passed, including S3 writes, bulk writes, JSON/JSONB payloads, and restart recovery. - `prek run` and pre-push hooks, including `cargo clippy --all-features --all-targets -- -D warnings`, passed. ## AI Usage 1. OpenAI Codex. 2. Implementation, regression test, and validation. 3. The numeric regression test was observed failing with the previous Arrow Serde conversion; unit tests and real-backend integration tests verify the fix. 4. The change is confined to two serialization boundaries: Delta decodes JSON using the table's Arrow schema, and PostgreSQL emits serde_json values using serde_json's serializer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
