BOKJUNSOO opened a new pull request, #4498:
URL: https://github.com/apache/arrow-adbc/pull/4498
## Summary
This adds a fast path for PostgreSQL binary COPY encoding of Arrow
`decimal128` values.
The existing path converts every Arrow decimal value to a decimal string,
splits
that string into integer/fractional parts, then parses those parts back into
PostgreSQL's base-10000 numeric representation. For `decimal128`, we can
avoid
that round trip: read the unscaled 128-bit integer directly, align it to
PostgreSQL's 4-decimal-digit groups, and write the numeric digits.
The fast path is guarded by `__SIZEOF_INT128__`. Platforms without native
`unsigned __int128` support keep using the existing implementation.
`decimal256` is unchanged.
## Local benchmark
Environment:
- macOS arm64
- PostgreSQL 15 container
- Release build
- Python `adbc_driver_postgresql.dbapi.adbc_ingest`
- table schema: `v numeric`
- input: `decimal128(38, 6)`, values `Decimal(i) / Decimal(100)`
Before, PyPI wheel `adbc-driver-postgresql==1.11.0`:
```text
100k rows: ~2.166s, ~46k rows/s
```
Before, local unpatched Release build from the ADBC source tree:
```text
100k rows: ~0.831s, ~120k rows/s
```
After this patch:
```text
100k rows: ~0.058s, ~1.73M rows/s
200k rows: ~0.097s, ~2.05M rows/s
1M rows: ~0.486s, ~2.06M rows/s
```
## Testing
```bash
PKG_CONFIG_PATH=/opt/homebrew/lib/postgresql@14/pkgconfig \
cmake -S c -B build-local \
-DADBC_DRIVER_POSTGRESQL=ON \
-DADBC_BUILD_SHARED=ON \
-DADBC_BUILD_STATIC=OFF \
-DADBC_BUILD_TESTS=ON \
-DADBC_BUILD_BENCHMARKS=OFF \
-DADBC_WITH_VENDORED_FMT=ON \
-DADBC_WITH_VENDORED_NANOARROW=ON \
-DCMAKE_BUILD_TYPE=Release
cmake --build build-local --target adbc-driver-postgresql-copy-test -j 8
ADBC_POSTGRESQL_TEST_URI='postgresql://postgres:...@localhost:9900/postgres?sslmode=disable'
\
build-local/driver/postgresql/adbc-driver-postgresql-copy-test
```
Result:
```text
[==========] 73 tests from 5 test suites ran.
[ PASSED ] 73 tests.
```
I also checked PostgreSQL roundtrips for scale 2/6/8/18, negative values,
small fractional values, and 38-digit `decimal128` values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
