GitHub user singhpratech edited a comment on the discussion: Apache Cloudberry
through ODBC into Apache Arrow (adbcBridge 0.1.0) — is multi-argument unnest a
safe bulk-ingest path on Cloudberry?
Thank you, Lirong. Those are your own numbers on your own host, and the gap you
measured is wider than anything I have recorded, which makes the point better
than my post did: the form costs the server almost nothing and saves a great
deal of client work.
On preserving it, there is something concrete Cloudberry could take: the
semantic probe the bridge runs once per connection before it uses the form. It
is one read-only statement, and its answer pins down everything the fast path
depends on — positional pairing of the arrays, a NULL element, an empty
element, a separator and a closing brace inside a quoted element, an escaped
quote and an escaped backslash:
```sql
SELECT count(*) || '|' || coalesce(sum(a)::text, '?') || '|' ||
coalesce(string_agg(coalesce('[' || b || ']', '<null>'), '' ORDER BY a),
'?')
FROM unnest('{1,2,3,4,5}'::bigint[],
'{"a,b}","",NULL,"x\"y","p\\q"}'::text[]) AS t(a, b);
```
PostgreSQL answers `5|15|[a,b}][]<null>[x"y][p\q]`, and so does Cloudberry
2.1.0-incubating. If that one line lived in your own regression suite, the
behaviour would be locked in where it belongs, and any planner or storage
change that altered it would be caught by Cloudberry rather than by a driver
months later. You are welcome to take the statement as is.
Your point about recommending it to driver and tool developers is well made,
and it applies to the bridge first: as Kxrma47 argued earlier in this thread,
deciding by capability is better than deciding by the `version()` banner, so
the fork allow-list that currently keeps Cloudberry off the fast path is being
removed and the probe will decide for every PostgreSQL-wire server, with the
multi-row `INSERT` still there as the fallback. That change is in flight in
adbcBridge (https://github.com/singhpratech/adbcbridge), tracked as issue #83
with Kxrma47's unequal-length array pair added to the probe. Once it lands,
Cloudberry gets the fast path by passing the test rather than by being named in
a list, which is exactly the contract you are describing.
When it does land I will re-measure Cloudberry end to end, both paths, and post
the figures here so there is a reproducible before and after for your users to
point at.
One housekeeping note, since the post at the top of this thread was written at
0.1.0: the current release is 0.1.3, and two changes in between matter on every
PostgreSQL-wire server, Cloudberry included. Columns declared with reduced
precision now read at that precision rather than being widened, and a
`timestamptz` now keeps its instant when the server's session time zone is not
UTC, which psqlodbc otherwise loses by handing the value over as wall-clock
time without its offset. The Cloudberry compatibility entry has been
re-verified on each of those releases and still needs no tolerance flags.
GitHub link:
https://github.com/apache/cloudberry/discussions/1939#discussioncomment-18522885
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
