vaefremov95 commented on issue #1755:
URL: https://github.com/apache/arrow-adbc/issues/1755#issuecomment-4837266338
Hi @lidavidm !
As far as I understand, this problem happens on databases with a huge number
of tables, for example Greenplum. As a quick fix, what if we reduce the number
of objects in the queries by dropping the unnecessary ones (partitions and
inheritance children, indexes, TOAST tables, sequences, Greenplum system
tables, etc.) and keeping only the needed ones (regular tables, views,
materialized views, composite types, foreign tables)?
New queries will look something like this:
```
pg_attribute:
SELECT a.attrelid, a.attname, a.atttypid
FROM pg_catalog.pg_attribute a
JOIN pg_catalog.pg_class c ON c.oid = a.attrelid
WHERE c.relkind IN ('r','v','m','c','f','p')
AND a.attnum > 0
AND NOT EXISTS (SELECT 1 FROM pg_catalog.pg_inherits i WHERE i.inhrelid =
a.attrelid)
ORDER BY a.attrelid, a.attnum
```
```
pg_type:
SELECT oid, typname, typreceive, typbasetype, typrelid, typarray
FROM pg_catalog.pg_type
WHERE (typreceive != 0 OR typsend != 0)
AND typtype != 'r'
AND typreceive::TEXT != 'array_recv'
AND (typrelid = 0 OR typrelid IN (
SELECT c.oid FROM pg_catalog.pg_class c
WHERE c.relkind IN ('r','v','m','c','f','p')
AND NOT EXISTS (SELECT 1 FROM pg_catalog.pg_inherits i WHERE
i.inhrelid = c.oid)
))
```
Checked locally - connect time dropped from 2.5 minutes to 8 seconds, and
memory usage down to 128 MB.
I tested it on regular queries and everything works as expected. And for
backward compatibility we could put this behind a flag/option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
