[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1164433932 Removed the "old" metadata methods in favor of the new one. I'll merge this for now but we can keep iterating. Thanks Kirill! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1161903416 Implemented for SQLite, though the constraint metadata ignores the column name filter for simplicity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160634749 Ah, something more like this then? ``` /// CONSTRAINT_SCHEMA is a Struct with fields: /// /// Field Name | Field Type| Comments /// -|---|- /// constraint_name | utf8 | /// constraint_type | utf8 not null | (1) /// column_names | list not null | (2) /// column_usage | list| (3) /// /// 1. One of 'CHECK', 'FOREIGN KEY', 'PRIMARY KEY', or 'UNIQUE'. /// 2. The columns on the current table that are constrained, in ///order. /// 3. For FOREIGN KEY only, the referenced table and columns. /// /// USAGE_SCHEMA is a Struct with fields: /// /// Field Name | Field Type| Comments /// -|---|- /// fk_catalog | utf8 | /// fk_db_schema | utf8 | /// fk_table | utf8 | /// fk_column_name | utf8 | ``` This would be easier to extend with additional columns if we need more metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1160481520 Is something like this what you were thinking? (This has less information than what Flight SQL provides but feels minimal, unless we also want to reflect the cascade/delete rules) ``` /// CONSTRAINT_SCHEMA is a Struct with fields: /// /// Field Name | Field Type| Comments /// -|---|- /// constraint_name | utf8 | /// constraint_type | utf8 not null | (1) /// column_names | list not null | (2) /// fk_catalog | utf8 | (3) /// fk_db_schema | utf8 | (3) /// fk_table | utf8 | (3) /// fk_column_names | list| (3) /// /// 1. One of 'CHECK', 'FOREIGN KEY', 'PRIMARY KEY', or 'UNIQUE'. /// 2. The columns on the current table that are constrained, in ///order. /// 3. For FOREIGN KEY only, the referenced table and columns. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159733606 Since we have all the other metadata here, I think it's reasonable to add that information as well. So the 'tables' would now be something like: ``` table_name: utf8 table_type: utf8 table_columns: list> table_constraints: list> ``` Flight SQL also has some calls that get "which foreign keys on other tables reference this table" which maybe we should include too. (Or maybe if those are annoying to model, we can keep those separate, and optional to implement.) Finally, I was thinking we could keep `GetTableTypes` and `GetTableSchema` since they're kind of separate from the metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1159135921 Implemented for SQLite. It's a little tedious to implement for SQLite but isn't especially complicated, at least. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158040008 Hmm, the filters (at least currently, and in Flight SQL) can be patterns as well as fixed strings. If filtering by catalog name gives you the schemas directly, that starts to sound like having different entrypoints to me, just exposed a little differently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1158027202 Hmm, that sounds reasonable. So the schema would be something like this? ``` catalog_name: utf8 catalog_schemas: list>> ``` And if you filtered by (say) depth=catalog, you'd instead get ``` catalog_name: utf8 catalog_schemas: list> ``` I suppose this shouldn't be too painful to assemble/work with, either (possibly we can add a second entrypoint for a flattened version if needed). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [arrow-adbc] lidavidm commented on pull request #18: Add table/column reflection
lidavidm commented on PR #18: URL: https://github.com/apache/arrow-adbc/pull/18#issuecomment-1156813679 CC @hannes, @krlmlr. This is roughly patterned off of Flight SQL (and seems similarish to DBI from quick look as well) I think we discussed an MSSQL style `information_schema` or other special table for this metadata instead at some point, but as with Flight SQL, that makes the implementor's job much harder unless they already have this feature, and it means the client doesn't know the syntax they need to use to query/filter said special table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
