zhuqi-lucas opened a new pull request, #23438:
URL: https://github.com/apache/datafusion/pull/23438
# Which issue does this PR close?
Closes #23437.
# Rationale for this change
`information_schema.routines` and `SHOW FUNCTIONS` currently enumerate only
scalar / aggregate / window UDFs. Table functions (UDTFs) registered via
`SessionContext::register_udtf` are omitted, making them undiscoverable through
SQL. Discovery matters because downstream tooling (DataFusion CLI, Massive's
atlas SQL surface, dbt-datafusion, etc.) uses these SQL surfaces to list
available functions.
# What changes are included in this PR?
**1. Snapshot table functions into the information_schema provider.**
`TaskContext` cannot expose table_functions without introducing a crate
cycle (`TableFunction` lives in `datafusion-catalog`, which depends on
`datafusion-execution` where `TaskContext` lives). Instead, snapshot
`SessionState.table_functions` into `InformationSchemaProvider` at construction
time. The provider is built per-query via `schema_for_ref`, so the snapshot
stays fresh.
**2. Emit UDTF rows.**
- `make_routines`: one row per UDTF with `routine_type = \"FUNCTION\"`,
`function_type = \"TABLE\"`, `data_type = \"TABLE\"`.
- `make_parameters`: a single synthetic `OUT` row per UDTF (`data_type =
\"TABLE\"`) so the JOIN in `SHOW FUNCTIONS` resolves. UDTFs have no scalar
signature, so no `IN` rows are emitted.
**3. Rewrite the SHOW FUNCTIONS SQL.**
The old query joined `parameters p (INNER)` requiring both IN and OUT rows
before joining `routines`. UDTFs have no IN parameters, so they fell out. New
shape:
\`\`\`
FROM (parameters WHERE mode='OUT') o
LEFT JOIN (parameters WHERE mode='IN') i
ON matching keys
JOIN information_schema.routines r
ON name matches
\`\`\`
# Are these changes tested?
Verified locally by registering a UDTF and running SHOW FUNCTIONS:
\`\`\`
> SHOW FUNCTIONS LIKE 'stale%'
+---------------+-------------+---------------+
| function_name | return_type | function_type |
| stale_files | TABLE | TABLE |
+---------------+-------------+---------------+
\`\`\`
Happy to add sqllogictests in
`datafusion/sqllogictest/test_files/information_schema.slt` if maintainers
prefer.
# Are there any user-facing changes?
Yes: SHOW FUNCTIONS output now includes table functions.
`information_schema.routines` and `information_schema.parameters` gain rows for
each registered UDTF. Existing rows are unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
