roykoand opened a new issue, #73091:
URL: https://github.com/apache/airflow/issues/73091
### Description
`BigQueryHook.create_table()`
(`providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py`)
normalizes every accepted `table_resource` shape (`dict`, `Table`,
`TableReference`,
`TableListItem`) into a plain `dict` via `.to_api_repr()`, runs it through
`_resolve_table_reference()`, and rebuilds a `Table` via
`Table.from_api_repr()` before handing it
to the underlying client's `create_table()`.
That work is only actually necessary for two cases:
- `table_resource` is a plain `dict` — the underlying
`google.cloud.bigquery.Client.create_table()`
does not accept `dict` (its `table` parameter is typed `Union[str, Table,
TableReference,
TableListItem]`), so it has to become a `Table` first. A plain dict is
also the only shape that
can legitimately lack a `tableReference` (e.g. a bare `{"view": {...}}`),
which is what
`_resolve_table_reference()` exists to backfill from the method's own
`project_id`/`dataset_id`/
`table_id` arguments.
- `schema_fields` is provided — the client has no parameter to attach a
schema separately, so the
hook has to merge it into the resource before constructing the `Table`.
When `table_resource` is already a `Table`, `TableReference`, or
`TableListItem` **and**
`schema_fields` is `None`:
- The client's own `create_table()` already normalizes those three types
internally (via
`_table_arg_to_table()`), so they could be passed straight through.
- `_resolve_table_reference()` is a guaranteed no-op for all three, since
each type enforces a
complete `tableReference` at construction time (`Table.__init__` requires
a `TableReference`;
`TableReference.to_api_repr()` always has
`projectId`/`datasetId`/`tableId`; `TableListItem.__init__`
raises `ValueError` if any of those are missing).
### Proposal
Short-circuit `create_table()` to call
`self.get_client(...).create_table(table=table_resource, ...)`
directly when `table_resource` is already an object type and no
`schema_fields` were given, falling
back to the existing dict-normalization path only when needed (plain `dict`
input, or `schema_fields`
present).
### Context
Noticed while fixing `create_table()` crashing when passed a `Table`,
`TableReference`, or
`TableListItem` instead of a `dict` (that crash is already fixed
separately). This is a follow-up
efficiency/clarity improvement only — not a bug, and not required by that
fix.
### Use case/motivation
Avoids redundant `to_api_repr()`/`from_api_repr()` round-tripping and a
no-op `_resolve_table_reference()`
call on the common path where a caller already has a `Table`,
`TableReference`, or `TableListItem`
object in hand (e.g. from `BigQueryHook.get_table()` or `list_tables()`) and
no schema to merge in.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
