kaxil opened a new pull request, #73053:
URL: https://github.com/apache/airflow/pull/73053
`DocumentLoaderOperator` reported the wrong argument in two of its checks.
A Dag passing `source_path="{{ ... }}"` where the template renders to `None`
was told to
"Provide exactly one of 'source_path' or 'source_bytes'" -- advice it had
already followed.
`__init__` settles provision, so the only way to reach that check is a
*supplied* field that
rendered away. Both messages now name the field that rendered to `None` and
point at the
template or the upstream XCom value behind it. Remove either guard and the
same input
fails instead with `TypeError: argument of type 'NoneType' is not iterable`
out of
`_resolve_files`, or `AttributeError: 'NoneType' object has no attribute
'startswith'` out
of `_parse_bytes`.
`file_type` is required with `source_bytes`, and whether it was supplied is
knowable without
rendering, so that check moves to `__init__` beside the
`source_path`/`source_bytes` pair --
the same Dag-parse-time direction as #70628, applied to the argument it left
behind. A
`file_type` supplied as a template string is non-`None` at construction, so
it still passes
`__init__` and is re-checked after rendering. The only newly-rejected Dag is
`DocumentLoaderOperator(source_bytes=...)` with no `file_type` at all, which
could never have
succeeded at run time either.
The comment above those checks claimed provision "already happened in
`__init__`" for both
fields, which was never true of `file_type` -- `__init__` had no `file_type`
check. #70503
designates this file the reference other operators copy, so the comment now
describes the
split it actually implements.
`test_source_path_none_after_render_raises` set the state with
`op.source_path = None`, which
passes whether or not the field is still rendered at all. It now drives the
same state through
real templating under `render_template_as_native_obj`, and the
template-fields test renders
its fields instead of asserting tuple membership.
Related to #70628 and #70503.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
