potiuk commented on PR #36537:
URL: https://github.com/apache/airflow/pull/36537#issuecomment-1882058753
> Traceback (most recent call last):
File "/home/jscheffl/Workspace/airflow/.test-hatch/bin/airflow", line 8,
in <module>
sys.exit(main())
File "/home/jscheffl/Workspace/airflow/airflow/__main__.py", line 54, in
main
conf = write_default_airflow_configuration_if_needed()
File "/home/jscheffl/Workspace/airflow/airflow/configuration.py", line
1989, in write_default_airflow_configuration_if_needed
conf.write(
File "/home/jscheffl/Workspace/airflow/airflow/configuration.py", line
673, in write
with
self.make_sure_configuration_loaded(with_providers=include_providers):
File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/home/jscheffl/Workspace/airflow/airflow/configuration.py", line
527, in make_sure_configuration_loaded
ProvidersManager()._initialize_providers_configuration()
File "/home/jscheffl/Workspace/airflow/airflow/utils/singleton.py", line
32, in __call__
cls._instances[cls] = super().__call__(*args, **kwargs)
File "/home/jscheffl/Workspace/airflow/airflow/providers_manager.py", line
445, in __init__
self._init_airflow_core_hooks()
File "/home/jscheffl/Workspace/airflow/airflow/providers_manager.py", line
471, in _init_airflow_core_hooks
self._hook_provider_dict[hook_info.connection_type] = HookClassProvider(
AttributeError: 'NoneType' object has no attribute 'connection_type'
Yeah - thos were the last erors that I've been tracking - this is the reason
our K8S and DB tests were failing in CI.
And I just finish fixing it, basically this is about special handling of the
"preinstalled providers". Since we **just** separated out `fab` as a provider,
it's dependencies are not any more in`airflow` which means that when you
install airflow without it (which editable build does), it means that you do
not have it by default. and since that provider was "not ready" (i.e we are not
releasing it) - the dependencies did not come in "regular" installation -
neither by preinstalled provider, nor by direct dependencies coming from
`install-requires`
> developer making a test run when calling just pip install -e .
For **now** I made it so that all the pre-installed provider dependencies
will always get installed when you run:
```
pip install -e ".['devel']"
```
Which is what I'd sonsider as "minimum" development environment. I think we
should make it in the way that `devel` is the default extra when you create
hatch env (and update it in the docs that you need to install airflow with it
to get airflow running this way. And this is pretty much expected - you won't
be able to run any tests without using `devel`. I think just clearly
explaining it should be more than enough.
We **could** potentially add ``devel`` dependencies - or at least some of
them - to "install-requires` in pyproject.toml and then remove them when we are
preparing a real wheel package in the `hatch_build.py` plugin. But that would
be quite ugly and I think I'd prefer to get our contributors to use to `pip
install -e .[devel]` instead - but I am open to discussing it.
> ...uuups, just made a regression on main and even there we have a problem.
w/o fab provider airflow is not starting at all. even worse, here adding fab
via pip install -e .[fab] fails miserable in a cyclic version dependency?!?!
Yes. That's expected (both). This is the main `problem` that this PR is
supposed to solve.
In main you are supposed to install airflow now via:
```bash
INSTALL_PROVIDERS_FROM_SOURCES="true" pip installl -e "[fab]"
```
This was the ugly hack (different behaviour when
INSTALL_PROVIDERS_FROM_SOURCES was used) that I had before to achieve pretty
much the same that current `--editable` flag can do via the build hook. But it
could have been also broken by fab separation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
