potiuk commented on pull request #22162:
URL: https://github.com/apache/airflow/pull/22162#issuecomment-1064577920
> Hmmm _something_ is still keeping it at pandas v1.3.5. I guess it's still
using the constraints from main for this build?
My bext guess is some of the dependencies in google provider (apache-beam) -
but this is coming more from intuitiona and guess. Unfortunately (and
@uranusjr might have some other tricks which I am not aware of) it might be not
easy guess if this is done through an extra (or even transitive extra as it
happens) because `pip` only uses extra's limitations during installation, and
any metadata connected to extra limitations is not available for inspection
after installation is complete.
The way to check it (at least what I use):
1) Check with `pipdeptree` - if this is not an extra-originated limit - you
will see the limit there
```
docker run -it ghcr.io/apache/airflow/main/ci/python3.7 \
ghcr.io/apache/airflow/main/ci/python3.7:2276fd417180d89da60d661023bf9b48d6b71c7f
bash
```
Then run:
```
pipdeptree | ls
```
Then look for pandas.
2) You can also take look at `pip logs` (starting here :
https://github.com/apache/airflow/runs/5501411485?check_suite_focus=true#step:10:2520)
and try to guess from the logs where the limitation came from. But so far I
think it did not have an information that could allow to determine it easliy.
The change is coming in `pip` to make the log more useful to track the source
of conflicts (https://github.com/pypa/pip/pull/10937), and I hope it's going to
help a lot, but for now I could not really figure out much from the log.
3) Guess (intelligently). There are a few big packages with lots of
dependencies. Apache Beam is one but few other google and amazon depencies that
are the usual culprits, so the problem space is usually much smaller than 580
dependencies we have.
What's helpful here is https://pypi.org/pypi/PACKAGE/json and specifically
https://pypi.org/pypi/PACKAGE/VERSION/json that can help as it contains the
meta-data. But when it comes to extras and transitive extras it's more of a
manual effort than automation, I think it's rather dificult to replicate what
`pip` does fully (because of the complexity of qualifiers for example) but it
is useful for manual verification as you do not have to download and unpack
pacjage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
