Joris Van den Bossche created ARROW-14967:
---------------------------------------------
Summary: [CI][Python] Ability to include pip packages in the conda
environments
Key: ARROW-14967
URL: https://issues.apache.org/jira/browse/ARROW-14967
Project: Apache Arrow
Issue Type: Improvement
Components: Continuous Integration, Python
Reporter: Joris Van den Bossche
For creating various conda environments, we currently have files like
{{conda_env_cpp.txt}}, {{conda_env_sphinx.txt}}, {{conda_env_python.txt}}, etc
Those can then be combined to create a specific conda environment with the
subset of features you want, eg from the python docs:
{code}
conda create -y -n pyarrow-dev -c conda-forge \
--file arrow/ci/conda_env_unix.txt \
--file arrow/ci/conda_env_cpp.txt \
--file arrow/ci/conda_env_python.txt \
--file arrow/ci/conda_env_gandiva.txt \
compilers \
python=3.9 \
pandas
{code}
or installed as additional packages into an existing one (eg {{conda install
--file arrow/ci/conda_env_python.txt}} in conda-python.dockerfile).
One disadvantage of this approach is that you cannot (as far as I am aware) not
list pip packages in those .txt files.
You can do that with environment.yml files, but those then don't really compose
together as we do with the txt files, I think.
cc [~kszucs]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
