potiuk commented on a change in pull request #7128: [AIRFLOW-6529] Pickle error
occurs when the scheduler tries to run on macOS.
URL: https://github.com/apache/airflow/pull/7128#discussion_r366435964
##########
File path: .travis.yml
##########
@@ -87,6 +91,18 @@ jobs:
BACKEND=sqlite
PYTHON_VERSION=3.6
stage: test
+ - name: "Tests [Sqlite][3.7][multiprocessing(spawn)]"
+ env: >-
+ AIRFLOW__CORE__MP_START_METHOD=spawn
Review comment:
It will not work the way you think. Those environment variables are set in
the Host not inside docker - you would have to also change
docker-compose/base.yaml in order to propagate those variables to docker.
However I think you should not do it this way. It should be enough to run ONE
dag test with spawn method, not necessarily all of them. Running all the tests
will take a lot of extra time needed on CI.
What I think should work is simply one "dag" test where os.environment is
overridden for the time of execution. Internally those tests run separate
process with "airflow run" so I think the env variable should be propagated to
the new process. And one test is enough.
Do you need some more pointers/help with that?
I am happy to help more if needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
