[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-26 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-604448755
 
 
   Merged, thanks for your help @TomAugspurger 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-25 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-603834676
 
 
   Something is wrong with Travis, but the Build is at 
https://travis-ci.org/github/apache/airflow/builds/666783056


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-25 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-603829919
 
 
   Added a commit to not pickle if using DaskExecutor


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-24 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-603610022
 
 
   
https://github.com/apache/airflow/pull/7786/commits/d652062b19f0820a0ba5636ec2c4ff94f8aef9b3
 should fix another 2 tests.
   
   ```
   
tests/executors/test_dask_executor.py::TestDaskExecutor::test_backfill_integration
 FAILED 

 [ 25%]
   
tests/executors/test_dask_executor.py::TestDaskExecutor::test_dask_executor_functions
 PASSED 

  [ 50%]
   
tests/executors/test_dask_executor.py::TestDaskExecutorTLS::test_gauge_executor_metrics
 PASSED 

[ 75%]
   tests/executors/test_dask_executor.py::TestDaskExecutorTLS::test_tls PASSED  


  [100%]
   ```
   
   I got the following error while running the test related to backfill:
   
   **Error Log:**
   ```
   
---
 Captured stderr call 

   distributed.worker - WARNING -  Compute Failed
   Function:  airflow_run
   args:  ()
   kwargs:{}
   Exception: TypeError('expected str, bytes or os.PathLike object, not int',)
   
   distributed.worker - WARNING -  Compute Failed
   Function:  airflow_run
   args:  ()
   kwargs:{}
   Exception: TypeError('expected str, bytes or os.PathLike object, not int',)
   
   distributed.worker - WARNING -  Compute Failed
   Function:  airflow_run
   args:  ()
   kwargs:{}
   Exception: TypeError('expected str, bytes or os.PathLike object, not int',)
   
   distributed.worker - WARNING -  Compute Failed
   Function:  airflow_run
   args:  ()
   kwargs:{}
   Exception: TypeError('expected str, bytes or os.PathLike object, not int',)
   
   
-
 Captured log call 
--
   INFO airflow.models.dagbag.DagBag:dagbag.py:371 Filling up the DagBag 
from /opt/airflow/tests/dags
   INFO root:test_task_view_type_check.py:48 class_instance type: 
   INFO airflow.models.dagbag.DagBag:dagbag.py:269 File 
/opt/airflow/tests/dags/test_zip.zip assumed to contain no DAGs. Skipping.
   INFO airflow.executors.dask_executor.DaskExecutor:base_executor.py:75 
Adding to queue: ['airflow', 'tasks', 'run', 'example_bash_operator', 
'runme_0', '2017-01-01T00:00:00+00:00', '--pickle', 1, 
'--ignore-depends-on-past', '--local', '--pool', 'default_pool']
   INFO airflow.executors.dask_executor.DaskExecutor:base_executor.py:75 
Adding to queue: ['airflow', 'tasks', 'run', 'example_bash_operator', 
'runme_1', '2017-01-01T00:00:00+00:00', '--pickle', 1, 
'--ignore-depends-on-past', '--local', '--pool', 'default_pool']
   INFO airflow.executors.dask_executor.DaskExecutor:base_executor.py:75 
Adding to queue: ['airflow', 'tasks', 'run', 'example_bash_operator', 
'runme_2', '2017-01-01T00:00:00+00:00', '--pickle', 1, 
'--ignore-depends-on-past', '--local', '--pool', 'default_pool']
   INFO airflow.executors.dask_executor.DaskExecutor:base_executor.py:75 
Adding to queue: ['airflow', 'tasks', 'run', 'example_bash_operator', 
'also_run_this', '2017-01-01T00:00:00+00:00', '--pickle', 1, 
'--ignore-depends-on-past', '--local', '--pool', 'default_pool']
   INFO airflow.jobs.backfill_job.BackfillJob:backfill_job.py:379 [backfill 
progress] | finished run 0 of 1 | tasks waiting: 2 | succeeded: 0 | running: 4 
| failed: 0 | skipped: 0 | deadlocked: 0 | not ready: 2
   ERRORairflow.executors.dask_executor.DaskExecutor:dask_executor.py:86 
Failed to execute task: TypeError('expected str, bytes or os.PathLike object, 
not int',)
   ERRORairflow.executors.dask_executor.DaskExecutor:dask_executor.py:86 
Failed to execute task: TypeError('expected str, bytes or os.PathLike object, 
not int',)
   ERRORairflow.executors.dask_executor.DaskExecutor:dask_executor.py:86 
Failed to execute task: TypeError('expected str, bytes or os.PathLike object, 
not int',)
   ERRORairflow.executors.dask_executor.DaskExecutor:dask_executor.py:86 
Failed to execute task: TypeError('expected str, bytes or os.PathLike object,

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-22 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-602224559
 
 
   I think we need to add `dask` in 
https://github.com/apache/airflow/blob/0974aabd31abc6aeadf4f07c9c310b6750e28e2f/setup.py#L459-L463


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-22 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-602223266
 
 
   Also, the following tests failed:
   
   ```
   FAILED 
tests/executors/test_dask_executor.py::TestDaskExecutor::test_backfill_integration
   FAILED 
tests/executors/test_dask_executor.py::TestDaskExecutor::test_dask_executor_functions
   FAILED 
tests/executors/test_dask_executor.py::TestDaskExecutorTLS::test_gauge_executor_metrics
   FAILED tests/executors/test_dask_executor.py::TestDaskExecutorTLS::test_tls
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #7786: Restore DaskExecutor tests

2020-03-20 Thread GitBox 
kaxil commented on issue #7786: Restore DaskExecutor tests
URL: https://github.com/apache/airflow/pull/7786#issuecomment-601912897
 
 
   Awesome, really excited for this, thanks @TomAugspurger 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services