Hi Craig, This is something that needs to be fixed. I agree with you this is very dirty. In your installation you're not installing the kubernetes stuff, so the KubernetesPodOperator is ignored. We need to figure out how to have example dags that are not compatible with the vanilla installation, or we need to remove the kubernetes example for now, and move it to the documentation.
Cheers, Fokko 2018-05-30 2:11 GMT+02:00 Craig Rodrigues <crodr...@gmail.com>: > I tested master branch by putting the following in my requirements.txt: > > git+https://github.com/rodrigc/incubator-airflow@ > master#egg=apache-airflow[celery,crypto,emr,hive,hdfs, > ldap,mysql,postgres,redis,slack,s3] > > and did a pip install -r requirements.txt > > When I started the airflow webserver, I saw deprecation warnings. I > put some additional debugging in models.py to through an exception so that > I could see the > full stacktrace: > > [2018-05-29 14:00:34,419] {models.py:307} ERROR - Failed to import: > /Users/c-craigr/airflow2/lib/python2.7/site-packages/ > airflow/example_dags/example_kubernetes_operator.py > Traceback (most recent call last): > File > "/Users/c-craigr/airflow2/lib/python2.7/site-packages/airflow/models.py", > line 304, in process_file > m = imp.load_source(mod_name, filepath) > File "/Users/c-craigr/airflow2/lib/python2.7/site-packages/ > airflow/example_dags/example_kubernetes_operator.py", line 53, in <module> > dag=dag) > File > "/Users/c-craigr/airflow2/lib/python2.7/site-packages/airflow/utils/decorators.py", > line 98, in wrapper > result = func(*args, **kwargs) > File > "/Users/c-craigr/airflow2/lib/python2.7/site-packages/airflow/models.py", > line 2308, in __init__ > raise Exception("Invalid use of args or kwargs") > Exception: Invalid use of args or kwargs > > > If looks like example_kubernetes_operator.py, this code is the source of > the exception: > > k = KubernetesPodOperator( > namespace='default', > image="ubuntu:16.04", > cmds=["bash", "-cx"], > arguments=["echo", "10"], > labels={"foo": "bar"}, > name="airflow-test-pod", > in_cluster=False, > task_id="task", > get_logs=True, > dag=dag) > > > Without my extra debugging, the deprecation warning looks like this: > > [2018-05-29 14:06:27,567] {example_kubernetes_operator.py:30} WARNING - > Could not import KubernetesPodOperator > /Users/c-craigr/airflow2/lib/python2.7/site-packages/airflow/models.py:2315: > PendingDeprecationWarning: Invalid arguments were passed to BaseOperator. > Support for passing such arguments will be dropped in Airflow 2.0. Invalid > arguments were: > *args: () > **kwargs: {'name': 'airflow-test-pod', 'image': 'ubuntu:16.04', 'labels': > {'foo': 'bar'}, 'namespace': 'default', 'cmds': ['bash', '-cx'], > 'arguments': ['echo', '10'], 'in_cluster': False, 'get_logs': True} > category=PendingDeprecationWarning > > > > What is the correct fix for this? It looks like a lot of operators pass > in arguments which are not > processed by BaseOperator, and thus trip over this deprecation warning. > > -- > Craig >
