I did get 1.8.0 installed and running at Agari. I did run into 2 problems. 1. Most of our DAGs broke due the way Operators are now imported. https://github.com/apache/incubator-airflow/blob/master/UPDATING.md#deprecated-features
According to the documentation, these deprecations would only cause an issue in 2.0. However, I needed to fix them now. So, I needed to change "from airflow.operators import PythonOperator" to from "from airflow.operators.python_operator import PythonOperator". Am I missing something? 2. I ran into a migration problem that seems to have cleared itself up. I did notice that some dags do not have data in their "DAG Runs" column on the overview page computed. I am looking into that issue presently. https://www.dropbox.com/s/cn058mtu3vcv8sq/Screenshot%202017-02-06%2018.45.07.png?dl=0 -s On Mon, Feb 6, 2017 at 4:30 PM, Dan Davydov <[email protected]> wrote: > Bolke, attached is the patch for the cgroups fix. Let me know which > branches you would like me to merge it to. If anyone has complaints about > the patch let me know (but it does not touch the core of airflow, only the > new cgroups task runner). > > On Mon, Feb 6, 2017 at 4:24 PM, siddharth anand <[email protected]> wrote: > >> Actually, I see the error is further down.. >> >> File >> "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", >> line >> 469, in do_execute >> >> cursor.execute(statement, parameters) >> >> sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in >> column "dag_id" violates not-null constraint >> >> DETAIL: Failing row contains (null, running, 1, f). >> >> [SQL: 'INSERT INTO dag_stats (state, count, dirty) VALUES (%(state)s, >> %(count)s, %(dirty)s)'] [parameters: {'count': 1L, 'state': u'running', >> 'dirty': False}] >> >> It looks like an autoincrement is missing for this table. >> >> >> I'm running `SQLAlchemy==1.1.4` - I see our setup.py specifies any version >> greater than 0.9.8 >> >> -s >> >> >> >> On Mon, Feb 6, 2017 at 4:11 PM, siddharth anand <[email protected]> >> wrote: >> >> > I tried upgrading to 1.8.0rc1 from 1.7.1.3 via pip install >> > https://dist.apache.org/repos/dist/dev/incubator/airflow/ >> > airflow-1.8.0rc1+apache.incubating.tar.gz and then running airflow >> > upgradedb didn't quite work. First, I thought it completed successfully, >> > then saw errors some tables were indeed missing. I ran it again and >> > encountered the following exception : >> > >> > DB: postgresql://[email protected]:543 >> 2/airflow >> > >> > [2017-02-07 00:03:20,309] {db.py:284} INFO - Creating tables >> > >> > INFO [alembic.runtime.migration] Context impl PostgresqlImpl. >> > >> > INFO [alembic.runtime.migration] Will assume transactional DDL. >> > >> > INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> >> > 211e584da130, add TI state index >> > >> > INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> >> > 64de9cddf6c9, add task fails journal table >> > >> > INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> >> > f2ca10b85618, add dag_stats table >> > >> > INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> >> > 4addfa1236f1, Add fractional seconds to mysql tables >> > >> > INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> >> > 8504051e801b, xcom dag task indices >> > >> > INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> >> > 5e7d17757c7a, add pid field to TaskInstance >> > >> > INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> >> > 127d2bf2dfa7, Add dag_id/state index on dag_run table >> > >> > /usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/crud.py:692: >> > SAWarning: Column 'dag_stats.dag_id' is marked as a member of the >> primary >> > key for table 'dag_stats', but has no Python-side or server-side default >> > generator indicated, nor does it indicate 'autoincrement=True' or >> > 'nullable=True', and no explicit value is passed. Primary key columns >> > typically may not store NULL. Note that as of SQLAlchemy 1.1, >> > 'autoincrement=True' must be indicated explicitly for composite (e.g. >> > multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is >> > expected for one of the columns in the primary key. CREATE TABLE >> statements >> > are impacted by this change as well on most backends. >> > >> > >
