Bolke, attached is the patch for the cgroups fix. Let me know which branches you would like me to merge it to. If anyone has complaints about the patch let me know (but it does not touch the core of airflow, only the new cgroups task runner).
On Mon, Feb 6, 2017 at 4:24 PM, siddharth anand <[email protected]> wrote: > Actually, I see the error is further down.. > > File > "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", > line > 469, in do_execute > > cursor.execute(statement, parameters) > > sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in > column "dag_id" violates not-null constraint > > DETAIL: Failing row contains (null, running, 1, f). > > [SQL: 'INSERT INTO dag_stats (state, count, dirty) VALUES (%(state)s, > %(count)s, %(dirty)s)'] [parameters: {'count': 1L, 'state': u'running', > 'dirty': False}] > > It looks like an autoincrement is missing for this table. > > > I'm running `SQLAlchemy==1.1.4` - I see our setup.py specifies any version > greater than 0.9.8 > > -s > > > > On Mon, Feb 6, 2017 at 4:11 PM, siddharth anand <[email protected]> wrote: > > > I tried upgrading to 1.8.0rc1 from 1.7.1.3 via pip install > > https://dist.apache.org/repos/dist/dev/incubator/airflow/ > > airflow-1.8.0rc1+apache.incubating.tar.gz and then running airflow > > upgradedb didn't quite work. First, I thought it completed successfully, > > then saw errors some tables were indeed missing. I ran it again and > > encountered the following exception : > > > > DB: postgresql://[email protected]: > 5432/airflow > > > > [2017-02-07 00:03:20,309] {db.py:284} INFO - Creating tables > > > > INFO [alembic.runtime.migration] Context impl PostgresqlImpl. > > > > INFO [alembic.runtime.migration] Will assume transactional DDL. > > > > INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> > > 211e584da130, add TI state index > > > > INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> > > 64de9cddf6c9, add task fails journal table > > > > INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> > > f2ca10b85618, add dag_stats table > > > > INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> > > 4addfa1236f1, Add fractional seconds to mysql tables > > > > INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> > > 8504051e801b, xcom dag task indices > > > > INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> > > 5e7d17757c7a, add pid field to TaskInstance > > > > INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> > > 127d2bf2dfa7, Add dag_id/state index on dag_run table > > > > /usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/crud.py:692: > > SAWarning: Column 'dag_stats.dag_id' is marked as a member of the primary > > key for table 'dag_stats', but has no Python-side or server-side default > > generator indicated, nor does it indicate 'autoincrement=True' or > > 'nullable=True', and no explicit value is passed. Primary key columns > > typically may not store NULL. Note that as of SQLAlchemy 1.1, > > 'autoincrement=True' must be indicated explicitly for composite (e.g. > > multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is > > expected for one of the columns in the primary key. CREATE TABLE > statements > > are impacted by this change as well on most backends. > > >
