:) you rock as always Bolke!! On Wed, Dec 6, 2017 at 11:17 AM, Bolke de Bruin <[email protected]> wrote:
> 1. Naive dates are supported for initalization of DAGs and will be, after > the patch of Sumit, for directly instantiated TaskInstances. XCom can store > arbitrary information so it is not affected. > 2. Any arithmetic you do on dates will need to happen with a timezone > associated with it. This rare, you normally do date + datetime. However if > you want to compare dates you will need to make it timezone aware. > 3. Timezone support will not be in 1.9.0 > 4. Logging.info will be supported in 1.9.0 final > > > So no reason to panic I would say ;-) > > Bolke > > Verstuurd vanaf mijn iPad > > > Op 6 dec. 2017 om 16:35 heeft Boris Tyukin <[email protected]> het > volgende geschreven: > > > > hm...can you elaborate why we cannot use "from datetime import > datetime". > > it worked before just fine :) i was under an impression that any pickle > > serializable type can be using with xcom. > > > > Along with a recent change in logs (logging.info does not write to log > > files anymore), this makes upgrade to 1.9 quite difficult. > > > >> On Wed, Dec 6, 2017 at 9:19 AM, Bolke de Bruin <[email protected]> > wrote: > >> > >> You should not use a naive datetime as it mentioned. So instead of of > >> using “from datetime import datetime” use “from airflow.utils.timezone > >> import datetime”. This sets the timezone information that is required to > >> the default configured in airflow.cfg. > >> > >> A patch that sets the default timezone for TaskInstance is also helpful > in > >> this case and will be welcomed. How to do it you can find in the DAG > class. > >> > >> Bolke. > >> > >>> On 6 Dec 2017, at 15:13, Sumit Maheshwari <[email protected]> > >> wrote: > >>> > >>> Folks, > >>> > >>> I've some testes, which used to pass earlier, but recently they started > >> failing. When I looked closely it appeared that xcom_push is behaving > >> differently. > >>> > >>> > >>> So this is is the current code, which throws the error mentioned below > >> to it > >>> DEFAULT_DATE = datetime(2017, 1, 1) > >>> ti = TaskInstance(task=task, execution_date=DEFAULT_DATE) > >>> ti.xcom_push('abcd', 12345) > >>> > >>> File "build/bdist.macosx-10.11-intel/egg/sqlalchemy_utc.py", line 31, > >> in process_bind_param > >>> raise ValueError('naive datetime is disallowed') > >>> StatementError: (exceptions.ValueError) naive datetime is disallowed > >> [SQL: u'DELETE FROM xcom WHERE xcom."key" = ? AND xcom.execution_date = > ? > >> AND xcom.task_id = ? AND xcom.dag_id = ?'] [parameters: [{}]] > >>> > >>> When I changed the execution date param to a string, it fails with > other > >> error. > >>> ti = TaskInstance(task=task, execution_date=DEFAULT_DATE.isoformat()) > >>> ti.xcom_push('abcd', 12345) > >>> > >>> File "build/bdist.macosx-10.11-intel/egg/sqlalchemy_utc.py", line 29, > >> in process_bind_param > >>> repr(value)) > >>> StatementError: (exceptions.TypeError) expected datetime.datetime, not > >> '2017-01-01T00:00:00' [SQL: u'DELETE FROM xcom WHERE xcom."key" = ? AND > >> xcom.execution_date = ? AND xcom.task_id = ? AND xcom.dag_id = ?'] > >> [parameters: [{}]] > >>> > >>> Somehow I feel that it is linked with sqlalchemy_utc library, though I > >> may be wrong and its just missing some timezone related settings. > >>> Please let me know if anyone else has faced this issue as well. > >>> > >>> Thanks, > >>> Sumit > >>> > >>> > >> > >> >
