Re: Simple question about schedule_interval establishing clear interval boundaries.

2017-02-21 Thread Gerard Toonstra
Hi Bolke, Yep, that would work. So weekly and monthly processing can then be executed quite easily. The only issue that remains is then that these are dates, so wouldn't work for a datetime and thus e.g. hourly processing? I base that on my observation that: ds =

Re: Simple question about schedule_interval establishing clear interval boundaries.

2017-02-21 Thread Bolke de Bruin
Hi Gerard, In 1.8 we introduced prev_execution_date and next_execution_date. Is that what you were looking for? https://github.com/apache/incubator-airflow/blob/50702d06187035c99e51ea936c756c00332c4a4a/airflow/models.py#L1489

Simple question about schedule_interval establishing clear interval boundaries.

2017-02-21 Thread Gerard Toonstra
Hey all, I'm writing up a bit more about best practices for airflow and realize that there may be one important macro that's missing, but which sounds really useful. This is a list of the default macro's: https://airflow.incubator.apache.org/code.html#macros The "execution_date" or "ds" is some

Re: Non-deterministic error in example_subdag_operator DAG

2017-02-21 Thread Jeremiah Lowin
Good thought -- but I believe it has existed for a while, popping up infrequently. I never connected that those random errors had the same root until I happened to be looking closely at a few cases this weekend. But that's just a hunch, can't prove it. I'm sort of at a loss as to the cause. On

Re: Non-deterministic error in example_subdag_operator DAG

2017-02-21 Thread Chris Riccomini
OK, just a thought. :) On Tue, Feb 21, 2017 at 11:29 AM, Bolke de Bruin wrote: > Multiple tests use example bash operator, the mark success unit tests I > refactored to use a different dag. > > Nevertheless I had some issues that required me to refactor to use a > different

Re: Non-deterministic error in example_subdag_operator DAG

2017-02-21 Thread Bolke de Bruin
Multiple tests use example bash operator, the mark success unit tests I refactored to use a different dag. Nevertheless I had some issues that required me to refactor to use a different test. However I think what Jeremiah is observing existed before my update. Bolke Sent from my iPhone >

Re: Non-deterministic error in example_subdag_operator DAG

2017-02-21 Thread Chris Riccomini
Wonder if this has to do with the recent changes to subdag mark success for RC4? On Sun, Feb 19, 2017 at 10:05 AM, Jeremiah Lowin wrote: > The unit test that runs the example_subdag_operator DAG is failing > non-deterministically. I see it sporadically in the Travis results,

Re: [RESULT] [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4

2017-02-21 Thread Chris Riccomini
Ack that the vote has already passed, but belated +1 (binding) On Tue, Feb 21, 2017 at 7:42 AM, Bolke de Bruin wrote: > IPMC Voting can be found here: > > http://mail-archives.apache.org/mod_mbox/incubator-general/201702.mbox/% >

Re: [RESULT] [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc4

2017-02-21 Thread Bolke de Bruin
IPMC Voting can be found here: http://mail-archives.apache.org/mod_mbox/incubator-general/201702.mbox/%3c676bdc9f-1b55-4469-92a7-9ff309ad0...@gmail.com%3e Kind regards,