Re: [VOTE] Airflow 1.9.0rc4

I think so. I did some follow up PRs to clean up the logging further but they can arrive in 1.9.1. Cheers Bolke Verstuurd vanaf mijn iPad > Op 1 dec. 2017 om 20:09 heeft Chris Riccomini het > volgende geschreven: > > Hey all, > > My current understanding is that

Re: [VOTE] Airflow 1.9.0rc4

Hey all, My current understanding is that we're going to cancel this release in order to fix these two issues: https://issues.apache.org/jira/browse/AIRFLOW-1873 https://issues.apache.org/jira/browse/AIRFLOW-1872 Is this correct? Are these the only two known issues? Cheers, Chris On Thu, Nov

Re: [VOTE] Airflow 1.9.0rc4

And here's a patch for that one: https://github.com/apache/incubator-airflow/pull/2832 Any feedback I'll address in the morning (which is only 8 hours away.) Anyone is welcome to change this branch or if you have an idea for a better fix

Re: [VOTE] Airflow 1.9.0rc4

Thanks for picking that up so speedily Bolke! I've opened a jira ticket for the other problem I found: https://issues.apache.org/jira/browse/AIRFLOW-1873 and I'll come up with a quick patch for that now. (Quick in the sense that a fix I've

Re: [VOTE] Airflow 1.9.0rc4

Patch here: https://github.com/apache/incubator-airflow/pull/2831 Bolke > On 30 Nov 2017, at 20:53, Bolke de Bruin wrote: > > Ill have a patch in an hour or so > > >> On 30 Nov 2017, at 19:28, Chris Riccomini wrote: >> >> K, I'm going to let this

Re: [VOTE] Airflow 1.9.0rc4

Ill have a patch in an hour or so > On 30 Nov 2017, at 19:28, Chris Riccomini wrote: > > K, I'm going to let this vote hang here until we get confirmation as to > what's going on. > > @bolke/@ash, thanks for looking into this! > > On Thu, Nov 30, 2017 at 10:23 AM,

Re: [VOTE] Airflow 1.9.0rc4

K, I'm going to let this vote hang here until we get confirmation as to what's going on. @bolke/@ash, thanks for looking into this! On Thu, Nov 30, 2017 at 10:23 AM, Bolke de Bruin wrote: > Ok, I think I can confirm the issue. I don’t see any output either with a > clean

Re: [VOTE] Airflow 1.9.0rc4

Ok, I think I can confirm the issue. I don’t see any output either with a clean install. I’ll investigate a bit further. *sigh* Bolke > On 30 Nov 2017, at 19:13, Ash Berlin-Taylor > wrote: > > Yes, that is in my logging config. > > I think I agree about

Re: [VOTE] Airflow 1.9.0rc4

Yes, that is in my logging config. I think I agree about airflow.task.raw -ash > On 30 Nov 2017, at 18:06, Joy Gao wrote: > > Hi Ash, > > Can you verify that your logging config has root logger set to 'console'? > Something like: > >'': { >'handlers':

Re: [VOTE] Airflow 1.9.0rc4

@bolke/@fokko, thoughts? As an aside, we had an issue seeing task logs, but were able to fix it by modifying our logging config (and perhaps patching some stuff that we merged into 1.9.0, don't recall). William Pursell/Joy Gao might be able to comment more. On Thu, Nov 30, 2017 at 9:08 AM, Ash

Re: [VOTE] Airflow 1.9.0rc4

If anyone else is seeing this (and I think I'm on stock configuration now) yes I do, as I'm not seeing _any_ logs form task operators appearing. There's another problem after applying a local fix for `handler.set_context()`, to do with try_numbers: The `airflow run tests test-logging

Re: [VOTE] Airflow 1.9.0rc4

@Ash, do you think this issue is blocking for 1.9.0? On Thu, Nov 30, 2017 at 8:22 AM, Ash Berlin-Taylor < ash_airflowl...@firemirror.com> wrote: > Ah, I think I've got it. > > The problem is that airflow.task.raw doesn't have an explicit config > anymore(?), so when we do: > > log =

Re: [VOTE] Airflow 1.9.0rc4

Ah, I think I've got it. The problem is that airflow.task.raw doesn't have an explicit config anymore(?), so when we do: log = logging.getLogger('airflow.task.raw') and look at its handlers, it doesn't have any. But log.parent.handlers _does_ have them. A few options to fix this: - I can

Re: [VOTE] Airflow 1.9.0rc4

I'm fairly sure I'm back to stock config now. Here is the DAG I am testing with: from datetime import datetime from airflow import DAG from airflow.operators.python_operator import PythonOperator dag_args = { "start_date": datetime(2017, 11, 20), } dag = DAG("tests", default_args=dag_args)

Re: [VOTE] Airflow 1.9.0rc4

airflow.task.raw points to airflow.task by default, which does have a handler and that is the one that is travelled. And yes we know the code is incorrect in this area, but the default does work correctly. Bolke. > On 30 Nov 2017, at 16:27, Ash Berlin-Taylor

Re: [VOTE] Airflow 1.9.0rc4

Hmmm is anyone else seeing logs from inside operators? Right now we're not getting any logs out of operators. I'm in the process of debugging to work out if it's our logging config or something in Airflow, but right now I don't seem to get any output from the operator itself, neither print(),

Re: [VOTE] Airflow 1.9.0rc4

We're running more tests (day of meetings meant we couldn't do as many as I'd have liked yesterday) but looking good so far. +0.5 (non binding) -ash > On 29 Nov 2017, at 18:31, Chris Riccomini wrote: > > Anyone else? Last day before I close the vote off. > > On Mon,

Re: [VOTE] Airflow 1.9.0rc4

+1 Op ma 27 nov. 2017 om 21:24 schreef Bolke de Bruin > +1, binding > > Let’s get it out! > > Sent from my iPhone > > > On 27 Nov 2017, at 20:32, Chris Riccomini wrote: > > > > Hey all, > > > > I have cut Airflow 1.9.0 RC4. This email is calling a vote

Re: [VOTE] Airflow 1.9.0rc4

+1, binding Let’s get it out! Sent from my iPhone > On 27 Nov 2017, at 20:32, Chris Riccomini wrote: > > Hey all, > > I have cut Airflow 1.9.0 RC4. This email is calling a vote on the release, > which will last for 72 hours. Consider this my (binding) +1. > > Airflow