Re: Airflow Meetup at Stripe in September?

[Jeff Balogh]

Hey! I finally worked out the date with folks over here.

Wednesday, September 21st from 6:30 to 9:00 in San Francisco. Our max
is about 200 people.

We can follow the format of
http://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/230236598/:

6:30-7:00 Meet, greet, and eat.
7:00-8:00 Talks
8:00-8:30 Updates on the project, roadmap & upcoming Airbnb sprints,
and Q/A for the core project team.
8:30-9:00 Community open mic, step up and make announcements if you're
recruiting, looking for help, planning on working on a feature, ...

I'll give a talk, and Ben from apigee can give a talk, so we need one
more volunteer. Is anyone else interested?

@sumit: sorry, we won't be able to support any remote talks this time.

On Wed, Aug 10, 2016 at 11:45 PM, Ben Tallman <b...@apigee.com> wrote:
> Happy to give a talk about Apigee's use case as well...
>
>
> Thanks,
> Ben
>
> *--*
> *ben tallman* | *apigee
>
>
> On Wed, Aug 10, 2016 at 11:20 PM, siddharth anand <san...@apache.org> wrote:
>
>> Jeff,
>> Any updates on this?
>>
>> Sumit,
>> You are also welcome to host a meet-up in your neck of the woods and some
>> of us can present remotely. Presenting this option as not an "either-or",
>> but an "and" - you are welcome to present remotely here and host in your
>> home country.
>>
>> -s
>>
>> On Thu, Aug 4, 2016 at 10:56 PM, Sumit Maheshwari <sumeet.ma...@gmail.com>
>> wrote:
>>
>> > Hi Jeff,
>> >
>> > If possible I would also love to give a small talk remotely.
>> >
>> >
>> > Thanks,
>> > Sumit
>> >
>> > On Fri, Aug 5, 2016 at 10:39 AM, siddharth anand <san...@apache.org>
>> > wrote:
>> >
>> > > You can also ask on this list if there are folks interested in
>> speaking.
>> > > Also, should you folks ever want to speak in other meet-ups too, it's
>> > fully
>> > > welcome.
>> > >
>> > > -s
>> > >
>> > > On Thu, Aug 4, 2016 at 7:39 PM, Siddharth Anand
>> <san...@agari.com.invalid
>> > >
>> > > wrote:
>> > >
>> > > > Jeff,
>> > > > That sounds great.  We encourage any and all in the community to host
>> > > > Airflow meet-ups.
>> > > >
>> > > > We recommend the following meet-up format:
>> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Meetups
>> > > >
>> > > > Though the format is a suggestion, we do prefer you shoot for 1 host
>> > > > speaker and 2 guest speakers.
>> > > >
>> > > > When you are ready (i.e. confirmed with our organization of the date,
>> > > time,
>> > > > venue, and 3 speakers), shoot us a note on this list and one of the
>> > > > committers will set up the Meet-up and tweet/announce the meet-up
>> using
>> > > the
>> > > > Airflow twitter channel. We also suggest, as mentioned on the page,
>> > that
>> > > > you stream and record the video for posterity.
>> > > >
>> > > > -s
>> > > >
>> > > > On Thu, Aug 4, 2016 at 6:40 PM, Jeff Balogh <jbal...@stripe.com>
>> > wrote:
>> > > >
>> > > > > Hey y'all, we're heavy users of Airflow at Stripe and we'd be happy
>> > to
>> > > > > host a meetup in September. Our office is in San Francisco down by
>> > > > > AT park. How do we make that happen?
>> > > > >
>> > > >
>> > >
>> >
>>

Airflow Meetup at Stripe in September?

[Jeff Balogh]

Hey y'all, we're heavy users of Airflow at Stripe and we'd be happy to
host a meetup in September. Our office is in San Francisco down by
AT park. How do we make that happen?

Re: Airflow Meetup tomorrow

[Jeff Balogh]

Unfortunately, we moved to a new office earlier this month and don't
have our VC setup going yet, so we won't be able to stream or record
the event tomorrow.

On Tue, Sep 20, 2016 at 11:21 AM, Sumit Maheshwari
 wrote:
> Big +1 for live streaming..
>
>
>
> On Tue, Sep 20, 2016 at 11:14 PM, siddharth anand  wrote:
>>
>> Jeff,
>> For the meet-up tomorrow, I'm assuming there will be a streaming feed and
>> possibly also a video conferencing component since a lot of the committers
>> and community members are remote (committers : bolke, jeremiah, sumit, li
>> xuanji aka zodiac).
>>
>> Can you update the Meetup with a link for the stream and conferencing
>> software at the earliest? I'd still recommend people show up in person to
>> get the most out of the meet-up, but want to ensure we don't exclude our
>> remote community members.
>>
>> -s
>
>

Re: Scheduler getting stuck - request for details

[Jeff Balogh]

When we bumped to 1.7.1.3 we found that tasks would go into the new
SCHEDULED state and get stuck there. We haven't determined why this
happens.

We put a hacky patch into our scheduler that sets state to None for
any tasks that are SCHEDULED at the beginning of the schedule loop.

Name: airflow
Version: 1.7.1.3
Name: celery
Version: 3.1.23
Name: kombu
Version: 3.0.35

redis_version:2.6.13

On Sun, Sep 4, 2016 at 6:34 AM, Bolke de Bruin  wrote:
> Hi All,
>
> We have had some reports on this list and sometimes on Jira that the 
> scheduler sometimes seems to get stuck. I would like to track down this 
> issue, but until now much of the reporting has been a bit light on the 
> details.
>
> First and foremost I am assuming that getting “stuck” is only happening when 
> using a CeleryExecutor. To further track down the issue I would like to know 
> the following
>
> - Airflow version (pip show airflow)
> - Celery version (pip show celery)
> - Kombu version (pip show kombu)
>
> - Redis version (if applicable)
> - RabbitMQ version (if applicable)
>
> - Sanitized airflow configuration
> - Sanitized broker configuration
>
> If possible supply, preferably debug, logs of broker, scheduler and worker.
>
> Thanks!
> Bolke
>