What I can say is that we use it a lot, but very lightly. We basically use it to communicate the S3 key for a flat file between operators.
Definitely don't use it to send actual data :) Cheers On Wed, Aug 3, 2016 at 6:21 PM, Andrew Phillips <[email protected]> wrote: > Let me know if that helps, or if I completely misunderstood :) >> > > That helps, indeed - thanks, Joe! We were in fact going down exactly this > path as an alternative; we were just a bit hesitant to use XComs based on > the following comment in the docs [1]: > > "If it absolutely can’t be avoided, Airflow does have a feature for > operator cross-communication called XCom that is described elsewhere in > this document." > > The statements talks about sharing information *between* tasks, but we > weren't sure if this should be read as "stay away from XComs unless there's > no other option". Curious to hear the community's thoughts on that. > > Thanks for the quick response! > > ap > > [1] https://pythonhosted.org/airflow/concepts.html#operators > -- *Joe Napolitano *| Sr. Data Engineer www.blueapron.com | 5 Crosby Street, New York, NY 10013
