Yeah, it is 30s by default; storing it in DB would be useless, apart from the most recent run, in my opinion.
On Fri, 14 Jun 2024 at 15:24, Constance Martineau <[email protected]> wrote: > I love the idea. If we were to store it in the DB, would we keep a history, > or only the latest stats from the most recent dag parsing loop? DAG parsing > by default is every 30s right? > > On Fri, Jun 14, 2024 at 6:53 AM Jarek Potiuk <[email protected]> wrote: > > > > I think we still need to enable the ability for DAGs at parse time to > > access Variables. > > > > It's actually DISCOURAGED ;) to access Variables at parse time - though > we > > have an experimental feature to make it efficient and we discussed > whether > > to treat it as "good practice" and there was strong opposition to that > .... > > So I am not actually sure what will be our position on it. > > > > On Fri, Jun 14, 2024 at 12:50 PM Ash Berlin-Taylor <[email protected]> > wrote: > > > > > > > > > > > > On 14 Jun 2024, at 10:22, Jarek Potiuk <[email protected]> wrote: > > > > > > > > think in the future of Airflow 3 where > > > > we will have task isolation, having `0` for all the DAGs will be a > > > > prerequisite for switching to "task isolation" mode and this could be > > > > actually verified in a migration tool. > > > > > > I think we still need to enable the ability for DAGs at parse time to > > > access Variables. > > > > > > Or at least I am not proposing we remove that ability. (I wouldn’t be > > > against it, but I was planning on continuing to support that for now) > > > > > > -ash > > >
