Well thank you all for the feedback so quickly! Really appreciate it. We will get started in the coming week internally and hope within two weeks to have a pr for you all to review! I.e. our fabulous engineer Euccas Chen will be :)
Thanks for the notes everyone - Ace > On Oct 16, 2020, at 10:30 AM, Vikram Koka <[email protected]> wrote: > > +1 for feature. Strong preference for storing it in the log table, rather > than in the task instance. > > > > On Fri, Oct 16, 2020 at 10:22 AM Tomasz Urbaszek <[email protected] > <mailto:[email protected]>> wrote: > +1, I know a customer who was requesting it too > > On Fri, Oct 16, 2020 at 6:52 PM Kaxil Naik <[email protected] > <mailto:[email protected]>> wrote: > +1 > > On Fri, Oct 16, 2020 at 5:48 PM Jarek Potiuk <[email protected] > <mailto:[email protected]>> wrote: > +1 > > On Fri, Oct 16, 2020 at 6:42 PM Sumit Maheshwari <[email protected] > <mailto:[email protected]>> wrote: > +1 for the feature. Looking at the schema of the log table, I think it's > perfect to store such kind of information. > > On Fri, Oct 16, 2020 at 9:55 PM Daniel Imberman <[email protected] > <mailto:[email protected]>> wrote: > This could be pretty valuable for future audits. I’d personally rather avoid > adding fields to the DB in general. Could we store it wherever we store > normal failure messages? > > via Newton Mail > <https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2> > On Fri, Oct 16, 2020 at 9:21 AM, Ace Haidrey <[email protected]> > wrote: > Hi team, > > My name is Ace, I’m a data engineer at Pinterest, and I wanted to get some > feedback from the community on how they’d propose designing a feature that > has been asked for us multiple times, that we’d like to implement in-house. > > The ask is the following: > - Users want to be able to add an optional reason/description when they click > on the Mark Success/Failed for both task level in graph views or dag level in > tree views. This is to clearly state why the actions were taken on higher > priority flows for when others are stepping in to look in it. > > For us the feature makes sense to have, and I was wondering if this is > something the greater group would like to have upstream as well. And if so > then we wanted to know what was preferred. > > 1. We can add this information in the UI for the confirmation view and an > optional textbook to add details similar to how users can add conf variables > when triggering a dag option is selected in the UI. > > Then after this, storing this information is the question. Do we want to > store this information in the action logging extra blob and then add a view > in the dag view where it has actions just pertaining to this dag and its > tasks. (We plan to add this personally anyways for ourselves as going to the > big list of actions for all dags and searching is not convenient always). > > Or do we want to add a new column in the task Instance model and dag run > model to store the descriptions and retrieve the information from there. > > Tradeoffs here include the addition of a schema change, where the new column > would generally be sparse, and the data is more resultant from an action vs > necessarily being stored as part of the task run/dag run. > > Please let me know on any feedback, concerns, ideas, and more! > > Cheers, > Ace > > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > M: +48 660 796 129 <tel:+48660796129> > <https://www.polidea.com/>
