+1 for the feature. Looking at the schema of the log table, I think it's perfect to store such kind of information.
On Fri, Oct 16, 2020 at 9:55 PM Daniel Imberman <[email protected]> wrote: > This could be pretty valuable for future audits. I’d personally rather > avoid adding fields to the DB in general. Could we store it wherever we > store normal failure messages? > > via Newton Mail > <https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2> > > On Fri, Oct 16, 2020 at 9:21 AM, Ace Haidrey > <[email protected]> wrote: > > Hi team, > > My name is Ace, I’m a data engineer at Pinterest, and I wanted to get some > feedback from the community on how they’d propose designing a feature that > has been asked for us multiple times, that we’d like to implement in-house. > > The ask is the following: > - Users want to be able to add an optional reason/description when they > click on the Mark Success/Failed for both task level in graph views or dag > level in tree views. This is to clearly state why the actions were taken on > higher priority flows for when others are stepping in to look in it. > > For us the feature makes sense to have, and I was wondering if this is > something the greater group would like to have upstream as well. And if so > then we wanted to know what was preferred. > > 1. We can add this information in the UI for the confirmation view and an > optional textbook to add details similar to how users can add conf > variables when triggering a dag option is selected in the UI. > > Then after this, storing this information is the question. Do we want to > store this information in the action logging extra blob and then add a view > in the dag view where it has actions just pertaining to this dag and its > tasks. (We plan to add this personally anyways for ourselves as going to > the big list of actions for all dags and searching is not convenient > always). > > Or do we want to add a new column in the task Instance model and dag run > model to store the descriptions and retrieve the information from there. > > Tradeoffs here include the addition of a schema change, where the new > column would generally be sparse, and the data is more resultant from an > action vs necessarily being stored as part of the task run/dag run. > > Please let me know on any feedback, concerns, ideas, and more! > > Cheers, > Ace > >
