+1 On Fri, Oct 16, 2020 at 5:48 PM Jarek Potiuk <[email protected]> wrote:
> +1 > > On Fri, Oct 16, 2020 at 6:42 PM Sumit Maheshwari <[email protected]> > wrote: > >> +1 for the feature. Looking at the schema of the log table, I think it's >> perfect to store such kind of information. >> >> On Fri, Oct 16, 2020 at 9:55 PM Daniel Imberman < >> [email protected]> wrote: >> >>> This could be pretty valuable for future audits. I’d personally rather >>> avoid adding fields to the DB in general. Could we store it wherever we >>> store normal failure messages? >>> >>> via Newton Mail >>> <https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2> >>> >>> On Fri, Oct 16, 2020 at 9:21 AM, Ace Haidrey >>> <[email protected]> wrote: >>> >>> Hi team, >>> >>> My name is Ace, I’m a data engineer at Pinterest, and I wanted to get >>> some feedback from the community on how they’d propose designing a feature >>> that has been asked for us multiple times, that we’d like to implement >>> in-house. >>> >>> The ask is the following: >>> - Users want to be able to add an optional reason/description when they >>> click on the Mark Success/Failed for both task level in graph views or dag >>> level in tree views. This is to clearly state why the actions were taken on >>> higher priority flows for when others are stepping in to look in it. >>> >>> For us the feature makes sense to have, and I was wondering if this is >>> something the greater group would like to have upstream as well. And if so >>> then we wanted to know what was preferred. >>> >>> 1. We can add this information in the UI for the confirmation view and >>> an optional textbook to add details similar to how users can add conf >>> variables when triggering a dag option is selected in the UI. >>> >>> Then after this, storing this information is the question. Do we want to >>> store this information in the action logging extra blob and then add a view >>> in the dag view where it has actions just pertaining to this dag and its >>> tasks. (We plan to add this personally anyways for ourselves as going to >>> the big list of actions for all dags and searching is not convenient >>> always). >>> >>> Or do we want to add a new column in the task Instance model and dag run >>> model to store the descriptions and retrieve the information from there. >>> >>> Tradeoffs here include the addition of a schema change, where the new >>> column would generally be sparse, and the data is more resultant from an >>> action vs necessarily being stored as part of the task run/dag run. >>> >>> Please let me know on any feedback, concerns, ideas, and more! >>> >>> Cheers, >>> Ace >>> >>> > > -- > > Jarek Potiuk > Polidea <https://www.polidea.com/> | Principal Software Engineer > > M: +48 660 796 129 <+48660796129> > [image: Polidea] <https://www.polidea.com/> > >
