+1 for feature. Strong preference for storing it in the log table, rather than in the task instance.
On Fri, Oct 16, 2020 at 10:22 AM Tomasz Urbaszek <[email protected]> wrote: > +1, I know a customer who was requesting it too > > On Fri, Oct 16, 2020 at 6:52 PM Kaxil Naik <[email protected]> wrote: > >> +1 >> >> On Fri, Oct 16, 2020 at 5:48 PM Jarek Potiuk <[email protected]> >> wrote: >> >>> +1 >>> >>> On Fri, Oct 16, 2020 at 6:42 PM Sumit Maheshwari <[email protected]> >>> wrote: >>> >>>> +1 for the feature. Looking at the schema of the log table, I think >>>> it's perfect to store such kind of information. >>>> >>>> On Fri, Oct 16, 2020 at 9:55 PM Daniel Imberman < >>>> [email protected]> wrote: >>>> >>>>> This could be pretty valuable for future audits. I’d personally rather >>>>> avoid adding fields to the DB in general. Could we store it wherever we >>>>> store normal failure messages? >>>>> >>>>> via Newton Mail >>>>> <https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2> >>>>> >>>>> On Fri, Oct 16, 2020 at 9:21 AM, Ace Haidrey >>>>> <[email protected]> wrote: >>>>> >>>>> Hi team, >>>>> >>>>> My name is Ace, I’m a data engineer at Pinterest, and I wanted to get >>>>> some feedback from the community on how they’d propose designing a feature >>>>> that has been asked for us multiple times, that we’d like to implement >>>>> in-house. >>>>> >>>>> The ask is the following: >>>>> - Users want to be able to add an optional reason/description when >>>>> they click on the Mark Success/Failed for both task level in graph views >>>>> or >>>>> dag level in tree views. This is to clearly state why the actions were >>>>> taken on higher priority flows for when others are stepping in to look in >>>>> it. >>>>> >>>>> For us the feature makes sense to have, and I was wondering if this is >>>>> something the greater group would like to have upstream as well. And if so >>>>> then we wanted to know what was preferred. >>>>> >>>>> 1. We can add this information in the UI for the confirmation view and >>>>> an optional textbook to add details similar to how users can add conf >>>>> variables when triggering a dag option is selected in the UI. >>>>> >>>>> Then after this, storing this information is the question. Do we want >>>>> to store this information in the action logging extra blob and then add a >>>>> view in the dag view where it has actions just pertaining to this dag and >>>>> its tasks. (We plan to add this personally anyways for ourselves as going >>>>> to the big list of actions for all dags and searching is not convenient >>>>> always). >>>>> >>>>> Or do we want to add a new column in the task Instance model and dag >>>>> run model to store the descriptions and retrieve the information from >>>>> there. >>>>> >>>>> Tradeoffs here include the addition of a schema change, where the new >>>>> column would generally be sparse, and the data is more resultant from an >>>>> action vs necessarily being stored as part of the task run/dag run. >>>>> >>>>> Please let me know on any feedback, concerns, ideas, and more! >>>>> >>>>> Cheers, >>>>> Ace >>>>> >>>>> >>> >>> -- >>> >>> Jarek Potiuk >>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>> >>> M: +48 660 796 129 <+48660796129> >>> [image: Polidea] <https://www.polidea.com/> >>> >>>
