Did anyone consider having a more generic comments feature? The feature Ace described is an example of a comment being useful. E.g. when a user clears a task or marks it success/failed, he probably wants to include a comment for his action when he clicks OK. If a task failed and the user decided to do nothing about it, he may also wants to leave a comment with the reason.
It'll be nice if these comments can optionally be displayed on the UI somewhere. To do that, it's likely cleaner to create a dedicated comments table where each comment has a foreign key to associate it with the TaskInstance. On Sat, Oct 17, 2020 at 1:30 AM Vikram Koka <[email protected]> wrote: > +1 for feature. Strong preference for storing it in the log table, rather > than in the task instance. > > > > On Fri, Oct 16, 2020 at 10:22 AM Tomasz Urbaszek <[email protected]> > wrote: > >> +1, I know a customer who was requesting it too >> >> On Fri, Oct 16, 2020 at 6:52 PM Kaxil Naik <[email protected]> wrote: >> >>> +1 >>> >>> On Fri, Oct 16, 2020 at 5:48 PM Jarek Potiuk <[email protected]> >>> wrote: >>> >>>> +1 >>>> >>>> On Fri, Oct 16, 2020 at 6:42 PM Sumit Maheshwari < >>>> [email protected]> wrote: >>>> >>>>> +1 for the feature. Looking at the schema of the log table, I think >>>>> it's perfect to store such kind of information. >>>>> >>>>> On Fri, Oct 16, 2020 at 9:55 PM Daniel Imberman < >>>>> [email protected]> wrote: >>>>> >>>>>> This could be pretty valuable for future audits. I’d personally >>>>>> rather avoid adding fields to the DB in general. Could we store it >>>>>> wherever >>>>>> we store normal failure messages? >>>>>> >>>>>> via Newton Mail >>>>>> <https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2> >>>>>> >>>>>> On Fri, Oct 16, 2020 at 9:21 AM, Ace Haidrey >>>>>> <[email protected]> wrote: >>>>>> >>>>>> Hi team, >>>>>> >>>>>> My name is Ace, I’m a data engineer at Pinterest, and I wanted to get >>>>>> some feedback from the community on how they’d propose designing a >>>>>> feature >>>>>> that has been asked for us multiple times, that we’d like to implement >>>>>> in-house. >>>>>> >>>>>> The ask is the following: >>>>>> - Users want to be able to add an optional reason/description when >>>>>> they click on the Mark Success/Failed for both task level in graph views >>>>>> or >>>>>> dag level in tree views. This is to clearly state why the actions were >>>>>> taken on higher priority flows for when others are stepping in to look in >>>>>> it. >>>>>> >>>>>> For us the feature makes sense to have, and I was wondering if this >>>>>> is something the greater group would like to have upstream as well. And >>>>>> if >>>>>> so then we wanted to know what was preferred. >>>>>> >>>>>> 1. We can add this information in the UI for the confirmation view >>>>>> and an optional textbook to add details similar to how users can add conf >>>>>> variables when triggering a dag option is selected in the UI. >>>>>> >>>>>> Then after this, storing this information is the question. Do we want >>>>>> to store this information in the action logging extra blob and then add a >>>>>> view in the dag view where it has actions just pertaining to this dag and >>>>>> its tasks. (We plan to add this personally anyways for ourselves as going >>>>>> to the big list of actions for all dags and searching is not convenient >>>>>> always). >>>>>> >>>>>> Or do we want to add a new column in the task Instance model and dag >>>>>> run model to store the descriptions and retrieve the information from >>>>>> there. >>>>>> >>>>>> Tradeoffs here include the addition of a schema change, where the new >>>>>> column would generally be sparse, and the data is more resultant from an >>>>>> action vs necessarily being stored as part of the task run/dag run. >>>>>> >>>>>> Please let me know on any feedback, concerns, ideas, and more! >>>>>> >>>>>> Cheers, >>>>>> Ace >>>>>> >>>>>> >>>> >>>> -- >>>> >>>> Jarek Potiuk >>>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>>> >>>> M: +48 660 796 129 <+48660796129> >>>> [image: Polidea] <https://www.polidea.com/> >>>> >>>>
