Here is the first screenshot if the image didn’t come through: https://r.hmlt.in/8Lubnr2e
The second one is moot since I saw that Benoit already addressed my suggestion in the provided code. -rh > On Apr 13, 2021, at 1:04 PM, Xinbin Huang <[email protected]> wrote: > > > Hi Ryan > > Thank you for correcting me. I was thinking of daily DAGs only when I > suggested graph view, and tree view definitely makes more sense for multiple > DAG runs per day. > > Your images seem to have some problems showing up on my side, not sure if > other people can see them. > > Best > Bin > >> On Tue, Apr 13, 2021 at 9:49 AM Ryan Hamilton >> <[email protected]> wrote: >> In general, I really like this idea—it should be a useful visualization. >> >> For the click destination, I think the Tree view does make more sense given >> multiple runs can occur per day. The Graph view is limited to a single run >> (which might not be the problematic one that instigated the click). >> >> I agree w/ Xinbin, it should probably have a base date/range selection. >> Displaying “all time” history is a bit inconsistent with all of the other >> views. >> >> I like Sumit’s suggestion of having month and week views as well. Certainly >> something this could evolve to add in the future. >> >> I’m wondering if we could modify the presentation to remove the gaps between >> months and instead outline months (similar to the following screenshot)? At >> first glance, they could be misconstrued as gaps in runs. >> >> >> We should also add a link to the shortcuts to keep the navigation consistent: >> >> >> >> >>> On Tue, Apr 13, 2021 at 12:05 PM Xinbin Huang <[email protected]> wrote: >>> Really like it! >>> >>> Some quick thoughts: >>> - I think it will be better to have clicking redirect you to the graph view >>> instead of the tree view >>> - In the proposal, you mentioned both scheduled and manual triggered DAGs. >>> Are they different "view options" for the calendar view that you can switch >>> between? Or they are shown together probably with some visual differences? >>> - Would be nice to have an option for the user to choose a "start_date" and >>> "end_date" for the calendar view? But I am not sure about this, because it >>> seems overlay with the tree view >>> >>> Cheers >>> Bin >>> >>>> On Mon, Apr 12, 2021 at 9:12 PM Sumit Maheshwari <[email protected]> >>>> wrote: >>>> Nice thoughts, it would be a good addition to Airflow. >>>> >>>> A couple of suggestions: >>>> A 3yr history is a lot, and most probably everyone out there cleanup data >>>> older than 3-6 months. Also, it might involve a heavy query for the >>>> datastore to handle. I would prefer a week or month view like we have in >>>> the Google calendar and an option to switch between them and also move >>>> back and forward. >>>> Maybe use yellow or orange color to denote days where some failures and >>>> some successes happened. >>>> The color codes used to represent task states need to be removed from the >>>> Calendar view and maybe introduce similar color codes to represent DAG >>>> states. >>>> >>>> On Tue, Apr 13, 2021 at 5:52 AM Kaxil Naik <[email protected]> wrote: >>>>> Nice, I like it too, only minor suggestion is that it should be after >>>>> Tree View and Graph View in the tab above. >>>>> >>>>> Regards, >>>>> Kaxil >>>>> >>>>>> On Mon, Apr 12, 2021 at 11:22 PM Brent Bovenzi >>>>>> <[email protected]> wrote: >>>>>> Ryan Hamilton and I were talking about exactly this! Super excited to >>>>>> see it. I'd be more than happy to help out if you need it. >>>>>> >>>>>> Quick thoughts: >>>>>> - I wonder if we could provide more context at a glance than just >>>>>> green/red. Possibly a gradient of percentage success/failed per day? >>>>>> - I don't believe it should be the default view for a DAG as it is >>>>>> mainly a historical view rather than a recent view. >>>>>> >>>>>> - Brent >>>>>> >>>>>> >>>>>>> On Mon, Apr 12, 2021 at 6:00 PM Benoit H <[email protected]> wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I would like to share with you a proposal for the implementation of a >>>>>>> dag "calendar view" in the Airflow UI, which is a feature that I find >>>>>>> very useful when managing dags with a large number of dag runs. >>>>>>> >>>>>>> The aim is to provide visibility over the full state of the dag by >>>>>>> displaying the aggregated dag runs' states in a calendar. >>>>>>> Each day is displayed with a color according to the dag runs' states >>>>>>> for that day: >>>>>>> - If at least one dag run has failed for a day, that day will be >>>>>>> displayed as "failed". >>>>>>> - If all dag runs have succeeded the day will be shown as "succeeded". >>>>>>> - If there are still running dag runs (and no failed dag run) for that >>>>>>> day, the day will be shown as "running". >>>>>>> Clicking on a day redirects to the tree view for that day. >>>>>>> >>>>>>> This makes it possible to monitor the state of thousands of dag runs in >>>>>>> a single view that is concise and easy to understand. It is >>>>>>> particularly useful to monitor the state of large backfills. >>>>>>> >>>>>>> You may find screenshots, as well as additional details, in the >>>>>>> following Google doc: >>>>>>> https://docs.google.com/document/d/1fayWWbia7r1iPuHL23JeKJCP5JcKdOlHpLzrdAH0nT4/edit?usp=sharing. >>>>>>> A prototype implementation is available at >>>>>>> https://github.com/BenoitHanotte/airflow/pull/2/files. >>>>>>> >>>>>>> I'd gladly get your feedback on the idea, and on whether it is worth >>>>>>> moving forward by creating an AIP to formalize this proposal. >>>>>>> >>>>>>> Thank you! >>>>>>> >>>>>>> Benoit Hanotte
