Hi Ryan Thank you for correcting me. I was thinking of daily DAGs only when I suggested graph view, and tree view definitely makes more sense for multiple DAG runs per day.
Your images seem to have some problems showing up on my side, not sure if other people can see them. Best Bin On Tue, Apr 13, 2021 at 9:49 AM Ryan Hamilton <[email protected]> wrote: > In general, I really like this idea—it should be a useful visualization. > > > For the click destination, I think the Tree view does make more sense > given multiple runs can occur per day. The Graph view is limited to a > single run (which might not be the problematic one that instigated the > click). > > > I agree w/ Xinbin, it should probably have a base date/range selection. > Displaying “all time” history is a bit inconsistent with all of the other > views. > > > I like Sumit’s suggestion of having month and week views as well. > Certainly something this could evolve to add in the future. > > > I’m wondering if we could modify the presentation to remove the gaps > between months and instead outline months (similar to the following > screenshot)? At first glance, they could be misconstrued as gaps in runs. > > [image: image.png] > > > We should also add a link to the shortcuts to keep the navigation > consistent: > > > [image: image.png] > > > > On Tue, Apr 13, 2021 at 12:05 PM Xinbin Huang <[email protected]> > wrote: > >> Really like it! >> >> Some quick thoughts: >> - I think it will be better to have clicking redirect you to the graph >> view instead of the tree view >> - In the proposal, you mentioned both scheduled and manual triggered >> DAGs. Are they different "view options" for the calendar view that you can >> switch between? Or they are shown together probably with some visual >> differences? >> - Would be nice to have an option for the user to choose a "start_date" >> and "end_date" for the calendar view? But I am not sure about this, because >> it seems overlay with the tree view >> >> Cheers >> Bin >> >> On Mon, Apr 12, 2021 at 9:12 PM Sumit Maheshwari <[email protected]> >> wrote: >> >>> Nice thoughts, it would be a good addition to Airflow. >>> >>> A couple of suggestions: >>> >>> - A 3yr history is a lot, and most probably everyone out there >>> cleanup data older than 3-6 months. Also, it might involve a heavy query >>> for the datastore to handle. I would prefer a week or month view like we >>> have in the Google calendar and an option to switch between them and also >>> move back and forward. >>> - Maybe use yellow or orange color to denote days where some >>> failures and some successes happened. >>> - The color codes used to represent task states need to be removed >>> from the Calendar view and maybe introduce similar color codes to >>> represent >>> DAG states. >>> >>> >>> On Tue, Apr 13, 2021 at 5:52 AM Kaxil Naik <[email protected]> wrote: >>> >>>> Nice, I like it too, only minor suggestion is that it should be after >>>> Tree View and Graph View in the tab above. >>>> >>>> Regards, >>>> Kaxil >>>> >>>> On Mon, Apr 12, 2021 at 11:22 PM Brent Bovenzi >>>> <[email protected]> wrote: >>>> >>>>> Ryan Hamilton and I were talking about exactly this! Super excited to >>>>> see it. I'd be more than happy to help out if you need it. >>>>> >>>>> Quick thoughts: >>>>> - I wonder if we could provide more context at a glance than just >>>>> green/red. Possibly a gradient of percentage success/failed per day? >>>>> - I don't believe it should be the default view for a DAG as it is >>>>> mainly a historical view rather than a recent view. >>>>> >>>>> - Brent >>>>> >>>>> >>>>> On Mon, Apr 12, 2021 at 6:00 PM Benoit H <[email protected]> >>>>> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> >>>>>> I would like to share with you a proposal for the implementation of a >>>>>> dag "calendar view" in the Airflow UI, which is a feature that I find >>>>>> very >>>>>> useful when managing dags with a large number of dag runs. >>>>>> >>>>>> >>>>>> The aim is to provide visibility over the full state of the dag by >>>>>> displaying the aggregated dag runs' states in a calendar. >>>>>> >>>>>> Each day is displayed with a color according to the dag runs' states >>>>>> for that day: >>>>>> >>>>>> - If at least one dag run has failed for a day, that day will be >>>>>> displayed as "failed". >>>>>> >>>>>> - If all dag runs have succeeded the day will be shown as >>>>>> "succeeded". >>>>>> >>>>>> - If there are still running dag runs (and no failed dag run) for >>>>>> that day, the day will be shown as "running". >>>>>> >>>>>> Clicking on a day redirects to the tree view for that day. >>>>>> >>>>>> >>>>>> This makes it possible to monitor the state of thousands of dag runs >>>>>> in a single view that is concise and easy to understand. It is >>>>>> particularly >>>>>> useful to monitor the state of large backfills. >>>>>> >>>>>> >>>>>> You may find screenshots, as well as additional details, in the >>>>>> following Google doc: >>>>>> https://docs.google.com/document/d/1fayWWbia7r1iPuHL23JeKJCP5JcKdOlHpLzrdAH0nT4/edit?usp=sharing >>>>>> . >>>>>> >>>>>> A prototype implementation is available at >>>>>> https://github.com/BenoitHanotte/airflow/pull/2/files. >>>>>> >>>>>> >>>>>> I'd gladly get your feedback on the idea, and on whether it is worth >>>>>> moving forward by creating an AIP to formalize this proposal. >>>>>> >>>>>> >>>>>> Thank you! >>>>>> >>>>>> >>>>>> Benoit Hanotte >>>>>> >>>>>
