In general, I really like this idea—it should be a useful visualization.
For the click destination, I think the Tree view does make more sense given multiple runs can occur per day. The Graph view is limited to a single run (which might not be the problematic one that instigated the click). I agree w/ Xinbin, it should probably have a base date/range selection. Displaying “all time” history is a bit inconsistent with all of the other views. I like Sumit’s suggestion of having month and week views as well. Certainly something this could evolve to add in the future. I’m wondering if we could modify the presentation to remove the gaps between months and instead outline months (similar to the following screenshot)? At first glance, they could be misconstrued as gaps in runs. [image: image.png] We should also add a link to the shortcuts to keep the navigation consistent: [image: image.png] On Tue, Apr 13, 2021 at 12:05 PM Xinbin Huang <[email protected]> wrote: > Really like it! > > Some quick thoughts: > - I think it will be better to have clicking redirect you to the graph > view instead of the tree view > - In the proposal, you mentioned both scheduled and manual triggered DAGs. > Are they different "view options" for the calendar view that you can switch > between? Or they are shown together probably with some visual differences? > - Would be nice to have an option for the user to choose a "start_date" > and "end_date" for the calendar view? But I am not sure about this, because > it seems overlay with the tree view > > Cheers > Bin > > On Mon, Apr 12, 2021 at 9:12 PM Sumit Maheshwari <[email protected]> > wrote: > >> Nice thoughts, it would be a good addition to Airflow. >> >> A couple of suggestions: >> >> - A 3yr history is a lot, and most probably everyone out there >> cleanup data older than 3-6 months. Also, it might involve a heavy query >> for the datastore to handle. I would prefer a week or month view like we >> have in the Google calendar and an option to switch between them and also >> move back and forward. >> - Maybe use yellow or orange color to denote days where some failures >> and some successes happened. >> - The color codes used to represent task states need to be removed >> from the Calendar view and maybe introduce similar color codes to >> represent >> DAG states. >> >> >> On Tue, Apr 13, 2021 at 5:52 AM Kaxil Naik <[email protected]> wrote: >> >>> Nice, I like it too, only minor suggestion is that it should be after >>> Tree View and Graph View in the tab above. >>> >>> Regards, >>> Kaxil >>> >>> On Mon, Apr 12, 2021 at 11:22 PM Brent Bovenzi >>> <[email protected]> wrote: >>> >>>> Ryan Hamilton and I were talking about exactly this! Super excited to >>>> see it. I'd be more than happy to help out if you need it. >>>> >>>> Quick thoughts: >>>> - I wonder if we could provide more context at a glance than just >>>> green/red. Possibly a gradient of percentage success/failed per day? >>>> - I don't believe it should be the default view for a DAG as it is >>>> mainly a historical view rather than a recent view. >>>> >>>> - Brent >>>> >>>> >>>> On Mon, Apr 12, 2021 at 6:00 PM Benoit H <[email protected]> wrote: >>>> >>>>> Hello, >>>>> >>>>> >>>>> I would like to share with you a proposal for the implementation of a >>>>> dag "calendar view" in the Airflow UI, which is a feature that I find very >>>>> useful when managing dags with a large number of dag runs. >>>>> >>>>> >>>>> The aim is to provide visibility over the full state of the dag by >>>>> displaying the aggregated dag runs' states in a calendar. >>>>> >>>>> Each day is displayed with a color according to the dag runs' states >>>>> for that day: >>>>> >>>>> - If at least one dag run has failed for a day, that day will be >>>>> displayed as "failed". >>>>> >>>>> - If all dag runs have succeeded the day will be shown as "succeeded". >>>>> >>>>> - If there are still running dag runs (and no failed dag run) for that >>>>> day, the day will be shown as "running". >>>>> >>>>> Clicking on a day redirects to the tree view for that day. >>>>> >>>>> >>>>> This makes it possible to monitor the state of thousands of dag runs >>>>> in a single view that is concise and easy to understand. It is >>>>> particularly >>>>> useful to monitor the state of large backfills. >>>>> >>>>> >>>>> You may find screenshots, as well as additional details, in the >>>>> following Google doc: >>>>> https://docs.google.com/document/d/1fayWWbia7r1iPuHL23JeKJCP5JcKdOlHpLzrdAH0nT4/edit?usp=sharing >>>>> . >>>>> >>>>> A prototype implementation is available at >>>>> https://github.com/BenoitHanotte/airflow/pull/2/files. >>>>> >>>>> >>>>> I'd gladly get your feedback on the idea, and on whether it is worth >>>>> moving forward by creating an AIP to formalize this proposal. >>>>> >>>>> >>>>> Thank you! >>>>> >>>>> >>>>> Benoit Hanotte >>>>> >>>>
