(airflow) branch main updated: Only show latest dataset event timestamp after last run (#38340)

[bbovenzi]

This is an automated email from the ASF dual-hosted git repository.

bbovenzi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new b9576b5aac Only show latest dataset event timestamp after last run 
(#38340)
b9576b5aac is described below

commit b9576b5aaca9271026f87fe61af07d19b7a1c545
Author: Brent Bovenzi <br...@astronomer.io>
AuthorDate: Mon Mar 25 10:23:41 2024 -0700

    Only show latest dataset event timestamp after last run (#38340)
    
    * Only show latest dataset event timestamp after last run
    
    * Move date filter to join statement in query
    
    * Update airflow/www/views.py
    
    Co-authored-by: Tzu-ping Chung <uranu...@gmail.com>
    
    ---------
    
    Co-authored-by: Tzu-ping Chung <uranu...@gmail.com>
---
 airflow/www/templates/airflow/dataset_next_run_modal.html | 2 +-
 airflow/www/views.py                                      | 9 ++++++++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/airflow/www/templates/airflow/dataset_next_run_modal.html 
b/airflow/www/templates/airflow/dataset_next_run_modal.html
index 5698c99681..6371cd17b2 100644
--- a/airflow/www/templates/airflow/dataset_next_run_modal.html
+++ b/airflow/www/templates/airflow/dataset_next_run_modal.html
@@ -42,7 +42,7 @@
             <thead>
               <tr>
                 <th>Dataset URI</th>
-                <th>Latest Update</th>
+                <th>Latest Update since last Dag Run</th>
               </tr>
             </thead>
             <tbody id="datasets_tbody">
diff --git a/airflow/www/views.py b/airflow/www/views.py
index c4a4f40898..9d66fe5bda 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -3290,6 +3290,8 @@ class Airflow(AirflowBaseView):
         with create_session() as session:
             dag_model = DagModel.get_dagmodel(dag_id, session=session)
 
+            latest_run = dag_model.get_last_dagrun(session=session)
+
             events = [
                 dict(info)
                 for info in session.execute(
@@ -3311,7 +3313,12 @@ class Airflow(AirflowBaseView):
                     )
                     .join(
                         DatasetEvent,
-                        DatasetEvent.dataset_id == DatasetModel.id,
+                        and_(
+                            DatasetEvent.dataset_id == DatasetModel.id,
+                            DatasetEvent.timestamp >= latest_run.execution_date
+                            if latest_run and latest_run.execution_date
+                            else True,
+                        ),
                         isouter=True,
                     )
                     .where(DagScheduleDatasetReference.dag_id == dag_id, 
~DatasetModel.is_orphaned)