jedcunningham commented on code in PR #29058:
URL: https://github.com/apache/airflow/pull/29058#discussion_r1090860622
##########
airflow/utils/db_cleanup.py:
##########
@@ -123,6 +127,15 @@ def _check_for_rows(*, query: Query, print_rows=False):
return num_entities
+def _dump_db(*, target_table, file_path, export_format, session):
Review Comment:
```suggestion
def _dump_table_to_file(*, target_table, file_path, export_format, session):
```
nit
##########
airflow/cli/cli_parser.py:
##########
@@ -1589,6 +1608,15 @@ class GroupCommand(NamedTuple):
ARG_DB_SKIP_ARCHIVE,
),
),
+ ActionCommand(
+ name="export-cleaned",
+ help="Export cleaned data from the archive tables",
+
func=lazy_load_command("airflow.cli.commands.db_command.export_archived"),
Review Comment:
We have `export-cleaned` and `export_archived`, we should pick one.
##########
airflow/utils/db_cleanup.py:
##########
@@ -370,3 +383,19 @@ def run_cleanup(
session=session,
)
session.commit()
+
+
+@provide_session
+def export_archived_records(export_format, output_path, session: Session =
NEW_SESSION):
+ """Export cleaned data to the given output path in the given format."""
+ inspector = inspect(session.bind)
+ table_names = inspector.get_table_names()
+ for table_name in table_names:
+ if table_name.startswith(ARCHIVE_TABLE_PREFIX):
+ _dump_db(
+ target_table=table_name,
+ file_path=os.path.join(output_path,
f"{table_name}.{export_format}"),
+ export_format=export_format,
+ session=session,
+ )
Review Comment:
I feel like we should have a flag to drop the table as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
