ferruzzi commented on code in PR #31362:
URL: https://github.com/apache/airflow/pull/31362#discussion_r1196790955
##########
tests/system/providers/amazon/aws/example_dynamodb_to_s3.py:
##########
@@ -56,9 +56,24 @@ def set_up_table(table_name: str):
boto3.client("dynamodb").get_waiter("table_exists").wait(
TableName=table_name, WaiterConfig={"Delay": 10, "MaxAttempts": 10}
)
+ boto3.client("dynamodb").update_continuous_backups(
+ TableName=table_name,
+ PointInTimeRecoverySpecification={
+ "PointInTimeRecoveryEnabled": True,
+ },
+ )
table.put_item(Item={"ID": "123", "Value": "Testing"})
+@task
+def get_export_time(table_name: str):
+ r = boto3.client("dynamodb").describe_continuous_backups(
+ TableName=table_name,
+ )
+
+ return
r["ContinuousBackupsDescription"]["PointInTimeRecoveryDescription"]["EarliestRestorableDateTime"]
Review Comment:
Oh, that's nicer than my created time idea. Nice find.
##########
airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py:
##########
@@ -163,7 +168,7 @@ def _export_table_to_point_in_time(self):
def _export_entire_data(self):
"""Export all data from the table."""
- table = self.hook.get_conn().Table(self.dynamodb_table_name)
+ table = self.hook.conn.Table(self.dynamodb_table_name)
Review Comment:
Thanks.
##########
airflow/providers/amazon/aws/transfers/dynamodb_to_s3.py:
##########
@@ -97,9 +97,14 @@ class DynamoDBToS3Operator(AwsToAwsBaseOperator):
template_fields: Sequence[str] = (
*AwsToAwsBaseOperator.template_fields,
+ "dynamodb_table_name",
"s3_bucket_name",
+ "file_size",
+ "dynamodb_scan_kwargs",
"s3_key_prefix",
- "dynamodb_table_name",
+ "process_func",
+ "export_time",
+ "export_format",
Review Comment:
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
