rkkalluri commented on issue #5455:
URL: https://github.com/apache/hudi/issues/5455#issuecomment-1114302556
@stevenayers you should be able to use the glue catalog to load hudi table
like any other hive external table.
See if you can emulate the below for you needs.
# Read dataframe from source
input_dyf = glueContext.create_dynamic_frame.from_catalog(
database=src_database,
table_name=src_table_name,
push_down_predicate=f"(sdwh_update_year = '{start_date[:4]}' and
sdwh_update_month = '{start_date[5:7]}' and sdwh_update_day =
'{start_date[8:10]}')",
transformation_ctx="datasource0",
additional_options={"useS3ListImplementation": True, "groupFiles":
"inPartition", "boundedSize": "6516192768"},
)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
