noahtaite commented on issue #6048: URL: https://github.com/apache/hudi/issues/6048#issuecomment-1177685164
@yihua Yes, we can try EMR 5.36 which is released with the following Hudi version: 0.10.1-amzn-1. Do we need to re-generate the metadata table, or will the existing one for our large experiment work ok? Yes, here is the content of the metadata table: ``` 2022-07-04 13:27:11 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/.aux/.bootstrap/.fileids_$folder$ 2022-07-04 13:27:10 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/.aux/.bootstrap/.partitions_$folder$ 2022-07-04 13:27:10 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/.aux/.bootstrap_$folder$ 2022-07-04 13:27:09 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/.aux_$folder$ 2022-07-04 13:27:09 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/.temp_$folder$ 2022-07-04 13:27:26 2205 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704141215.deltacommit 2022-07-04 13:27:22 1189 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704141215.deltacommit.inflight 2022-07-04 13:27:21 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704141215.deltacommit.requested 2022-07-04 14:33:09 2454 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704162653.deltacommit 2022-07-04 14:33:04 1907 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704162653.deltacommit.inflight 2022-07-04 14:33:03 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704162653.deltacommit.requested 2022-07-04 15:44:30 2532 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704170712.deltacommit 2022-07-04 15:44:25 1907 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704170712.deltacommit.inflight 2022-07-04 15:44:24 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704170712.deltacommit.requested 2022-07-04 16:36:17 2605 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704181907.deltacommit 2022-07-04 16:36:13 1907 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704181907.deltacommit.inflight 2022-07-04 16:36:12 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704181907.deltacommit.requested 2022-07-05 03:08:03 2673 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704190618.deltacommit 2022-07-05 03:07:59 1904 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704190618.deltacommit.inflight 2022-07-05 03:07:58 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220704190618.deltacommit.requested 2022-07-05 03:08:15 2761 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705031627.deltacommit 2022-07-05 03:08:09 1908 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705031627.deltacommit.inflight 2022-07-05 03:08:07 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705031627.deltacommit.requested 2022-07-05 12:55:26 2837 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705122401.deltacommit 2022-07-05 12:55:21 1908 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705122401.deltacommit.inflight 2022-07-05 12:55:19 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/20220705122401.deltacommit.requested 2022-07-04 13:27:09 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/archived_$folder$ 2022-07-04 13:27:11 361 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie/hoodie.properties 2022-07-04 13:27:08 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/.hoodie_$folder$ 2022-07-04 14:33:08 294258 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.1_0-66-77005 2022-07-04 15:44:30 315616 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.2_0-66-148500 2022-07-04 16:36:16 314918 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.3_0-69-86707 2022-07-05 03:08:03 5947 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.4_0-69-239755 2022-07-05 03:08:14 971618 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.5_0-81-242837 2022-07-05 12:55:25 985699 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_20220704141215.log.6_0-74-271022 2022-07-04 13:27:23 93 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/.hoodie_partition_metadata 2022-07-04 13:27:25 469982 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files/2110f5ce-9f6b-4a12-9ff2-b74889a03aa5-0_0-55-127122_20220704141215.hfile 2022-07-04 13:27:12 0 hudi_experiment/20220704/transactions.all_hudi/.hoodie/metadata/files_$folder$ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
