codope commented on code in PR #12097:
URL: https://github.com/apache/hudi/pull/12097#discussion_r1800477602
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseActionExecutor.java:
##########
@@ -61,10 +64,18 @@ public BaseActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
*
* @param metadata commit metadata of interest.
*/
- protected final void writeTableMetadata(HoodieCommitMetadata metadata,
HoodieData<WriteStatus> writeStatus, String actionType) {
+ protected final void writeTableMetadata(HoodieCommitMetadata metadata,
+ HoodieData<WriteStatus> writeStatus,
+ WriteOperationType operationType) {
Review Comment:
This arg is not needed, operationType is available in commitMetadata.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseActionExecutor.java:
##########
@@ -61,10 +64,18 @@ public BaseActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
*
* @param metadata commit metadata of interest.
*/
- protected final void writeTableMetadata(HoodieCommitMetadata metadata,
HoodieData<WriteStatus> writeStatus, String actionType) {
+ protected final void writeTableMetadata(HoodieCommitMetadata metadata,
+ HoodieData<WriteStatus> writeStatus,
+ WriteOperationType operationType) {
Option<HoodieTableMetadataWriter> metadataWriterOpt =
table.getMetadataWriter(instantTime);
if (metadataWriterOpt.isPresent()) {
try (HoodieTableMetadataWriter metadataWriter = metadataWriterOpt.get())
{
+ // Recreate MDT for insert_overwrite_table operation.
+ if (WriteOperationType.INSERT_OVERWRITE_TABLE == operationType) {
+ HoodieTableMetadataUtil.deleteMetadataTable(table.getMetaClient(),
table.getContext(), false);
+ new RunIndexActionExecutor<>(context, config, table,
instantTime).execute();
Review Comment:
Since we are deleting the whole metadata table, I think it makes more sense
to follow the usual `BaseHoodieWriteClient.initMetadataTable` (check its spark
implementation). Let;s see if we can reuse that instead of using
RunIndexActionExecutor.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/BaseActionExecutor.java:
##########
@@ -61,10 +64,18 @@ public BaseActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
*
* @param metadata commit metadata of interest.
*/
- protected final void writeTableMetadata(HoodieCommitMetadata metadata,
HoodieData<WriteStatus> writeStatus, String actionType) {
+ protected final void writeTableMetadata(HoodieCommitMetadata metadata,
+ HoodieData<WriteStatus> writeStatus,
+ WriteOperationType operationType) {
Option<HoodieTableMetadataWriter> metadataWriterOpt =
table.getMetadataWriter(instantTime);
if (metadataWriterOpt.isPresent()) {
try (HoodieTableMetadataWriter metadataWriter = metadataWriterOpt.get())
{
+ // Recreate MDT for insert_overwrite_table operation.
+ if (WriteOperationType.INSERT_OVERWRITE_TABLE == operationType) {
+ HoodieTableMetadataUtil.deleteMetadataTable(table.getMetaClient(),
table.getContext(), false);
+ new RunIndexActionExecutor<>(context, config, table,
instantTime).execute();
+ }
Review Comment:
i believe this is being called within a transaction. Could you please
confirm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
