danny0405 commented on a change in pull request #4880: URL: https://github.com/apache/hudi/pull/4880#discussion_r817327210
########## File path: hudi-common/src/main/java/org/apache/hudi/common/model/DeleteKey.java ########## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hudi.common.model; + +import java.util.Objects; + +/** + * Delete key is a combination of HoodieKey and ordering value. + * The key is used for {@link org.apache.hudi.common.table.log.block.HoodieDeleteBlock} + * to support per-record deletions. The deletion block is always appended after the data block, + * we need to keep the ordering val to combine with the data records when merging, or the data may + * be dropped if there are intermediate deletions for the inputs + * (a new INSERT comes after a DELETE in one input batch). + */ +public class DeleteKey extends HoodieKey { Review comment: > It's non BWC Already agreed with that. > It's introducing new way of deleting records that make COW and MOR diverge Can you just go over with the work flow in detail of COW and MOR combining process first ? The DELETE records encode/decode for MOR table is always there for efficiency. And i didn't introduce new diverge because it is designed there before. This PR only tag the old delete keys with version number and fix the event time semantics, if you want to address that the abstraction is not correct, please give a correct abstraction and let's discuss there and i'm glad to see that. ########## File path: hudi-common/src/main/java/org/apache/hudi/common/model/DeleteKey.java ########## @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hudi.common.model; + +import java.util.Objects; + +/** + * Delete key is a combination of HoodieKey and ordering value. + * The key is used for {@link org.apache.hudi.common.table.log.block.HoodieDeleteBlock} + * to support per-record deletions. The deletion block is always appended after the data block, + * we need to keep the ordering val to combine with the data records when merging, or the data may + * be dropped if there are intermediate deletions for the inputs + * (a new INSERT comes after a DELETE in one input batch). + */ +public class DeleteKey extends HoodieKey { Review comment: > It's non BWC Already agreed with that. > It's introducing new way of deleting records that make COW and MOR diverge Can you just go over with the work flow in detail of COW and MOR combining process first ? The DELETE records encode/decode for MOR table is always there for efficiency. And i didn't introduce new diverge because it is designed there before. This PR only tag the old delete keys with version number and fix the event time semantics, if you want to address that the abstraction is not correct, please give a correct abstraction and let's discuss there and i'm glad to see that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
