vinothchandar commented on a change in pull request #1540:
URL: https://github.com/apache/incubator-hudi/pull/1540#discussion_r414295319
##
File path: hudi-client/src/main/java/org/apache/hudi/io/AppendHandleFactory.java
##
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io;
+
+import org.apache.hudi.client.SparkTaskContextSupplier;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.table.HoodieTable;
+
+public class AppendHandleFactory extends
CreateHandleFactory {
Review comment:
This inheritance is kind of confusing..
##
File path: hudi-client/src/main/java/org/apache/hudi/io/AppendHandleFactory.java
##
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io;
+
+import org.apache.hudi.client.SparkTaskContextSupplier;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.table.HoodieTable;
+
+public class AppendHandleFactory extends
CreateHandleFactory {
Review comment:
if you need some method in both Append and Create Handle factories,
perhaps move it as a default method in the interface?
##
File path:
hudi-client/src/main/java/org/apache/hudi/io/WriteHandleCreatorFactory.java
##
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.io;
+
+import org.apache.hudi.client.SparkTaskContextSupplier;
+import org.apache.hudi.common.model.HoodieRecordPayload;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.table.HoodieTable;
+
+public interface WriteHandleCreatorFactory {
Review comment:
Rename to just `WriteHandleFactory`?
##
File path:
hudi-client/src/main/java/org/apache/hudi/execution/LazyInsertIterable.java
##
@@ -43,26 +44,34 @@
/**
* Lazy Iterable, that writes a stream of HoodieRecords sorted by the
partitionPath, into new files.
*/
-public class CopyOnWriteLazyInsertIterable
+public class LazyInsertIterable
extends LazyIterableIterator, List> {
protected final HoodieWriteConfig hoodieConfig;
protected final String instantTime;
protected final HoodieTable hoodieTable;
protected final String idPrefix;
- protected int numFilesWritten;
protected SparkTaskContextSupplier sparkTaskContextSupplier;
+ protected WriteHandleCreatorFactory writeHandleCreatorFactory;
Review comment:
please rename this accordingly as well
This
