[
https://issues.apache.org/jira/browse/HUDI-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1446:
---------------------------------
Labels: pull-request-available (was: )
> Support skip bootstrapIndex's init in abstract fs view init
> -----------------------------------------------------------
>
> Key: HUDI-1446
> URL: https://issues.apache.org/jira/browse/HUDI-1446
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: ann
> Priority: Major
> Labels: pull-request-available
>
> the bootstrapIndex default is HFileBootstrapIndex. If program doesn't include
> hbase pom, it would throw a exception as follows:
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/util/BytesCaused by: java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/util/Bytes at
> org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex.<clinit>(HFileBootstrapIndex.java:92)
> at java.lang.Class.forName0(Native Method) at
> java.lang.Class.forName(Class.java:264) at
> org.apache.hudi.common.util.ReflectionUtils.getClass(ReflectionUtils.java:53)
> at
> org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:87)
> at
> org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:98)
> at
> org.apache.hudi.common.bootstrap.index.BootstrapIndex.getBootstrapIndex(BootstrapIndex.java:159)
> at
> org.apache.hudi.common.table.view.AbstractTableFileSystemView.init(AbstractTableFileSystemView.java:106)
> at
> org.apache.hudi.common.table.view.HoodieTableFileSystemView.init(HoodieTableFileSystemView.java:106)
> at
> org.apache.hudi.common.table.view.HoodieTableFileSystemView.<init>(HoodieTableFileSystemView.java:100)
> at
> org.apache.hudi.common.table.view.FileSystemViewManager.createInMemoryFileSystemView(FileSystemViewManager.java:158)
> at
> org.apache.hudi.common.table.view.FileSystemViewManager.lambda$createViewManager$ab86fd6b$5(FileSystemViewManager.java:213)
> at
> org.apache.hudi.common.table.view.FileSystemViewManager.lambda$getFileSystemView$1(FileSystemViewManager.java:104)
> at
> java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
> at
> org.apache.hudi.common.table.view.FileSystemViewManager.getFileSystemView(FileSystemViewManager.java:103)
> at
> org.apache.hudi.table.HoodieTable.getBaseFileOnlyView(HoodieTable.java:242)
> at
> org.apache.hudi.table.action.commit.UpsertPartitioner.getSmallFiles(UpsertPartitioner.java:230)
> at
> org.apache.hudi.table.action.commit.UpsertPartitioner.lambda$getSmallFilesForPartitions$681ff4ab$1(UpsertPartitioner.java:213)
> at
> org.apache.hudi.client.common.function.FunctionWrapper.lambda$throwingMapToPairWrapper$3(FunctionWrapper.java:68)
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at
> org.apache.hudi.client.common.HoodieFlinkEngineContext.mapToPair(HoodieFlinkEngineContext.java:79)
> at
> org.apache.hudi.table.action.commit.UpsertPartitioner.getSmallFilesForPartitions(UpsertPartitioner.java:213)
> at
> org.apache.hudi.table.action.commit.UpsertPartitioner.assignInserts(UpsertPartitioner.java:136)
> at
> org.apache.hudi.table.action.commit.UpsertPartitioner.<init>(UpsertPartitioner.java:97)
> at
> org.apache.hudi.table.action.commit.BaseFlinkCommitActionExecutor.getUpsertPartitioner(BaseFlinkCommitActionExecutor.java:321)
> at
> org.apache.hudi.table.action.commit.BaseFlinkCommitActionExecutor.getPartitioner(BaseFlinkCommitActionExecutor.java:144)
> at
> org.apache.hudi.table.action.commit.BaseFlinkCommitActionExecutor.execute(BaseFlinkCommitActionExecutor.java:114)
> at
> org.apache.hudi.table.action.commit.BaseFlinkCommitActionExecutor.execute(BaseFlinkCommitActionExecutor.java:68)
> at
> org.apache.hudi.table.action.commit.AbstractWriteHelper.write(AbstractWriteHelper.java:55)
>
>
> When write data into hoodie table using operation api, shouldn't init a
> bootstrap index.
> The patch change Bootstrap api , it would return a null if execution config
> does not set bootstrap path, and skip some process when index is null in
> abstract fs view.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
