LiJie20190102 opened a new issue, #8257: URL: https://github.com/apache/hudi/issues/8257
**_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? - Join the mailing list to engage in conversations and get faster support at [email protected]. - If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly. **Describe the problem you faced** HoodieDeltaStreamer (0.13.0 ),FileSystem is null,resulting in a NullPointerException **To Reproduce** Steps to reproduce the behavior: 1. Download hudi-release-0.13.0.tar.gz 2. Adjust the frame version in the pom file 3. mvn clean package -DskipTests -Dspark3.2 -Dscala-2.12 4. Kafka needs to keep protuducing data 5. use hudi-utilities-bundle **Expected behavior** Can continuously consume data from kafka and synchronize the data to hive **Environment Description** * Hudi version :0.13.0 * Spark version : 3.2.3 * Hive version : 3.1.3 * Hadoop version : 3.3.4 * Storage (HDFS/S3/GCS..) : HDFS * Running on Docker? (yes/no) : no **Additional context** spark-submit --master yarn --deploy-mode client --driver-cores 1 --driver-memory 1G --num-executors 1 --executor-cores 2 --executor-memory 1g \ --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer /usr/lib/spark/jars/hudi-utilities-bundle_2.12-0.13.0.jar \ --props file:///home/lj/kafka-source.properties \ --hoodie-conf hoodie.datasource.hive_sync.mode=hms \ --schemaprovider-class org.apache.hudi.utilities.schema.FilebasedSchemaProvider \ --source-class org.apache.hudi.utilities.sources.JsonKafkaSource \ --source-ordering-field qsdiAccessTime \ --table-type COPY_ON_WRITE \ --target-table test_aa1 \ --target-base-path hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1 \ --continuous \ --enable-sync \ --op UPSERT **_pom(hudi) change section_** <kafka.spark3.version>2.8.1</kafka.spark3.version> <hadoop.version>3.3.4</hadoop.version> <hive.version>3.1.3</hive.version> <flink1.15.version>1.15.3</flink1.15.version> <hbase.version>2.4.13</hbase.version> <zookeeper.version>3.5.9</zookeeper.version> **_pom(hudi-utilities-bundle_2.12) change section_** <include>org.apache.hbase.thirdparty:hbase-unsafe</include> <include>mysql:mysql-connector-java</include> <include>com.alibaba:druid</include> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-service</artifactId> <version>${hive.version}</version> <scope>${utilities.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>org.apache.hbase</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <artifactId>servlet-api</artifactId> <groupId>javax.servlet</groupId> </exclusion> <exclusion> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.pentaho</groupId> <artifactId>*</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-jdbc</artifactId> <version>${hive.version}</version> <scope>${utilities.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-metastore</artifactId> <version>${hive.version}</version> <scope>${utilities.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>org.apache.hbase</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>javax.servlet</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>com.goole.guava</groupId> <artifactId>guava</artifactId> </exclusion> <exclusion> <groupId>org.datanucleus</groupId> <artifactId>datanucleus-core</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-common</artifactId> <version>${hive.version}</version> <scope>${utilities.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty.orbit</groupId> <artifactId>javax.servlet</artifactId> </exclusion> </exclusions> </dependency> <!-- hudi jetty--> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-server</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-util</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-webapp</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-http</artifactId> <version>${jetty.version}</version> </dependency> **_pom(hudi-spark-bundle_2.12) change section_** <!-- Hive --> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-service</artifactId> <version>${hive.version}</version> <scope>${spark.bundle.hive.scope}</scope> <exclusions> <exclusion> <artifactId>servlet-api</artifactId> <groupId>javax.servlet</groupId> </exclusion> <exclusion> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.pentaho</groupId> <artifactId>*</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-jdbc</artifactId> <version>${hive.version}</version> <scope>${spark.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-metastore</artifactId> <version>${hive.version}</version> <scope>${spark.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>javax.servlet</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>javax.servlet.jsp</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>com.goole.guava</groupId> <artifactId>guava</artifactId> </exclusion> <exclusion> <groupId>org.datanucleus</groupId> <artifactId>datanucleus-core</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>${hive.groupid}</groupId> <artifactId>hive-common</artifactId> <version>${hive.version}</version> <scope>${spark.bundle.hive.scope}</scope> <exclusions> <exclusion> <groupId>org.eclipse.jetty</groupId> <artifactId>*</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jetty.orbit</groupId> <artifactId>javax.servlet</artifactId> </exclusion> </exclusions> </dependency> <!-- hudi jetty--> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-server</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-util</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-webapp</artifactId> <version>${jetty.version}</version> </dependency> <dependency> <groupId>org.eclipse.jetty</groupId> <artifactId>jetty-http</artifactId> <version>${jetty.version}</version> </dependency> **_pom(hudi-hadoop-mr-bundle) change section_** <include>org.apache.hbase.thirdparty:hbase-unsafe</include>  **Stacktrace** ```Add the stacktrace of the error.``` 23/03/21 20:38:57 INFO HoodieBackedTableMetadata: Opened metadata base file from hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/files/files-0000_0-412-282_20230321154824057001.hfile at instant 20230321154824057001 in 806 ms 23/03/21 20:38:57 INFO HoodieActiveTimeline: Loaded instants upto : Option{val=[20230321203848506__rollback__COMPLETED]} 23/03/21 20:38:57 INFO HoodieTableMetaClient: Loading HoodieTableMetaClient from hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata 23/03/21 20:38:57 INFO HoodieTableConfig: Loading table properties from hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/.hoodie/hoodie.properties 23/03/21 20:38:57 INFO HoodieTableMetaClient: Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=HFILE) from hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata 23/03/21 20:39:03 INFO HoodieActiveTimeline: Loaded instants upto : Option{val=[20230321203848506__deltacommit__COMPLETED]} 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Scanning log file HoodieLogFile{pathStr='hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/files/.files-0000_20230321154824057001.log.1_0-419-289', fileLen=-1} 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Reading a data block from file hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/files/.files-0000_20230321154824057001.log.1_0-419-289 at instant 20230321154851882 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Scanning log file HoodieLogFile{pathStr='hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/files/.files-0000_20230321154824057001.log.1_0-419-289', fileLen=-1} 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Reading a data block from file hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/.hoodie/metadata/files/.files-0000_20230321154824057001.log.1_0-419-289 at instant 20230321154946837 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Merging the final data blocks 23/03/21 20:39:03 INFO AbstractHoodieLogRecordReader: Number of remaining logblocks to merge 2 23/03/21 20:39:03 ERROR AbstractHoodieLogRecordReader: Got exception when reading log file java.lang.NullPointerException at org.apache.hadoop.fs.FilterFileSystem.getConf(FilterFileSystem.java:524) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContext.<init>(ReaderContext.java:53) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContextBuilder.build(ReaderContextBuilder.java:94) at org.apache.hudi.io.storage.HoodieHFileUtils.createHFileReader(HoodieHFileUtils.java:83) at org.apache.hudi.io.storage.HoodieAvroHFileReader.<init>(HoodieAvroHFileReader.java:110) at org.apache.hudi.common.table.log.block.HoodieHFileDataBlock.deserializeRecords(HoodieHFileDataBlock.java:172) at org.apache.hudi.common.table.log.block.HoodieDataBlock.readRecordsFromBlockPayload(HoodieDataBlock.java:170) at org.apache.hudi.common.table.log.block.HoodieDataBlock.getRecordIterator(HoodieDataBlock.java:128) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.getRecordsIterator(AbstractHoodieLogRecordReader.java:805) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processDataBlock(AbstractHoodieLogRecordReader.java:630) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processQueuedBlocksForInstant(AbstractHoodieLogRecordReader.java:672) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternalV1(AbstractHoodieLogRecordReader.java:367) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternal(AbstractHoodieLogRecordReader.java:223) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.performScan(HoodieMergedLogRecordScanner.java:198) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:114) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:73) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:464) at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:218) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:546) at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:447) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getOrCreateReaders$13(HoodieBackedTableMetadata.java:430) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:430) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$3(HoodieBackedTableMetadata.java:239) at java.util.HashMap.forEach(HashMap.java:1289) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:237) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:152) at org.apache.hudi.metadata.BaseTableMetadata.fetchAllFilesInPartition(BaseTableMetadata.java:339) at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:150) at org.apache.hudi.metadata.HoodieMetadataFileSystemView.listPartition(HoodieMetadataFileSystemView.java:69) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.lambda$ensurePartitionLoadedCorrectly$15(AbstractTableFileSystemView.java:422) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.ensurePartitionLoadedCorrectly(AbstractTableFileSystemView.java:413) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.getLatestBaseFilesBeforeOrOn(AbstractTableFileSystemView.java:650) at org.apache.hudi.timeline.service.handlers.BaseFileHandler.getLatestDataFilesBeforeOrOn(BaseFileHandler.java:61) at org.apache.hudi.timeline.service.RequestHandler.lambda$registerDataFilesAPI$12(RequestHandler.java:273) at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:525) at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:20) at io.javalin.http.JavalinServlet.addHandler$lambda-0(JavalinServlet.kt:96) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:99) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981) at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:89) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:52) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.apache.hudi.org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.apache.hudi.org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181) at org.apache.hudi.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.hudi.org.eclipse.jetty.server.Server.handle(Server.java:516) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.apache.hudi.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.apache.hudi.org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.apache.hudi.org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.apache.hudi.org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:748) 23/03/21 20:39:03 ERROR RequestHandler: Got runtime exception servicing request partition=lisi&maxinstant=20230321154946837&basepath=hdfs%3A%2F%2Fbigdata.hadoop.master01%3A8020%2Ftest%2Fhudi2%2Ftest_aa1&lastinstantts=20230321203848506&timelinehash=15a7871bb524b70c567cbdf3cd28e6b61697c65b85fbfe6833eb1cd6293277fc org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve files in partition hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/lisi from metadata at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:152) at org.apache.hudi.metadata.HoodieMetadataFileSystemView.listPartition(HoodieMetadataFileSystemView.java:69) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.lambda$ensurePartitionLoadedCorrectly$15(AbstractTableFileSystemView.java:422) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.ensurePartitionLoadedCorrectly(AbstractTableFileSystemView.java:413) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.getLatestBaseFilesBeforeOrOn(AbstractTableFileSystemView.java:650) at org.apache.hudi.timeline.service.handlers.BaseFileHandler.getLatestDataFilesBeforeOrOn(BaseFileHandler.java:61) at org.apache.hudi.timeline.service.RequestHandler.lambda$registerDataFilesAPI$12(RequestHandler.java:273) at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:525) at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:20) at io.javalin.http.JavalinServlet.addHandler$lambda-0(JavalinServlet.kt:96) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:99) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981) at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:89) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:52) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.apache.hudi.org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.apache.hudi.org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181) at org.apache.hudi.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.hudi.org.eclipse.jetty.server.Server.handle(Server.java:516) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.apache.hudi.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.apache.hudi.org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.apache.hudi.org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.apache.hudi.org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hudi.exception.HoodieException: Exception when reading log file at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternalV1(AbstractHoodieLogRecordReader.java:376) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternal(AbstractHoodieLogRecordReader.java:223) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.performScan(HoodieMergedLogRecordScanner.java:198) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:114) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:73) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:464) at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:218) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:546) at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:447) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getOrCreateReaders$13(HoodieBackedTableMetadata.java:430) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:430) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$3(HoodieBackedTableMetadata.java:239) at java.util.HashMap.forEach(HashMap.java:1289) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:237) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:152) at org.apache.hudi.metadata.BaseTableMetadata.fetchAllFilesInPartition(BaseTableMetadata.java:339) at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:150) ... 46 more Caused by: java.lang.NullPointerException at org.apache.hadoop.fs.FilterFileSystem.getConf(FilterFileSystem.java:524) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContext.<init>(ReaderContext.java:53) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContextBuilder.build(ReaderContextBuilder.java:94) at org.apache.hudi.io.storage.HoodieHFileUtils.createHFileReader(HoodieHFileUtils.java:83) at org.apache.hudi.io.storage.HoodieAvroHFileReader.<init>(HoodieAvroHFileReader.java:110) at org.apache.hudi.common.table.log.block.HoodieHFileDataBlock.deserializeRecords(HoodieHFileDataBlock.java:172) at org.apache.hudi.common.table.log.block.HoodieDataBlock.readRecordsFromBlockPayload(HoodieDataBlock.java:170) at org.apache.hudi.common.table.log.block.HoodieDataBlock.getRecordIterator(HoodieDataBlock.java:128) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.getRecordsIterator(AbstractHoodieLogRecordReader.java:805) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processDataBlock(AbstractHoodieLogRecordReader.java:630) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processQueuedBlocksForInstant(AbstractHoodieLogRecordReader.java:672) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternalV1(AbstractHoodieLogRecordReader.java:367) ... 63 more 23/03/21 20:39:03 WARN Javalin: Uncaught exception org.apache.hudi.exception.HoodieMetadataException: Failed to retrieve files in partition hdfs://bigdata.hadoop.master01:8020/test/hudi2/test_aa1/lisi from metadata at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:152) at org.apache.hudi.metadata.HoodieMetadataFileSystemView.listPartition(HoodieMetadataFileSystemView.java:69) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.lambda$ensurePartitionLoadedCorrectly$15(AbstractTableFileSystemView.java:422) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.ensurePartitionLoadedCorrectly(AbstractTableFileSystemView.java:413) at org.apache.hudi.common.table.view.AbstractTableFileSystemView.getLatestBaseFilesBeforeOrOn(AbstractTableFileSystemView.java:650) at org.apache.hudi.timeline.service.handlers.BaseFileHandler.getLatestDataFilesBeforeOrOn(BaseFileHandler.java:61) at org.apache.hudi.timeline.service.RequestHandler.lambda$registerDataFilesAPI$12(RequestHandler.java:273) at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:525) at io.javalin.core.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:20) at io.javalin.http.JavalinServlet.addHandler$lambda-0(JavalinServlet.kt:96) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServlet$lifecycle$2$1$1.invoke(JavalinServlet.kt:43) at io.javalin.http.JavalinServletHandler.executeNextTask(JavalinServletHandler.kt:99) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$lambda-1(JavalinServletHandler.kt:85) at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981) at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124) at io.javalin.http.JavalinServletHandler.queueNextTaskOrFinish$javalin(JavalinServletHandler.kt:85) at io.javalin.http.JavalinServlet.service(JavalinServlet.kt:89) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58) at javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:52) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.apache.hudi.org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.apache.hudi.org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.apache.hudi.org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.apache.hudi.org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.apache.hudi.org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181) at org.apache.hudi.org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.apache.hudi.org.eclipse.jetty.server.Server.handle(Server.java:516) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.apache.hudi.org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.apache.hudi.org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.apache.hudi.org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.apache.hudi.org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.apache.hudi.org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.apache.hudi.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hudi.exception.HoodieException: Exception when reading log file at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternalV1(AbstractHoodieLogRecordReader.java:376) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternal(AbstractHoodieLogRecordReader.java:223) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.performScan(HoodieMergedLogRecordScanner.java:198) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:114) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:73) at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:464) at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:218) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:546) at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:447) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getOrCreateReaders$13(HoodieBackedTableMetadata.java:430) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:430) at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$3(HoodieBackedTableMetadata.java:239) at java.util.HashMap.forEach(HashMap.java:1289) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:237) at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:152) at org.apache.hudi.metadata.BaseTableMetadata.fetchAllFilesInPartition(BaseTableMetadata.java:339) at org.apache.hudi.metadata.BaseTableMetadata.getAllFilesInPartition(BaseTableMetadata.java:150) ... 46 more Caused by: java.lang.NullPointerException at org.apache.hadoop.fs.FilterFileSystem.getConf(FilterFileSystem.java:524) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContext.<init>(ReaderContext.java:53) at org.apache.hudi.org.apache.hadoop.hbase.io.hfile.ReaderContextBuilder.build(ReaderContextBuilder.java:94) at org.apache.hudi.io.storage.HoodieHFileUtils.createHFileReader(HoodieHFileUtils.java:83) at org.apache.hudi.io.storage.HoodieAvroHFileReader.<init>(HoodieAvroHFileReader.java:110) at org.apache.hudi.common.table.log.block.HoodieHFileDataBlock.deserializeRecords(HoodieHFileDataBlock.java:172) at org.apache.hudi.common.table.log.block.HoodieDataBlock.readRecordsFromBlockPayload(HoodieDataBlock.java:170) at org.apache.hudi.common.table.log.block.HoodieDataBlock.getRecordIterator(HoodieDataBlock.java:128) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.getRecordsIterator(AbstractHoodieLogRecordReader.java:805) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processDataBlock(AbstractHoodieLogRecordReader.java:630) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processQueuedBlocksForInstant(AbstractHoodieLogRecordReader.java:672) at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scanInternalV1(AbstractHoodieLogRecordReader.java:367) ... 63 more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
