[
https://issues.apache.org/jira/browse/HUDI-1151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1151.
--------------------------
Resolution: Fixed
Fixed via master branch: b51646dcc76acc68e97dd6a67cc7557e362b590d
> Fix NPE when no new data in kafka using HoodieDeltaStreamer
> -----------------------------------------------------------
>
> Key: HUDI-1151
> URL: https://issues.apache.org/jira/browse/HUDI-1151
> Project: Apache Hudi
> Issue Type: Bug
> Components: DeltaStreamer
> Reporter: wangxianghu
> Assignee: wangxianghu
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.6.0
>
>
> I get this NPE when using HoodieDeltaStreamer to etl kafka data to hudi
> table. the topic is newly created, and there is no data in the topic when the
> job is started
> {code:java}
> 20/08/05 21:18:00 INFO helpers.KafkaOffsetGen: SourceLimit not configured,
> set numEvents to default value : 5000000
> 20/08/05 21:18:00 INFO sources.JsonKafkaSource: About to read 0 from Kafka
> for topic :hudi_test_callback
> 20/08/05 21:18:00 INFO deltastreamer.DeltaSync: No new data, source
> checkpoint has not changed. Nothing to commit. Old
> checkpoint=(Option{val=hudi_test_callback,0:0}). New
> Checkpoint=(hudi_test_callback,0:0)
> 20/08/05 21:18:00 ERROR deltastreamer.HoodieDeltaStreamer: Shutting down
> delta-sync due to exception
> java.lang.NullPointerException
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:570)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 20/08/05 21:18:00 INFO deltastreamer.HoodieDeltaStreamer: Delta Sync
> shutdown. Error ?true
> 20/08/05 21:18:00 ERROR async.AbstractAsyncService: Monitor noticed one or
> more threads failed. Requesting graceful shutdown of other threads
> java.util.concurrent.ExecutionException:
> org.apache.hudi.exception.HoodieException
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hudi.async.AbstractAsyncService.lambda$monitorThreads$0(AbstractAsyncService.java:136)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hudi.exception.HoodieException
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:585)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> ... 3 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:570)
> ... 4 more
> 20/08/05 21:18:00 ERROR async.AbstractAsyncService: Service shutdown with
> error
> java.util.concurrent.ExecutionException:
> org.apache.hudi.exception.HoodieException
> at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
> at
> org.apache.hudi.async.AbstractAsyncService.waitForShutdown(AbstractAsyncService.java:72)
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.lambda$sync$1(HoodieDeltaStreamer.java:152)
> at org.apache.hudi.common.util.Option.ifPresent(Option.java:96)
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:149)
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:454)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:673)
> Caused by: org.apache.hudi.exception.HoodieException
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:585)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:570)
> ... 4 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
