[
https://issues.apache.org/jira/browse/NIFI-14415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dariusz Seweryn updated NIFI-14415:
-----------------------------------
Description:
On 2025-02-11, aws-kinesis-client-library-version was bumped [from 2.6.0 to
2.6.1 which transitively updated protobuf-java from 3.21.12 to
4.27.0|https://github.com/awslabs/amazon-kinesis-client/compare/v2.6.0...v2.6.1].
Along this bump protobuf-java dependency override was not removed, pinning to
version 3.25.6.
{quote}<!-- Override protobuf-java from amazon-kinesis-client -->
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.25.6</version>
</dependency>{quote}
As a result aggregated records are skipped with following warning:
{quote}runtime-server-1 | [APP] 2025-04-01 14:06:05,990 WARN
[ConsumeKinesisStream-Scheduler-runtime-server:bb1bafd3-cde7-4be6-a3a1-dc0a698baa10]
software.amazon.kinesis.lifecycle.ShardConsumer single_stream_mode : Exception
occurred while dispatching incoming data. The incoming data has been skipped
runtime-server-1 | java.lang.NoClassDefFoundError:
com/google/protobuf/RuntimeVersion$RuntimeDomain
runtime-server-1 | at
software.amazon.kinesis.retrieval.kpl.Messages$AggregatedRecord.<clinit>(Messages.java:1909)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:126)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:70)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.deaggregateAnyKplRecords(ProcessTask.java:204)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.call(ProcessTask.java:158)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.executeTask(ShardConsumer.java:412)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.processData(ShardConsumer.java:396)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.handleInput(ShardConsumer.java:201)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:181)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:36)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:56)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:27)
runtime-server-1 | at
io.reactivex.rxjava3.internal.util.HalfSerializer.onNext(HalfSerializer.java:46)
runtime-server-1 | at
io.reactivex.rxjava3.internal.subscribers.StrictSubscriber.onNext(StrictSubscriber.java:97)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:403)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
runtime-server-1 | at java.base/java.lang.Thread.run(Thread.java:1583)
runtime-server-1 | Caused by: java.lang.ClassNotFoundException:
com.google.protobuf.RuntimeVersion$RuntimeDomain
runtime-server-1 | at
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
runtime-server-1 | ... 22 common frames omitted
{quote}
was:
On 2025-02-11, aws-kinesis-client-library-version was bumped from 2.6.0 to
2.6.1 which transitively updated protobuf-java from 3.21.12 to 4.27.0. Along
this bump protobuf-java dependency override was not removed, pinning to version
3.25.6.
As a result aggregated records are skipped with following warning:
{quote}runtime-server-1 | [APP] 2025-04-01 14:06:05,990 WARN
[ConsumeKinesisStream-Scheduler-runtime-server:bb1bafd3-cde7-4be6-a3a1-dc0a698baa10]
software.amazon.kinesis.lifecycle.ShardConsumer single_stream_mode : Exception
occurred while dispatching incoming data. The incoming data has been skipped
runtime-server-1 | java.lang.NoClassDefFoundError:
com/google/protobuf/RuntimeVersion$RuntimeDomain
runtime-server-1 | at
software.amazon.kinesis.retrieval.kpl.Messages$AggregatedRecord.<clinit>(Messages.java:1909)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:126)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:70)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.deaggregateAnyKplRecords(ProcessTask.java:204)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.call(ProcessTask.java:158)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.executeTask(ShardConsumer.java:412)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.processData(ShardConsumer.java:396)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.handleInput(ShardConsumer.java:201)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:181)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:36)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:56)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:27)
runtime-server-1 | at
io.reactivex.rxjava3.internal.util.HalfSerializer.onNext(HalfSerializer.java:46)
runtime-server-1 | at
io.reactivex.rxjava3.internal.subscribers.StrictSubscriber.onNext(StrictSubscriber.java:97)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:403)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
runtime-server-1 | at java.base/java.lang.Thread.run(Thread.java:1583)
runtime-server-1 | Caused by: java.lang.ClassNotFoundException:
com.google.protobuf.RuntimeVersion$RuntimeDomain
runtime-server-1 | at
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
runtime-server-1 | ... 22 common frames omitted
{quote}
> ConsumeKinesisStream skips aggregated records
> ---------------------------------------------
>
> Key: NIFI-14415
> URL: https://issues.apache.org/jira/browse/NIFI-14415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.3.0
> Reporter: Dariusz Seweryn
> Assignee: Dariusz Seweryn
> Priority: Major
>
> On 2025-02-11, aws-kinesis-client-library-version was bumped [from 2.6.0 to
> 2.6.1 which transitively updated protobuf-java from 3.21.12 to
> 4.27.0|https://github.com/awslabs/amazon-kinesis-client/compare/v2.6.0...v2.6.1].
> Along this bump protobuf-java dependency override was not removed, pinning
> to version 3.25.6.
> {quote}<!-- Override protobuf-java from amazon-kinesis-client -->
> <dependency>
> <groupId>com.google.protobuf</groupId>
> <artifactId>protobuf-java</artifactId>
> <version>3.25.6</version>
> </dependency>{quote}
> As a result aggregated records are skipped with following warning:
> {quote}runtime-server-1 | [APP] 2025-04-01 14:06:05,990 WARN
> [ConsumeKinesisStream-Scheduler-runtime-server:bb1bafd3-cde7-4be6-a3a1-dc0a698baa10]
> software.amazon.kinesis.lifecycle.ShardConsumer single_stream_mode :
> Exception occurred while dispatching incoming data. The incoming data has
> been skipped
> runtime-server-1 | java.lang.NoClassDefFoundError:
> com/google/protobuf/RuntimeVersion$RuntimeDomain
> runtime-server-1 | at
> software.amazon.kinesis.retrieval.kpl.Messages$AggregatedRecord.<clinit>(Messages.java:1909)
> runtime-server-1 | at
> software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:126)
> runtime-server-1 | at
> software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:70)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ProcessTask.deaggregateAnyKplRecords(ProcessTask.java:204)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ProcessTask.call(ProcessTask.java:158)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ShardConsumer.executeTask(ShardConsumer.java:412)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ShardConsumer.processData(ShardConsumer.java:396)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ShardConsumer.handleInput(ShardConsumer.java:201)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:181)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:36)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:56)
> runtime-server-1 | at
> software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:27)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.util.HalfSerializer.onNext(HalfSerializer.java:46)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.subscribers.StrictSubscriber.onNext(StrictSubscriber.java:97)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:403)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
> runtime-server-1 | at
> io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
> runtime-server-1 | at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> runtime-server-1 | at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> runtime-server-1 | at java.base/java.lang.Thread.run(Thread.java:1583)
> runtime-server-1 | Caused by: java.lang.ClassNotFoundException:
> com.google.protobuf.RuntimeVersion$RuntimeDomain
> runtime-server-1 | at
> java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
> runtime-server-1 | at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
> runtime-server-1 | at
> java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
> runtime-server-1 | ... 22 common frames omitted
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
