In 1.15.1 it is working, in 1.15.3 it's not (don't know about 1.15.2). So there
must be a change between these versions and not only from 1.15.0
This is the stack trace:
2022-01-21 14:27:16,314 ERROR [Timer-Driven Process Thread-86]
org.apache.hadoop.hive.common.FileUtils The jar file path
file:///usr/hdp/current/phoenix-client/phoenix-hive.jar doesn't exist
2022-01-21 14:27:16,315 WARN [Timer-Driven Process Thread-86]
org.apache.hadoop.hive.conf.HiveConf HiveConf of name
hive.hook.proto.base-directory does not exist
2022-01-21 14:27:16,315 WARN [Timer-Driven Process Thread-86]
org.apache.hadoop.hive.conf.HiveConf HiveConf of name
hive.strict.managed.tables does not exist
2022-01-21 14:27:16,315 WARN [Timer-Driven Process Thread-86]
org.apache.hadoop.hive.conf.HiveConf HiveConf of name
hive.stats.fetch.partition.stats does not exist
2022-01-21 14:27:16,315 WARN [Timer-Driven Process Thread-86]
org.apache.hadoop.hive.conf.HiveConf HiveConf of name hive.heapsize does not
exist
2022-01-21 14:27:16,316 ERROR [Timer-Driven Process Thread-86]
o.a.n.processors.hive.PutHive3Streaming
PutHive3Streaming[id=1b2611dc-1176-1000-4de4-f1912c6e99dc] Failed to properly
initialize Processor. If still scheduled to run, NiFi will attempt to
initialize and run the Processor again after the 'Administrative Yield
Duration' has elapsed. Failure is due to java.lang.NoClassDefFoundError:
org/apache/hadoop/tracing/SpanReceiverHost: java.lang.NoClassDefFoundError:
org/apache/hadoop/tracing/SpanReceiverHost
java.lang.NoClassDefFoundError: org/apache/hadoop/tracing/SpanReceiverHost
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:634)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3354)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3403)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3371)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:477)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226)
at
org.apache.nifi.util.hive.HiveConfigurator.preload(HiveConfigurator.java:75)
at
org.apache.nifi.processors.hive.PutHive3Streaming.setup(PutHive3Streaming.java:372)
at sun.reflect.GeneratedMethodAccessor587.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
at
org.apache.nifi.controller.StandardProcessorNode.lambda$initiateStart$6(StandardProcessorNode.java:1653)
at org.apache.nifi.engine.FlowEngine$3.call(FlowEngine.java:123)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
-----Ursprüngliche Nachricht-----
Von: Bryan Bende [mailto:[email protected]]
Gesendet: Freitag, 21. Januar 2022 15:29
An: [email protected]
Betreff: Re: nifi 15.3 - puthive3streaming
This is a known issue starting with the 1.15.0 release.
It was introduced by this change:
https://issues.apache.org/jira/browse/NIFI-9248
The JIRA to correct the problem is:
https://issues.apache.org/jira/browse/NIFI-9392
On Fri, Jan 21, 2022 at 9:17 AM Joe Witt <[email protected]> wrote:
>
> Hello
>
> We need to see the full stack trace. Most likely involves log4j
>
> Thanks
>
> On Fri, Jan 21, 2022 at 7:14 AM Hengesbach, Martin
> <[email protected]> wrote:
>>
>> Hi Mike,
>>
>>
>>
>> I have downloaded the compiled version of nifi-hive3-nar-1.15.3.nar and have
>> the same problem as you. In my NiFi 15.3 installation I have replaced the
>> nifi-hive3-nar-1.15.3.nar through the nifi-hive3-nar-1.15.1.nar and it
>> worked again. So it must be a problem with the 15.3 version and not a
>> compilation problem on your side.
>>
>>
>>
>> Kind regards
>>
>> Martin
>>
>>
>>
>> Von: Michal Tomaszewski [mailto:[email protected]]
>> Gesendet: Freitag, 21. Januar 2022 13:30
>> An: [email protected]
>> Betreff: nifi 15.3 - puthive3streaming
>>
>>
>>
>> Hi again,
>>
>>
>>
>> After compiling 15.3 with hive3 using options:
>>
>> mvn -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
>> -Dhbase.version=2.0.2
>>
>>
>>
>> processor puthive3streaming is not working with error message:
>>
>> „java.lang.NoClassDefFoundError: org/apache/hadoop/tracing/SpanReceiverHost”
>>
>>
>>
>> Isn’t it related to this change:
>>
>> https://github.com/apache/nifi/commit/b7c81d6007852d933961d0067030bc807363e2ee
>>
>> ?
>>
>>
>>
>> What should we change to make puthive3streaming working?
>>
>>
>>
>> Regards,
>>
>> Mike
>>
>> ________________________________
>>
>> FIZ Karlsruhe - Leibniz-Institut für Informationsinfrastruktur GmbH.
>> Sitz der Gesellschaft: Eggenstein-Leopoldshafen, Amtsgericht Mannheim HRB
>> 101892.
>> Geschäftsführerin: Sabine Brünger-Weilandt.
>> Vorsitzender des Aufsichtsrats: MinR Dr. Hans-Josef Linkens.
>>
>> FIZ Karlsruhe ist zertifiziert mit dem Siegel "audit berufundfamilie".
------------------------------------------------------------------------------
FIZ Karlsruhe - Leibniz-Institut für Informationsinfrastruktur GmbH.
Sitz der Gesellschaft: Eggenstein-Leopoldshafen, Amtsgericht Mannheim HRB
101892.
Geschäftsführerin: Sabine Brünger-Weilandt.
Vorsitzender des Aufsichtsrats: MinR Dr. Hans-Josef Linkens.
FIZ Karlsruhe ist zertifiziert mit dem Siegel "audit berufundfamilie".
