[
https://issues.apache.org/jira/browse/NIFI-7817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Denissov updated NIFI-7817:
-------------------------------------
Description:
We have a customer Processor that uses ParquetReader to parse incoming Flow
File and then send records to the target system. It worked before, but broke
with NiFi 1.12.0 release.
The exception stack trace:
{{{{Caused by: java.lang.NullPointerException: Name is null}}}}
{{ at java.lang.Enum.valueOf(Enum.java:236)}}
{{ at
org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
{{ at
org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
{{ at
org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
{{ at
org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
{{ at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
{{ at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
{{ at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
{{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
{{ at
io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
{{ at
org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
The actual problem occurs here:
[https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
where
{{final String compressionTypeValue =
context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
comes back with the value of null, since "compression-type" property is not
exposed on ParquetReader and would not be set by the flow designers. The
returned null value is then passed to get the enum instance and fails there.
{{final CompressionCodecName codecName =
CompressionCodecName.valueOf(compressionTypeValue);}}
While there might be several solutions to this, including updating
parquet-specific defaulting logic, I traced the root cause of this regression
to the fix for NIFI-7635, to this commit:
[https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
where the default value of provided property descriptor, expressed previously
as
property.getDefaultValue()
is no longer used. That value used to contain UNCOMPRESSED value for the use
case in question and it used to work before this commit.
I'd think the issue needs to get fixed in this place as it might affect a
variety of other use cases.
was:
We have a customer Processor that uses ParquetReader to parse incoming Flow
File and then send records to the target system. It worked before, but broke
with NiFi 1.12.0 release.
The exception stack trace:
```
{{Caused by: java.lang.NullPointerException: Name is null}}
{{ at java.lang.Enum.valueOf(Enum.java:236)}}
{{ at
org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
{{ at
org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
{{ at
org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
{{ at
org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
{{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
{{ at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
{{ at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
{{ at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
{{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
{{ at
io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
{{ at
org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
```
Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
The actual problem occurs here:
[https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
where
{{final String compressionTypeValue =
context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
comes back with the value of null, since "compression-type" property is not
exposed on ParquetReader and would not be set by the flow designers. The
returned null value is then passed to get the enum instance and fails there.
{{final CompressionCodecName codecName =
CompressionCodecName.valueOf(compressionTypeValue);}}
While there might be several solutions to this, including updating
parquet-specific defaulting logic, I traced the root cause of this regression
to the fix for NIFI-7635, to this commit:
[https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
where the default value of provided property descriptor, expressed previously
as
property.getDefaultValue()
is no longer used. That value used to contain UNCOMPRESSED value for the use
case in question and it used to work before this commit.
I'd think the issue needs to get fixed in this place as it might affect a
variety of other use cases.
> ParquetReader fails to instantiate due to missing default value for
> compression-type property
> ---------------------------------------------------------------------------------------------
>
> Key: NIFI-7817
> URL: https://issues.apache.org/jira/browse/NIFI-7817
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.12.0
> Reporter: Alexander Denissov
> Priority: Major
>
> We have a customer Processor that uses ParquetReader to parse incoming Flow
> File and then send records to the target system. It worked before, but broke
> with NiFi 1.12.0 release.
> The exception stack trace:
> {{{{Caused by: java.lang.NullPointerException: Name is null}}}}
> {{ at java.lang.Enum.valueOf(Enum.java:236)}}
> {{ at
> org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)}}
> {{ at
> org.apache.nifi.parquet.utils.ParquetUtils.createParquetConfig(ParquetUtils.java:172)}}
> {{ at
> org.apache.nifi.parquet.ParquetReader.createRecordReader(ParquetReader.java:48)}}
> {{ at
> org.apache.nifi.serialization.RecordReaderFactory.createRecordReader(RecordReaderFactory.java:49)}}
> {{ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)}}
> {{ at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)}}
> {{ at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
> {{ at java.lang.reflect.Method.invoke(Method.java:498)}}
> {{ at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)}}
> {{ at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)}}
> {{ at com.sun.proxy.$Proxy100.createRecordReader(Unknown Source)}}
> {{ at
> io.pivotal.greenplum.nifi.processors.PutGreenplumRecord.lambda$null$1(PutGreenplumRecord.java:300)}}
> {{ at
> org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:127)}}
>
> Basically, the creation of a ParquetReader by the RecordReaderFactory fails.
> The actual problem occurs here:
> [https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/main/java/org/apache/nifi/parquet/utils/ParquetUtils.java#L171-L172]
> where
> {{final String compressionTypeValue =
> context.getProperty(ParquetUtils.COMPRESSION_TYPE).getValue();}}
> comes back with the value of null, since "compression-type" property is not
> exposed on ParquetReader and would not be set by the flow designers. The
> returned null value is then passed to get the enum instance and fails there.
> {{final CompressionCodecName codecName =
> CompressionCodecName.valueOf(compressionTypeValue);}}
> While there might be several solutions to this, including updating
> parquet-specific defaulting logic, I traced the root cause of this regression
> to the fix for NIFI-7635, to this commit:
> [https://github.com/apache/nifi/commit/4f11e3626093d3090f97c0efc5e229d83b6006e4#diff-782335ecee68f6939c3724dba3983d3d]
> where the default value of provided property descriptor, expressed previously
> as
> property.getDefaultValue()
> is no longer used. That value used to contain UNCOMPRESSED value for the use
> case in question and it used to work before this commit.
> I'd think the issue needs to get fixed in this place as it might affect a
> variety of other use cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
