[jira] [Updated] (NIFI-4579) When Strict Type Checking property is set to "false", ValidateRecord does not coerce fields into the correct type.
[ https://issues.apache.org/jira/browse/NIFI-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4579: --- Resolution: Fixed Status: Resolved (was: Patch Available) > When Strict Type Checking property is set to "false", ValidateRecord does not > coerce fields into the correct type. > -- > > Key: NIFI-4579 > URL: https://issues.apache.org/jira/browse/NIFI-4579 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website, Extensions >Affects Versions: 1.4.0 >Reporter: Andrew Lim >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.9.0 > > > The description of the Strict Type Checking property for the ValidateRecord > processor states: > _If false, the Record will be considered valid and the field will be coerced > into the correct type (if possible, according to the type coercion supported > by the Record Writer)._ > In my testing I've confirmed that in this scenario, the records are > considered valid. But, none of the record fields are coerced into the > correct type. > We should either correct the documentation or implement the promised coercion > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4579) When Strict Type Checking property is set to "false", ValidateRecord does not coerce fields into the correct type.
[ https://issues.apache.org/jira/browse/NIFI-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720631#comment-16720631 ] ASF GitHub Bot commented on NIFI-4579: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2794 > When Strict Type Checking property is set to "false", ValidateRecord does not > coerce fields into the correct type. > -- > > Key: NIFI-4579 > URL: https://issues.apache.org/jira/browse/NIFI-4579 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website, Extensions >Affects Versions: 1.4.0 >Reporter: Andrew Lim >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.9.0 > > > The description of the Strict Type Checking property for the ValidateRecord > processor states: > _If false, the Record will be considered valid and the field will be coerced > into the correct type (if possible, according to the type coercion supported > by the Record Writer)._ > In my testing I've confirmed that in this scenario, the records are > considered valid. But, none of the record fields are coerced into the > correct type. > We should either correct the documentation or implement the promised coercion > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2794: NIFI-4579: Fix ValidateRecord type coercing
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2794 ---
[jira] [Commented] (NIFI-4579) When Strict Type Checking property is set to "false", ValidateRecord does not coerce fields into the correct type.
[ https://issues.apache.org/jira/browse/NIFI-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720629#comment-16720629 ] ASF GitHub Bot commented on NIFI-4579: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2794 +1 LGTM, verified the unit test illustrates the behavior and the patch fixes it. Thanks for the fix! Merging to master > When Strict Type Checking property is set to "false", ValidateRecord does not > coerce fields into the correct type. > -- > > Key: NIFI-4579 > URL: https://issues.apache.org/jira/browse/NIFI-4579 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website, Extensions >Affects Versions: 1.4.0 >Reporter: Andrew Lim >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.9.0 > > > The description of the Strict Type Checking property for the ValidateRecord > processor states: > _If false, the Record will be considered valid and the field will be coerced > into the correct type (if possible, according to the type coercion supported > by the Record Writer)._ > In my testing I've confirmed that in this scenario, the records are > considered valid. But, none of the record fields are coerced into the > correct type. > We should either correct the documentation or implement the promised coercion > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4579) When Strict Type Checking property is set to "false", ValidateRecord does not coerce fields into the correct type.
[ https://issues.apache.org/jira/browse/NIFI-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4579: --- Fix Version/s: 1.9.0 > When Strict Type Checking property is set to "false", ValidateRecord does not > coerce fields into the correct type. > -- > > Key: NIFI-4579 > URL: https://issues.apache.org/jira/browse/NIFI-4579 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website, Extensions >Affects Versions: 1.4.0 >Reporter: Andrew Lim >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.9.0 > > > The description of the Strict Type Checking property for the ValidateRecord > processor states: > _If false, the Record will be considered valid and the field will be coerced > into the correct type (if possible, according to the type coercion supported > by the Record Writer)._ > In my testing I've confirmed that in this scenario, the records are > considered valid. But, none of the record fields are coerced into the > correct type. > We should either correct the documentation or implement the promised coercion > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4579) When Strict Type Checking property is set to "false", ValidateRecord does not coerce fields into the correct type.
[ https://issues.apache.org/jira/browse/NIFI-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720630#comment-16720630 ] ASF subversion and git services commented on NIFI-4579: --- Commit 0efddf47d516b62d7d9c61142d20ce40bcec675f in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=0efddf4 ] NIFI-4579: Fix ValidateRecord type coercing Signed-off-by: Matthew Burgess This closes #2794 > When Strict Type Checking property is set to "false", ValidateRecord does not > coerce fields into the correct type. > -- > > Key: NIFI-4579 > URL: https://issues.apache.org/jira/browse/NIFI-4579 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation Website, Extensions >Affects Versions: 1.4.0 >Reporter: Andrew Lim >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.9.0 > > > The description of the Strict Type Checking property for the ValidateRecord > processor states: > _If false, the Record will be considered valid and the field will be coerced > into the correct type (if possible, according to the type coercion supported > by the Record Writer)._ > In my testing I've confirmed that in this scenario, the records are > considered valid. But, none of the record fields are coerced into the > correct type. > We should either correct the documentation or implement the promised coercion > functionality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2794: NIFI-4579: Fix ValidateRecord type coercing
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2794 +1 LGTM, verified the unit test illustrates the behavior and the patch fixes it. Thanks for the fix! Merging to master ---
[jira] [Commented] (NIFI-3988) SplitRecord processor missing fragment attributes
[ https://issues.apache.org/jira/browse/NIFI-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720492#comment-16720492 ] ASF GitHub Bot commented on NIFI-3988: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/3217 NIFI-3988: Add fragment attributes to SplitRecord Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-3988 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3217.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3217 commit 1b95cd85fe623e6f981340dbfdcdc3c1034dbd5d Author: Matthew Burgess Date: 2018-12-13T19:02:14Z NIFI-3988: Add fragment attributes to SplitRecord > SplitRecord processor missing fragment attributes > - > > Key: NIFI-3988 > URL: https://issues.apache.org/jira/browse/NIFI-3988 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework > Environment: Fedora 25 >Reporter: Uwe Geercken >Assignee: Matt Burgess >Priority: Minor > Labels: attributes, processor, split > > There are several SplitXXX processors available. They all add some attributes > for the relevant fragment to the flowfile. Such as e.g. fragment.index, > fragment.count and fragment.identifier. > The SplitRecord processor does not add these attributes. > For a consistent user experience, and partly for compatibility with > MergeContent and other processors please add the relevant attributes to > SplitRecord. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-3988) SplitRecord processor missing fragment attributes
[ https://issues.apache.org/jira/browse/NIFI-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-3988: --- Affects Version/s: (was: 1.2.0) Status: Patch Available (was: In Progress) > SplitRecord processor missing fragment attributes > - > > Key: NIFI-3988 > URL: https://issues.apache.org/jira/browse/NIFI-3988 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework > Environment: Fedora 25 >Reporter: Uwe Geercken >Assignee: Matt Burgess >Priority: Minor > Labels: attributes, processor, split > > There are several SplitXXX processors available. They all add some attributes > for the relevant fragment to the flowfile. Such as e.g. fragment.index, > fragment.count and fragment.identifier. > The SplitRecord processor does not add these attributes. > For a consistent user experience, and partly for compatibility with > MergeContent and other processors please add the relevant attributes to > SplitRecord. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3217: NIFI-3988: Add fragment attributes to SplitRecord
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/3217 NIFI-3988: Add fragment attributes to SplitRecord Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-3988 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3217.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3217 commit 1b95cd85fe623e6f981340dbfdcdc3c1034dbd5d Author: Matthew Burgess Date: 2018-12-13T19:02:14Z NIFI-3988: Add fragment attributes to SplitRecord ---
[jira] [Commented] (NIFIREG-215) Extension Bundle Improvements
[ https://issues.apache.org/jira/browse/NIFIREG-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720471#comment-16720471 ] ASF GitHub Bot commented on NIFIREG-215: Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/149 > Extension Bundle Improvements > - > > Key: NIFIREG-215 > URL: https://issues.apache.org/jira/browse/NIFIREG-215 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Labels: Extension_Registry > Fix For: 0.4.0 > > > This ticket is to implement some additional improvements to the work done in > NIFIREG-211. > * Store file size of a bundle in the DB and make it available in retrieved > objects > * Provide a method to check if a bundle exists (true or false) > * Provide an option to allow uploading and overwriting an existing bundle > * Consider making snapshots a special version that can always be overridden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #149: NIFIREG-215 Extension Bundle Improvements
Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/149 Thanks, @bbende! +1, merged to master. ---
[jira] [Commented] (NIFIREG-215) Extension Bundle Improvements
[ https://issues.apache.org/jira/browse/NIFIREG-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720472#comment-16720472 ] ASF GitHub Bot commented on NIFIREG-215: Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/149 Thanks, @bbende! +1, merged to master. > Extension Bundle Improvements > - > > Key: NIFIREG-215 > URL: https://issues.apache.org/jira/browse/NIFIREG-215 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Labels: Extension_Registry > Fix For: 0.4.0 > > > This ticket is to implement some additional improvements to the work done in > NIFIREG-211. > * Store file size of a bundle in the DB and make it available in retrieved > objects > * Provide a method to check if a bundle exists (true or false) > * Provide an option to allow uploading and overwriting an existing bundle > * Consider making snapshots a special version that can always be overridden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #149: NIFIREG-215 Extension Bundle Improvements
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/149 ---
[jira] [Assigned] (NIFI-3988) SplitRecord processor missing fragment attributes
[ https://issues.apache.org/jira/browse/NIFI-3988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-3988: -- Assignee: Matt Burgess > SplitRecord processor missing fragment attributes > - > > Key: NIFI-3988 > URL: https://issues.apache.org/jira/browse/NIFI-3988 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 > Environment: Fedora 25 >Reporter: Uwe Geercken >Assignee: Matt Burgess >Priority: Minor > Labels: attributes, processor, split > > There are several SplitXXX processors available. They all add some attributes > for the relevant fragment to the flowfile. Such as e.g. fragment.index, > fragment.count and fragment.identifier. > The SplitRecord processor does not add these attributes. > For a consistent user experience, and partly for compatibility with > MergeContent and other processors please add the relevant attributes to > SplitRecord. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720295#comment-16720295
]
ASF subversion and git services commented on NIFI-5891:
---
Commit c51512f5e33cbd413b1fda8700408aa95614680e in nifi's branch
refs/heads/master from gkkorir
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c51512f ]
NIFI-5891 fix handling of null logical types in Hive3Streaming processor
NIFI-5891: Fixed Checkstyle issues
Signed-off-by: Matthew Burgess
This closes #3216
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720296#comment-16720296
]
ASF GitHub Bot commented on NIFI-5891:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3216
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
> Fix For: 1.9.0
>
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5891:
---
Resolution: Fixed
Fix Version/s: 1.9.0
Status: Resolved (was: Patch Available)
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
> Fix For: 1.9.0
>
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3216: NIFI-5891 fix handling of null logical types in Hiv...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3216 ---
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720294#comment-16720294
]
ASF subversion and git services commented on NIFI-5891:
---
Commit c51512f5e33cbd413b1fda8700408aa95614680e in nifi's branch
refs/heads/master from gkkorir
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c51512f ]
NIFI-5891 fix handling of null logical types in Hive3Streaming processor
NIFI-5891: Fixed Checkstyle issues
Signed-off-by: Matthew Burgess
This closes #3216
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #3216: NIFI-5891 fix handling of null logical types in Hive3Strea...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3216 +1 LGTM, verified the unit tests illustrate the problem and the patch fixes it. There are 2 CheckStyle violations but I will fix them on merge. Thanks for the fix! Merging to master ---
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720292#comment-16720292
]
ASF GitHub Bot commented on NIFI-5891:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/3216
+1 LGTM, verified the unit tests illustrate the problem and the patch fixes
it. There are 2 CheckStyle violations but I will fix them on merge. Thanks for
the fix! Merging to master
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5891:
---
Status: Patch Available (was: Open)
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5891:
---
Affects Version/s: (was: 1.8.0)
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFIREG-215) Extension Bundle Improvements
[ https://issues.apache.org/jira/browse/NIFIREG-215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720274#comment-16720274 ] ASF GitHub Bot commented on NIFIREG-215: Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/149 @kevdoran I pushed up a commit to update the filter params documentation. I decided not to do anything with the lastModified value on the bundle table because technically nothing on that row is ever changing, it is really the entries in the version table which are changing, and this will have new create dates when redeploying any version. Let me know if there is anything else. > Extension Bundle Improvements > - > > Key: NIFIREG-215 > URL: https://issues.apache.org/jira/browse/NIFIREG-215 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Labels: Extension_Registry > Fix For: 0.4.0 > > > This ticket is to implement some additional improvements to the work done in > NIFIREG-211. > * Store file size of a bundle in the DB and make it available in retrieved > objects > * Provide a method to check if a bundle exists (true or false) > * Provide an option to allow uploading and overwriting an existing bundle > * Consider making snapshots a special version that can always be overridden -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #149: NIFIREG-215 Extension Bundle Improvements
Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/149 @kevdoran I pushed up a commit to update the filter params documentation. I decided not to do anything with the lastModified value on the bundle table because technically nothing on that row is ever changing, it is really the entries in the version table which are changing, and this will have new create dates when redeploying any version. Let me know if there is anything else. ---
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720227#comment-16720227
]
Gideon Korir commented on NIFI-5891:
This issue seems to affect all logical avro types: decimal, date, timestamp,
arrays
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3216: NIFI-5891 fix handling of null logical types in Hiv...
GitHub user gideonkorir opened a pull request: https://github.com/apache/nifi/pull/3216 NIFI-5891 fix handling of null logical types in Hive3Streaming processor Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gideonkorir/nifi NIFI-5891 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3216.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3216 commit 61a5e9507ac2def912b1a954ce0708a83a49c2db Author: gkkorir Date: 2018-12-13T14:25:37Z NIFI-5891 fix handling of null logical types in Hive3Streaming processor ---
[jira] [Commented] (NIFI-5891) NiFi record serde throws NPE when decimal field has null value
[
https://issues.apache.org/jira/browse/NIFI-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720220#comment-16720220
]
ASF GitHub Bot commented on NIFI-5891:
--
GitHub user gideonkorir opened a pull request:
https://github.com/apache/nifi/pull/3216
NIFI-5891 fix handling of null logical types in Hive3Streaming processor
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gideonkorir/nifi NIFI-5891
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3216.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3216
commit 61a5e9507ac2def912b1a954ce0708a83a49c2db
Author: gkkorir
Date: 2018-12-13T14:25:37Z
NIFI-5891 fix handling of null logical types in Hive3Streaming processor
> NiFi record serde throws NPE when decimal field has null value
> --
>
> Key: NIFI-5891
> URL: https://issues.apache.org/jira/browse/NIFI-5891
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.8.0
> Environment: RHEL 7.5
> Open JDK 1.8
>Reporter: Gideon Korir
>Priority: Major
>
> When an incoming record has a null decimal field, NiFiRecordSerDe (
> org.apache.hive.streaming) throws an NPE when trying to create a HiveDecimal.
> This causes the PutHive3Streaming processor to skip the record and log a
> warning.
>
> The lines:
> {code:java}
> val = HiveDecimal.create(record.getAsDouble(fieldName));
> {code}
> should be changed to something along the lines of:
> {code:java}
> Double d = record.getAsDouble(fieldName);
> val = d == null ? null : HiveRecord.create(d);{code}
> The bug is insidious and occurs during Auto-unboxing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-68) Bluetooth Low Emission Scanning
[ https://issues.apache.org/jira/browse/MINIFICPP-68?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720150#comment-16720150 ] Mr TheSegfault commented on MINIFICPP-68: - [~aboda] Feel free to take a look at this if you have a chance. IMO nanofi stuff has a greater priority but this would be pretty darn cool to see. > Bluetooth Low Emission Scanning > --- > > Key: MINIFICPP-68 > URL: https://issues.apache.org/jira/browse/MINIFICPP-68 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Arpad Boda >Priority: Major > > Provide the capability for minifi c++ to use a local host device bluetooth > 4.0 radio to scan nearby bluetooth low emission devices. This implementation > will blindly report back the information presented by all local BLE devices > and will report all information that is provided by the underlying linux > bluez library which is used for the implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (MINIFICPP-696) FlowIDs aren't persisted
Mr TheSegfault created MINIFICPP-696: Summary: FlowIDs aren't persisted Key: MINIFICPP-696 URL: https://issues.apache.org/jira/browse/MINIFICPP-696 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Mr TheSegfault This isn't an inherent "problem," but we can save a round trip by storing the flow ID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (MINIFICPP-68) Bluetooth Low Emission Scanning
[ https://issues.apache.org/jira/browse/MINIFICPP-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mr TheSegfault reassigned MINIFICPP-68: --- Assignee: Arpad Boda (was: Jeremy Dyer) > Bluetooth Low Emission Scanning > --- > > Key: MINIFICPP-68 > URL: https://issues.apache.org/jira/browse/MINIFICPP-68 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Arpad Boda >Priority: Major > > Provide the capability for minifi c++ to use a local host device bluetooth > 4.0 radio to scan nearby bluetooth low emission devices. This implementation > will blindly report back the information presented by all local BLE devices > and will report all information that is provided by the underlying linux > bluez library which is used for the implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5894) FTPTransfer NullPointerException - FTPFile.getTimestamp() returns null
henning ottesen created NIFI-5894: - Summary: FTPTransfer NullPointerException - FTPFile.getTimestamp() returns null Key: NIFI-5894 URL: https://issues.apache.org/jira/browse/NIFI-5894 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.8.0 Reporter: henning ottesen 2018-12-13 10:34:07,405 WARN [Timer-Driven Process Thread-2] o.a.n.controller.tasks.ConnectableTask Administratively Yielding ListFTP[id=a6cee9c8-0167-1000-2bc2-5e69d2e33af5] due to uncaught Exception: java.lang.NullPointerException java.lang.NullPointerException: null at org.apache.nifi.processors.standard.util.FTPTransfer.newFileInfo(FTPTransfer.java:305) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:270) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:260) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:191) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:471) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:413) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (NIFI-5888) QueryRecord processor handling timestamp
[
https://issues.apache.org/jira/browse/NIFI-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura updated NIFI-5888:
Comment: was deleted
(was: I believe this issue is caused by CALCITE-1703. I've submitted a PR to
Calcite project. Let's see how it goes.)
> QueryRecord processor handling timestamp
>
>
> Key: NIFI-5888
> URL: https://issues.apache.org/jira/browse/NIFI-5888
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Gideon Korir
>Assignee: Koji Kawamura
>Priority: Major
>
> Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
>
> {code:java}
> @Test public void testTimestampColumns() throws InitializationException {
> final MockRecordParser parser = new MockRecordParser();
> parser.addSchemaField("name", RecordFieldType.STRING);
> parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP);
> parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00"));
> parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00"));
> parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00"));
> final List colNames = new ArrayList<>();
> colNames.add("name");
> colNames.add("day_added");
> final ResultSetValidatingRecordWriter writer = new
> ResultSetValidatingRecordWriter(colNames);
> TestRunner runner = getRunner();
> runner.addControllerService("parser", parser);
> runner.enableControllerService(parser);
> runner.addControllerService("writer", writer);
> runner.enableControllerService(writer);
> runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added
> from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY,
> "parser");
> runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
> runner.enqueue("");
> runner.run();
> runner.assertTransferCount(REL_NAME, 1);
> }
> {code}
>
> *Expected*: Test will pass
> *Actual*: fails with ClassCastException with message "java.sql.Timestamp
> cannot be cast to java.lang.Long".
> //Also used sql: "select name, CAST(date_added, DATE) as day_added from
> FLOWFILE" and failed
>
> I (think) I've traced the issue to the NiFi usage of Apache Calcite;
> according to Julian Hyde's comment on [this
> issue|https://issues.apache.org/jira/browse/CALCITE-1427].
> Disclaimer: My knowledge on Calcite is zero to none so I could be completely
> wrong.
> Currently, QueryRecord uses FlowFileTable which only implements
> QueryableTable and TranslateableTable and therefore can not work with
> java.sql.Timestamp (at least based on Hyde's comment on the issue). This will
> mean almost all RecordReader services (Avro, Csv) would not work with
> QueryRecord.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5888) QueryRecord processor handling timestamp
[
https://issues.apache.org/jira/browse/NIFI-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719932#comment-16719932
]
Koji Kawamura commented on NIFI-5888:
-
I believe this issue is caused by CALCITE-1703. I've submitted a PR to Calcite
project. Let's see how it goes.
> QueryRecord processor handling timestamp
>
>
> Key: NIFI-5888
> URL: https://issues.apache.org/jira/browse/NIFI-5888
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Gideon Korir
>Assignee: Koji Kawamura
>Priority: Major
>
> Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
>
> {code:java}
> @Test public void testTimestampColumns() throws InitializationException {
> final MockRecordParser parser = new MockRecordParser();
> parser.addSchemaField("name", RecordFieldType.STRING);
> parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP);
> parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00"));
> parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00"));
> parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00"));
> final List colNames = new ArrayList<>();
> colNames.add("name");
> colNames.add("day_added");
> final ResultSetValidatingRecordWriter writer = new
> ResultSetValidatingRecordWriter(colNames);
> TestRunner runner = getRunner();
> runner.addControllerService("parser", parser);
> runner.enableControllerService(parser);
> runner.addControllerService("writer", writer);
> runner.enableControllerService(writer);
> runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added
> from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY,
> "parser");
> runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
> runner.enqueue("");
> runner.run();
> runner.assertTransferCount(REL_NAME, 1);
> }
> {code}
>
> *Expected*: Test will pass
> *Actual*: fails with ClassCastException with message "java.sql.Timestamp
> cannot be cast to java.lang.Long".
> //Also used sql: "select name, CAST(date_added, DATE) as day_added from
> FLOWFILE" and failed
>
> I (think) I've traced the issue to the NiFi usage of Apache Calcite;
> according to Julian Hyde's comment on [this
> issue|https://issues.apache.org/jira/browse/CALCITE-1427].
> Disclaimer: My knowledge on Calcite is zero to none so I could be completely
> wrong.
> Currently, QueryRecord uses FlowFileTable which only implements
> QueryableTable and TranslateableTable and therefore can not work with
> java.sql.Timestamp (at least based on Hyde's comment on the issue). This will
> mean almost all RecordReader services (Avro, Csv) would not work with
> QueryRecord.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5888) QueryRecord processor handling timestamp
[
https://issues.apache.org/jira/browse/NIFI-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719931#comment-16719931
]
Koji Kawamura commented on NIFI-5888:
-
I believe this issue is caused by CALCITE-1703. I've submitted a PR to Calcite
project. Let's see how it goes.
> QueryRecord processor handling timestamp
>
>
> Key: NIFI-5888
> URL: https://issues.apache.org/jira/browse/NIFI-5888
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.8.0
>Reporter: Gideon Korir
>Assignee: Koji Kawamura
>Priority: Major
>
> Add the test below to `org.apache.nifi.processors.standard.TestQueryRecord`
>
> {code:java}
> @Test public void testTimestampColumns() throws InitializationException {
> final MockRecordParser parser = new MockRecordParser();
> parser.addSchemaField("name", RecordFieldType.STRING);
> parser.addSchemaField("date_added", RecordFieldType.TIMESTAMP);
> parser.addRecord("Tom", Timestamp.valueOf("2018-12-03 09:12:00"));
> parser.addRecord("Jerry", Timestamp.valueOf("2018-12-04 10:26:00"));
> parser.addRecord("Tom", Timestamp.valueOf("2017-01-03 11:22:00"));
> final List colNames = new ArrayList<>();
> colNames.add("name");
> colNames.add("day_added");
> final ResultSetValidatingRecordWriter writer = new
> ResultSetValidatingRecordWriter(colNames);
> TestRunner runner = getRunner();
> runner.addControllerService("parser", parser);
> runner.enableControllerService(parser);
> runner.addControllerService("writer", writer);
> runner.enableControllerService(writer);
> runner.setProperty(REL_NAME, "select name, {fn YEAR(date_added)} as day_added
> from FLOWFILE"); runner.setProperty(QueryRecord.RECORD_READER_FACTORY,
> "parser");
> runner.setProperty(QueryRecord.RECORD_WRITER_FACTORY, "writer");
> runner.enqueue("");
> runner.run();
> runner.assertTransferCount(REL_NAME, 1);
> }
> {code}
>
> *Expected*: Test will pass
> *Actual*: fails with ClassCastException with message "java.sql.Timestamp
> cannot be cast to java.lang.Long".
> //Also used sql: "select name, CAST(date_added, DATE) as day_added from
> FLOWFILE" and failed
>
> I (think) I've traced the issue to the NiFi usage of Apache Calcite;
> according to Julian Hyde's comment on [this
> issue|https://issues.apache.org/jira/browse/CALCITE-1427].
> Disclaimer: My knowledge on Calcite is zero to none so I could be completely
> wrong.
> Currently, QueryRecord uses FlowFileTable which only implements
> QueryableTable and TranslateableTable and therefore can not work with
> java.sql.Timestamp (at least based on Hyde's comment on the issue). This will
> mean almost all RecordReader services (Avro, Csv) would not work with
> QueryRecord.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
