[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470588#comment-16470588
]
ASF subversion and git services commented on NIFI-4395:
---
Commit fd8f1bf9083d2baf4d43e0666c774a8d5f83e2d8 in nifi's branch
refs/heads/HDF-3.1-maint from [~Deon]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=fd8f1bf ]
NIFI-4395 GenerateTableFetch can't fetch column type by state after instance
reboot
Add multiple states recover
Signed-off-by: Matthew Burgess
This closes #2442
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350597#comment-16350597
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2442
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350595#comment-16350595
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/2442
+1 LGTM, ran the unit tests and tried on a live NiFi instance, verified the
issue with the "underpopulated" column type map is resolved. Thanks for the
fix! Merging to master
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350596#comment-16350596
]
ASF subversion and git services commented on NIFI-4395:
---
Commit b5938062a8ff0d7c8bb2c1985dc2605b2a69b6dd in nifi's branch
refs/heads/master from [~Deon]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b593806 ]
NIFI-4395 GenerateTableFetch can't fetch column type by state after instance
reboot
Add multiple states recover
Signed-off-by: Matthew Burgess
This closes #2442
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343548#comment-16343548
]
ASF GitHub Bot commented on NIFI-4395:
--
GitHub user yjhyjhyjh0 opened a pull request:
https://github.com/apache/nifi/pull/2442
NIFI-4395 GenerateTableFetch can't fetch column type by state after i…
…nstance reboot
Add multiple states recover
Add support for multiple state cache recover after instance reboot.
Current fix by add condition for column type not found to re-cache.
Also add unit test for multiple state recover situation.
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yjhyjhyjh0/nifi NIFI-4395
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2442.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2442
commit 4a3a212d218cf524f00f01d18f4e9d9fdf9ed403
Author: Deon Huang
Date: 2018-01-29T15:29:21Z
NIFI-4395 GenerateTableFetch can't fetch column type by state after
instance reboot
Add multiple states recover
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339346#comment-16339346
]
Sanjay Saini commented on NIFI-4395:
Yes, I think you should reopen the issue because the fix is partial.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338704#comment-16338704
]
Deon Huang commented on NIFI-4395:
--
Hi Sanjay and Sarthak,
Really appreciate for the elaboration.
Indeed, current fix version only handle single table scenario (unit test same
only test single state recover).
Original condition logic didn't handle well at multi-table state recover after
instance reboot problem.
Seems it's a single-line fixed bug.
I'll consider submit a pull request here to reopen the issue rather than open
another issue.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332218#comment-16332218
]
Sarthak Dubey commented on NIFI-4395:
-
Hello Deon Huang,
On reference to this issue
https://issues.apache.org/jira/browse/NIFI-4395?focusedCommentId=16330518&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16330518
There is a GenerateTableFetch processor which has states stored for multiple
tables. When the flow is run after reboot one of the table which has the stored
state gets through however the rest of the tables stay in the queue throwing
the exception (reference above comment link).
In our analysis,when we reboot the instance and run the flow the map
columnTypeMap on line number 419 which is used to get the value of "type" in
org.apache.nifi.processors.standard.GenerateTableFetch.getColumnType(GenerateTableFetch.java:423)
method contains null so it throws an exception and the flowfile gets stuck
there and for every other flowfile with respect to other remaining tables the
code(GenerateTableFetch.getColumnType method) tries to get "type" from the same
map and it gets "type" as null and so it throws an Exception.
But when i return 93(column type expected is timestamp) commenting out the
exception (changing code in GenerateTableFetch.getColumnType) , map gets
updated with the other entries as well and the flow starts running as it should.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331846#comment-16331846
]
Sanjay Saini commented on NIFI-4395:
Sorry I meant 1.4.0.
Will send more details shortly.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331705#comment-16331705
]
Deon Huang commented on NIFI-4395:
--
Hi Sanjay,
Thanks for your report.
I’ve test again with NiFi 1.4.0, 1.5.0 (Recent release) successfully.
Testing with MSSQL as source database.
Also I can’t see 1.4.2 release at
https://www.apache.org/dyn/closer.lua?path=/nifi/
Could you provide more detail or context about your comment?
That will help a lot for reproducing the error.
Thanks.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16330518#comment-16330518
]
Sanjay Saini commented on NIFI-4395:
This error isn't resolved. We are using v1.4.2
Error log:
2018-01-18 12:25:52,876 ERROR [Timer-Driven Process Thread-9]
o.a.n.p.standard.GenerateTableFetch
GenerateTableFetch[id=9603c9ff-b832-3d14-dc01-a25021fa745a] Error during
processing: No column type cache found for: RecordModifiedTime:
org.apache.nifi.processor.exception.ProcessException: No column type cache
found for: RecordModifiedTime
org.apache.nifi.processor.exception.ProcessException: No column type cache
found for: RecordModifiedTime
at
org.apache.nifi.processors.standard.GenerateTableFetch.getColumnType(GenerateTableFetch.java:423)
at
org.apache.nifi.processors.standard.GenerateTableFetch.lambda$onTrigger$0(GenerateTableFetch.java:251)
at
java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
at
org.apache.nifi.processors.standard.GenerateTableFetch.onTrigger(GenerateTableFetch.java:241)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-01-18 12:25:53,837 ERROR [Timer-Driven Proces
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
>Priority: Major
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176904#comment-16176904
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2166
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176901#comment-16176901
]
ASF subversion and git services commented on NIFI-4395:
---
Commit a29348f2a4d8ec6576f1ec73c57911b827d46315 in nifi's branch
refs/heads/master from [~Deon]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a29348f ]
NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance
reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess
This closes #2166
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176903#comment-16176903
]
ASF subversion and git services commented on NIFI-4395:
---
Commit a29348f2a4d8ec6576f1ec73c57911b827d46315 in nifi's branch
refs/heads/master from [~Deon]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a29348f ]
NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance
reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess
This closes #2166
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176900#comment-16176900
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/2166
The unit test doesn't pass when this PR is rebased against master, due to
[NIFI-3484](https://issues.apache.org/jira/browse/NIFI-3484). I can fix this
when I merge.
+1 LGTM, verified changes incorporated from @ijokarumawak 's review, built
and ran (altered) unit tests, and tried with MySQL using multiple tables. Thank
you for this fix! Merging to master.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175716#comment-16175716
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user yjhyjhyjh0 commented on the issue:
https://github.com/apache/nifi/pull/2166
Thanks for the detail comment and suggestion.
That helps me a lot.
Some fix by suggestion like.
1 - put back StringmaxValueColumnNames to GenerateTableFetch to avoid NPE
unit test fail.
(Originally forget to include this line)
2 - move re-cache method, calling super.setup(), before the getColumnType
to provide more
readable code.
3 - Remove ternary operator because evaluateAttributeExpressions already
can handle null flowfile.
Thanks
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175487#comment-16175487
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user paulgibeault commented on the issue:
https://github.com/apache/nifi/pull/2166
@ijokarumawak Thank you for the quick review. We will get right on these
changes and resubmit.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175429#comment-16175429
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2166#discussion_r140349370
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
---
@@ -221,8 +222,12 @@ protected PropertyDescriptor
getSupportedDynamicPropertyDescriptor(final String
return super.customValidate(validationContext);
}
-public void setup(final ProcessContext context) {
-final String maxValueColumnNames =
context.getProperty(MAX_VALUE_COLUMN_NAMES).evaluateAttributeExpressions().getValue();
+public void setup(final ProcessContext context){
+setup(context,true,null);
+}
+
+public void setup(final ProcessContext context, boolean
shouldCleanCache,FlowFile flowFile) {
+final String maxValueColumnNames = (flowFile == null) ?
context.getProperty(MAX_VALUE_COLUMN_NAMES).evaluateAttributeExpressions().getValue()
:
context.getProperty(MAX_VALUE_COLUMN_NAMES).evaluateAttributeExpressions(flowFile).getValue();
--- End diff --
I like the idea of being defensive, but flowFile can be null with
`evaluateAttributeExpressions`, so we don't need this check using ternary
operator. Just passing the flowFile (possibly null) is fine, as
GenerateTableFetch [existing
code](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java#L185)
does.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175430#comment-16175430
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2166#discussion_r140352593
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -243,14 +243,14 @@ public void onTrigger(final ProcessContext context,
final ProcessSessionFactory
maxValueSelectColumns.add("MAX(" + colName + ") " +
colName);
String maxValue = getColumnStateMaxValue(tableName,
statePropertyMap, colName);
if (!StringUtils.isEmpty(maxValue)) {
-Integer type = getColumnType(tableName, colName);
+Integer type = getColumnType(context, tableName,
colName, finalFileToProcess);
--- End diff --
Probably instead of change `getColumnType`, we can add calling `setup` if
`columnTypeMap.isEmpty()`, before `getColumnType` here. Which makes it more
readable.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175389#comment-16175389
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2166#discussion_r140346314
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
---
@@ -245,7 +250,9 @@ public void setup(final ProcessContext context) {
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
int numCols = resultSetMetaData.getColumnCount();
if (numCols > 0) {
+if (shouldCleanCache){
columnTypeMap.clear();
+}
--- End diff --
Need indentation at the line of `columnTypeMap.clear()`.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175388#comment-16175388
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2166#discussion_r140344171
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -153,13 +153,11 @@ public GenerateTableFetch() {
@Override
@OnScheduled
-public void setup(final ProcessContext context) {
-maxValueProperties =
getDefaultMaxValueProperties(context.getProperties());
--- End diff --
We should keep this line to initialize `maxValueProperties`. I got
following NPE when I restarted NiFi. Unit tests failed, too:
```
2017-09-22 04:55:35,898 WARN [Timer-Driven Process Thread-1]
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.NullPointerException: null
at
org.apache.nifi.processors.standard.GenerateTableFetch.onTrigger(GenerateTableFetch.java:208)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
```
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175387#comment-16175387
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2166#discussion_r140341776
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -402,16 +402,20 @@ private String getColumnStateMaxValue(String
tableName, Map stat
return maxValue;
}
-private Integer getColumnType(String tableName, String colName) {
+private Integer getColumnType(final ProcessContext context, String
tableName, String colName, FlowFile flowFile) {
final String fullyQualifiedStateKey = getStateKey(tableName,
colName);
Integer type = columnTypeMap.get(fullyQualifiedStateKey);
if (type == null && !isDynamicTableName) {
// If the table name is static and the fully-qualified key was
not found, try just the column name
type = columnTypeMap.get(getStateKey(null, colName));
}
+if (type == null || columnTypeMap.size() == 0) {
+// This means column type cache is clean after instance
reboot. We should re-cache column type
+super.setup(context, false, flowFile);
--- End diff --
Calling `setup()` only updates `columnTypeMap`. The `type` variable will
stay being null here. Doesn't it throw ProcessException? Shouldn't we add `type
= columnTypeMap.get` after calling setup?
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175198#comment-16175198
]
Paul Gibeault commented on NIFI-4395:
-
[~ijokarumawak] Are you able to review this change in time for the 1.4.0
release?
https://github.com/apache/nifi/pull/2166
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174995#comment-16174995
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/2166
Hey @yjhyjhyjh0 - Don't worry about the CI builds - it has been failing for
a while now. We should really find the reason why and fix it... but I didn't
look into it yet. Regarding your PR, I think @ijokarumawak is going to review
it when he has time for it.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174945#comment-16174945
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user yjhyjhyjh0 commented on the issue:
https://github.com/apache/nifi/pull/2166
Not quite understand result from travis-ci.
Seems it stuck at ANTLR Parser AttributeExpressionParser every time?
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174657#comment-16174657
]
ASF GitHub Bot commented on NIFI-4395:
--
Github user yjhyjhyjh0 commented on the issue:
https://github.com/apache/nifi/pull/2166
I've solved this issue by re-cache column type if needed and pass local
unit and instance test.
The root cause of this issue is that originally disign didn't recognize
that column type cache will be clean after instance reboot.
This cause later unable to find column type cache with existing state value
and throw unhandle IllegalArgumentException.
Now if state value exist without corresponding column type cache, we will
fetch and cache it.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174655#comment-16174655
]
ASF GitHub Bot commented on NIFI-4395:
--
GitHub user yjhyjhyjh0 opened a pull request:
https://github.com/apache/nifi/pull/2166
NIFI-4395 - GenerateTableFetch can't fetch column type by state after…
… instance reboot
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yjhyjhyjh0/nifi NIFI-4395
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2166.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2166
commit faf6f2dfc15ef1900b6b868e19b90e39d934d07f
Author: Deon Huang
Date: 2017-09-21T07:34:00Z
NIFI-4395 - GenerateTableFetch can't fetch column type by state after
instance reboot
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ```
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ```
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't grab existing column type from *columnTypeMap* through
> instance reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172784#comment-16172784
]
Deon Huang commented on NIFI-4395:
--
Thanks for the confirmation.
I'll self assign and start to work on this.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
>Assignee: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ---
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ---
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't existing grab column type from columnTypeMap through instance
> reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4395) GenerateTableFetch can't fetch column type by state after instance reboot
[
https://issues.apache.org/jira/browse/NIFI-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172728#comment-16172728
]
Koji Kawamura commented on NIFI-4395:
-
[~deonashh]] Thanks for reporting this. I was able to reproduce the same
exception with MySQL. Full stack-trace as below:
{code}
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1]
o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
GenerateTableFetch[id=9d82e299-015e-1000-ba64-35cceeb836e5] due to uncaught
Exception: java.lang.IllegalArgumentException: No column type found for: ID
2017-09-20 13:24:17,218 WARN [Timer-Driven Process Thread-1]
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.IllegalArgumentException: No column type found for: ID
at
org.apache.nifi.processors.standard.GenerateTableFetch.getColumnType(GenerateTableFetch.java:414)
at
org.apache.nifi.processors.standard.GenerateTableFetch.lambda$onTrigger$0(GenerateTableFetch.java:246)
at
java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
at
org.apache.nifi.processors.standard.GenerateTableFetch.onTrigger(GenerateTableFetch.java:240)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
If you are able to submit a PR, that'd be really appreciated! Since I have
already reproduced the issue, the review and confirm process should be quick.
> GenerateTableFetch can't fetch column type by state after instance reboot
> -
>
> Key: NIFI-4395
> URL: https://issues.apache.org/jira/browse/NIFI-4395
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Deon Huang
> Fix For: 1.4.0
>
> Attachments: GenerateTableFetch_Exception.png
>
>
> The problem can easily be reproduce.
> Once GenerateTableFetch store state and encounter NiFi instance reboot.
> (Dynamic naming table by expression language)
> The exception will occur.
> The error in source code is list below.
> ---
> if (type == null) {
> // This shouldn't happen as we are populating columnTypeMap when the
> processor is scheduled or when the first maximum is observed
> throw new IllegalArgumentException("No column type found for: " +
> colName);
> }
> ---
> When this situation happened. The FlowFile will also be grab and can't
> release or observed.
> Processor can't existing grab column type from columnTypeMap through instance
> reboot.
> Hence will inevidible get this exception, rollback FlowFile and never success.
> QueryDatabaseTable processor will not encounter this exception due to it
> setup(context) every time,
> While GenerateTableFetch will not pass the condition and thus try to fetch
> column type from 0 length columnTypeMap.
> ---
> if (!isDynamicTableName && !isDynamicMaxValues) {
> super.setup(context);
> }
> ---
> I can take the issue if it is recognize as bug.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
