[
https://issues.apache.org/jira/browse/NIFI-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140394#comment-15140394
]
Joseph Witt commented on NIFI-1484:
-----------------------------------
and the stack trace
{quote}
2016-02-10 01:16:17,047 WARN [Timer-Driven Process Thread-9]
o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
ListFile[id=4b4d8023-8a39-417e-a85e-530a9e9b6520] due to uncaught Exception:
java.lang.NullPointerException
2016-02-10 01:16:17,048 WARN [Timer-Driven Process Thread-9]
o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.NullPointerException: null
at
org.apache.nifi.processors.standard.AbstractListProcessor.onTrigger(AbstractListProcessor.java:385)
~[na:na]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-0.5.0-SNAPSHOT.jar:0.5.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
~[nifi-framework-core-0.5.0-SNAPSHOT.jar:0.5.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
[nifi-framework-core-0.5.0-SNAPSHOT.jar:0.5.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
[nifi-framework-core-0.5.0-SNAPSHOT.jar:0.5.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
[nifi-framework-core-0.5.0-SNAPSHOT.jar:0.5.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_66]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_66]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_66]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_66]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_66]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
{quote}
> ListFile holds unbounded list of files with matching time stamps
> ----------------------------------------------------------------
>
> Key: NIFI-1484
> URL: https://issues.apache.org/jira/browse/NIFI-1484
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI, Extensions
> Affects Versions: 0.4.0, 0.5.0
> Reporter: Joseph Witt
> Fix For: 0.5.0
>
>
> ListFile appears to hold an unbounded set of filenames that match the last
> timestamp. While this is understandable to handle the edge case of new data
> arriving at the same time it presents two problems. First we hold all of
> this information in state management which could put considerable pressure on
> both the local and remote stores but we also have it in memory before we
> persist it.
> Also, the entire state listing appears to show up in the UI without
> pagination or any limit on number of entries. This seems like a problem for
> the client-side as well. The server side should probably restrict this.
> Finally, it seems like the need for saving filenames seen at a given
> timestamp is only necessary if we're assuming the listing we do is 'as-of'
> RIGHT NOW. What is instead we did the listing based on a last modified time
> of 'RIGHTNOW'-1 millisecond or something like that? Then we should not have
> to worry at all about keeping a listing of names for the timestamp.
> The reason I think this is important is that it is not at all uncommon for a
> directory with large quantities of files to have data at the same time due to
> a copy operation not preserving original file attributes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
