[
https://issues.apache.org/jira/browse/NIFI-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609528#comment-17609528
]
Joe Witt commented on NIFI-10547:
---------------------------------
Hello. We will need more details about the configuration but specifically we
will need the stack traces. This looks like a grep for ERROR in the logs but
that will only take the first thing. We need the stacks that follow. There is
clearly an issue with the Cassandra QL processor. This being NiFi 1.8.0 it is
likely the solution will be to upgrade.
Thanks
> NiFi cluster is unstable with error "Administrative Yield Duration"
> -------------------------------------------------------------------
>
> Key: NIFI-10547
> URL: https://issues.apache.org/jira/browse/NIFI-10547
> Project: Apache NiFi
> Issue Type: Task
> Affects Versions: 1.8.0
> Environment: Prod
> Reporter: Anoop Ajit
> Priority: Major
>
> Nifi cluster is refusing to be stable and is throwing various errors like
> "Too many Open files" and "Administrative Yield Duration".
>
> Below is the error logs:
>
> 2022-09-26 21:25:25,745 ERROR [Timer-Driven Process Thread-83]
> o.a.n.p.cassandra.QueryCassandra
> QueryCassandra[id=daa02218-016b-1000-ffff-fffff4865b2f] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:25:25,775 ERROR [Timer-Driven Process Thread-139]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=d6ad5e1c-eec8-3e77-9636-247017134eee] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:25:57,865 ERROR [Timer-Driven Process Thread-31]
> o.a.n.p.cassandra.QueryCassandra
> QueryCassandra[id=daa02218-016b-1000-ffff-fffff4865b2f] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:25:57,974 ERROR [Timer-Driven Process Thread-44]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=f02c3fb7-3a9c-147c-9bc1-d30b6ba15183] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:25:58,006 ERROR [Timer-Driven Process Thread-164]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=d6ad5e1c-eec8-3e77-9636-247017134eee] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:20,039 ERROR [Timer-Driven Process Thread-72]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=318ec9e8-016b-1000-ffff-ffffc852c4db] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> dd7b8feb-4f95-495e-980f-05d89641daff in bucket
> 49b79584-8e25-4e3b-80cc-bf2b6ff4a452
> 2022-09-26 21:26:20,040 ERROR [Timer-Driven Process Thread-72]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=3190cbfe-016b-1000-ffff-ffffab2b0f70] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 70323c57-4dc0-4157-ba01-7efbef1a1e1b in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:26:20,041 ERROR [Timer-Driven Process Thread-72]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=f59f32d1-d71f-1cbc-86ad-9242e95745ad] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 4dc233a0-d21e-4a41-b271-80d3f4d9310f in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:26:20,042 ERROR [Timer-Driven Process Thread-72]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=b4ea3324-079f-1226-b6b6-7e93c8b1318d] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 67db4103-271f-43c8-8bc6-68c7729f3504 in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:26:28,174 ERROR [Timer-Driven Process Thread-220]
> o.a.n.p.cassandra.QueryCassandra
> QueryCassandra[id=daa02218-016b-1000-ffff-fffff4865b2f] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:28,259 ERROR [Timer-Driven Process Thread-145]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=f02c3fb7-3a9c-147c-9bc1-d30b6ba15183] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:28,297 ERROR [Timer-Driven Process Thread-257]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=d6ad5e1c-eec8-3e77-9636-247017134eee] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:58,468 ERROR [Timer-Driven Process Thread-100]
> o.a.n.p.cassandra.QueryCassandra
> QueryCassandra[id=daa02218-016b-1000-ffff-fffff4865b2f] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:58,545 ERROR [Timer-Driven Process Thread-300]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=f02c3fb7-3a9c-147c-9bc1-d30b6ba15183] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:26:58,578 ERROR [Timer-Driven Process Thread-237]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=d6ad5e1c-eec8-3e77-9636-247017134eee] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:27:20,045 ERROR [Timer-Driven Process Thread-70]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=318ec9e8-016b-1000-ffff-ffffc852c4db] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> dd7b8feb-4f95-495e-980f-05d89641daff in bucket
> 49b79584-8e25-4e3b-80cc-bf2b6ff4a452
> 2022-09-26 21:27:20,046 ERROR [Timer-Driven Process Thread-70]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=3190cbfe-016b-1000-ffff-ffffab2b0f70] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 70323c57-4dc0-4157-ba01-7efbef1a1e1b in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:27:20,047 ERROR [Timer-Driven Process Thread-70]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=f59f32d1-d71f-1cbc-86ad-9242e95745ad] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 4dc233a0-d21e-4a41-b271-80d3f4d9310f in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:27:20,048 ERROR [Timer-Driven Process Thread-70]
> o.a.nifi.groups.StandardProcessGroup Failed to synchronize
> StandardProcessGroup[identifier=b4ea3324-079f-1226-b6b6-7e93c8b1318d] with
> Flow Registry because could not retrieve version 1 of flow with identifier
> 67db4103-271f-43c8-8bc6-68c7729f3504 in bucket
> 0a33dfc5-dd66-49d7-977b-e82a56bcc4df
> 2022-09-26 21:27:28,772 ERROR [Timer-Driven Process Thread-156]
> o.a.n.p.cassandra.QueryCassandra
> QueryCassandra[id=daa02218-016b-1000-ffff-fffff4865b2f] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:27:28,836 ERROR [Timer-Driven Process Thread-276]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=f02c3fb7-3a9c-147c-9bc1-d30b6ba15183] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
> 2022-09-26 21:27:28,856 ERROR [Timer-Driven Process Thread-153]
> o.a.n.p.cassandra.PutCassandraQL
> PutCassandraQL[id=d6ad5e1c-eec8-3e77-9636-247017134eee] Failed to properly
> initialize Processor. If still scheduled to run, NiFi will attempt to
> initialize and run the Processor again after the 'Administrative Yield
> Duration' has elapsed. Failure is due to
> java.lang.reflect.InvocationTargetException:
> java.lang.reflect.InvocationTargetException
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
