Dirk Daems created NIFI-14383:
---------------------------------
Summary: Component states out of sync
Key: NIFI-14383
URL: https://issues.apache.org/jira/browse/NIFI-14383
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.16.3
Reporter: Dirk Daems
We run 3 3-node Nifi clusters with version 1.16.3.
We notice that component states (running, stopped) get out of sync between
cluster nodes. This can be verified by comparing the nifi-processors.log
between nodes using the following commands:
{code}
cat /dev/null > nifi-processors.log # optionally, cleanup logfile & wait for a
while
cat nifi-processors.log | sort | awk -F "|" '\{print $2, $3, $4, $5}' | uniq >
nifi-processors.uniq.log
{code}
Next, compare the processors.uniq.log over all nodes. To give an idea, states
start to diverge significantly after a while for a lot of different components
and component types:
{code}
$ diff nifi-prod-01-intern.processors.uniq.log
nifi-prod-02-intern.processors.uniq.log
6c6
< Add timestamp 000281e4-2b42-322c-8300-4ec632ed3905
UpdateRecord Stopped
---
> Add timestamp 000281e4-2b42-322c-8300-4ec632ed3905
> UpdateRecord Running
9c9
< AttributesToJSON 3d2ca68b-d9c7-339f-8665-ea8ee0261aa6
AttributesToJSON Stopped
---
> AttributesToJSON 3d2ca68b-d9c7-339f-8665-ea8ee0261aa6
> AttributesToJSON Running
13,14c13,14
< Build request Part 1 d8eb370e-8af1-1780-8f53-7ecc6325a034
ReplaceText Stopped
< Build request Part 2 01d73a24-c0ff-12af-ac6a-20a41b391c1c
ReplaceText Stopped
---
> Build request Part 1 d8eb370e-8af1-1780-8f53-7ecc6325a034
> ReplaceText Running
> Build request Part 2 01d73a24-c0ff-12af-ac6a-20a41b391c1c
> ReplaceText Running
30,31c30,31
< Calculate summer amounts c3c11882-69f7-3fdf-ab2a-b9290ae71949
ExecuteStreamCommand Stopped
< Calculate summer ogs 5f3325d6-5c1f-36aa-b049-e65971d84120
ExecuteStreamCommand Stopped
---
> Calculate summer amounts c3c11882-69f7-3fdf-ab2a-b9290ae71949
> ExecuteStreamCommand Running
> Calculate summer ogs 5f3325d6-5c1f-36aa-b049-e65971d84120
> ExecuteStreamCommand Running
36c36
< Calculate zoom level 399f3d10-490f-1799-9c93-bc4736271db9
ExecuteStreamCommand Stopped
---
> Calculate zoom level 399f3d10-490f-1799-9c93-bc4736271db9
> ExecuteStreamCommand Running
48c48
< Check job state 1a163c15-a8fe-1375-b9c8-28c6cb184c1e
RouteOnContent Stopped
---
> Check job state 1a163c15-a8fe-1375-b9c8-28c6cb184c1e
> RouteOnContent Running
51c51
< Cleanup file in working direct c97dee4a-0184-1000-ffff-fffff3711d83
ExecuteStreamCommand Stopped
---
> Cleanup file in working direct c97dee4a-0184-1000-ffff-fffff3711d83
> ExecuteStreamCommand Running
54c54
< Consume from pid-2-es cc107490-6b7c-32ef-a4b8-b5ae8edaf110
ConsumeKafka_2_0 Stopped
---
> Consume from pid-2-es cc107490-6b7c-32ef-a4b8-b5ae8edaf110
> ConsumeKafka_2_0 Running
59,60c59,60
< ConvertAvroToJSON e0cadb45-bfb1-39df-8a92-712b26103569
ConvertAvroToJSON Stopped
< ConvertJSON2Attributes 043b5802-8959-3c24-b6eb-f3431cb1285b
EvaluateJsonPath Stopped
---
> ConvertAvroToJSON e0cadb45-bfb1-39df-8a92-712b26103569
> ConvertAvroToJSON Running
> ConvertJSON2Attributes 043b5802-8959-3c24-b6eb-f3431cb1285b
> EvaluateJsonPath Running
63c63
< ConvertJSON2Attributes e1ed91b1-79aa-3a5e-8d85-d734a784863a
EvaluateJsonPath Stopped
---
> ConvertJSON2Attributes e1ed91b1-79aa-3a5e-8d85-d734a784863a
> EvaluateJsonPath Running
73c73
< Convert to date d6b53522-0e6a-1994-8462-8dcf38696f38
UpdateAttribute Stopped
---
> Convert to date d6b53522-0e6a-1994-8462-8dcf38696f38
> UpdateAttribute Running
86c86
< Crop by input file 74542437-59cc-3606-a55f-583cc4c3812d
ExecuteStreamCommand Stopped
---
> Crop by input file 74542437-59cc-3606-a55f-583cc4c3812d
> ExecuteStreamCommand Running
92,93c92,93
< Crop by search a3c8ec2f-d097-3482-a44f-d45bcba579f5
ExecuteStreamCommand Stopped
< Crop by search f04852cd-23dc-3b33-9f35-bd77a7ebdcda
ExecuteStreamCommand Stopped
---
> Crop by search a3c8ec2f-d097-3482-a44f-d45bcba579f5
> ExecuteStreamCommand Running
> Crop by search f04852cd-23dc-3b33-9f35-bd77a7ebdcda
> ExecuteStreamCommand Running
101c101
< Diff by input file 09b583c4-5bcc-32d3-b423-d933964ab13a
ExecuteStreamCommand Stopped
---
> Diff by input file 09b583c4-5bcc-32d3-b423-d933964ab13a
> ExecuteStreamCommand Running
103c103
< Diff by input file ecac9564-cf79-3cce-911a-dcc6390a4fd9
ExecuteStreamCommand Stopped
---
> Diff by input file ecac9564-cf79-3cce-911a-dcc6390a4fd9
> ExecuteStreamCommand Running
114,116c114,116
< Download tif file d5db309f-66cc-1e66-a6bb-ceb6f01b9d9a
InvokeHTTP Stopped
< ErrorMail 2af01e65-a296-319d-9418-67fd2cacd9ce
PutEmail Stopped
< ErrorMail 402662b2-3074-397a-ade9-c831f899a37b
PutEmail Stopped
---
> Download tif file d5db309f-66cc-1e66-a6bb-ceb6f01b9d9a
> InvokeHTTP Running
> ErrorMail 2af01e65-a296-319d-9418-67fd2cacd9ce
> PutEmail Running
> ErrorMail 402662b2-3074-397a-ade9-c831f899a37b
> PutEmail Running
118,119c118,119
< EvaluateJsonPath 45c8ab7e-c634-3df2-8b03-4ec8327f0e96
EvaluateJsonPath Stopped
< EvaluateJsonPath 5815b1d7-a6d6-395e-8716-9d4acfe7c2ab
EvaluateJsonPath Stopped
---
> EvaluateJsonPath 45c8ab7e-c634-3df2-8b03-4ec8327f0e96
> EvaluateJsonPath Running
> EvaluateJsonPath 5815b1d7-a6d6-395e-8716-9d4acfe7c2ab
> EvaluateJsonPath Running
122c122
< EvaluateJsonPath 91b48aaa-c538-3d9a-b42b-50a301f7f622
EvaluateJsonPath Stopped
---
> EvaluateJsonPath 91b48aaa-c538-3d9a-b42b-50a301f7f622
> EvaluateJsonPath Running
124c124
< EvaluateJsonPath ac17c5be-c6bf-3e41-a8dc-6a899ec137ec
EvaluateJsonPath Stopped
---
> EvaluateJsonPath ac17c5be-c6bf-3e41-a8dc-6a899ec137ec
> EvaluateJsonPath Running
126,130c126,130
< EvaluateJsonPath c86a1f68-5729-3468-84d6-5efc2055546b
EvaluateJsonPath Stopped
< EvaluateJsonPath fd0ccbeb-c24d-3470-9dbf-a59fea15a7b3
EvaluateJsonPath Stopped
< ExecuteStreamCommand 1966a865-0181-1000-0000-00005134f1ab
ExecuteStreamCommand Stopped
< Extract Aquafin id and amount 633c096c-9655-3246-82e1-871e96c8aa0a
ExtractText Stopped
< Extract country and croptype 200535c3-ecb8-165b-91a7-c146323a63cf
EvaluateJsonPath Stopped
---
> EvaluateJsonPath c86a1f68-5729-3468-84d6-5efc2055546b
> EvaluateJsonPath Running
> EvaluateJsonPath fd0ccbeb-c24d-3470-9dbf-a59fea15a7b3
> EvaluateJsonPath Running
> ExecuteStreamCommand 1966a865-0181-1000-0000-00005134f1ab
> ExecuteStreamCommand Running
> Extract Aquafin id and amount 633c096c-9655-3246-82e1-871e96c8aa0a
> ExtractText Running
> Extract country and croptype 200535c3-ecb8-165b-91a7-c146323a63cf
> EvaluateJsonPath Running
136c136
< Extract geometry 0d8f3ecb-36e7-16cb-975e-5f98aa966120
EvaluateJsonPath Stopped
---
> Extract geometry 0d8f3ecb-36e7-16cb-975e-5f98aa966120
> EvaluateJsonPath Running
144,145c144,145
< ExtractText 22f97325-b7a1-3fdc-87c6-62bc0d611037
ExtractText Stopped
< ExtractText 71efd496-38ab-3826-8aff-670176cc60ed
ExtractText Stopped
---
> ExtractText 22f97325-b7a1-3fdc-87c6-62bc0d611037
> ExtractText Running
> ExtractText 71efd496-38ab-3826-8aff-670176cc60ed
> ExtractText Running
149c149
< Fallback Kerberos TGT refresh ddfe068e-0193-1000-ffff-ffffb5e3126d
ExecuteStreamCommand Stopped
---
> Fallback Kerberos TGT refresh ddfe068e-0193-1000-ffff-ffffb5e3126d
> ExecuteStreamCommand Running
165c165
< Filter out functional errors e2a9670f-0184-1000-ffff-ffff96ca55e7
RouteOnAttribute Stopped
---
> Filter out functional errors e2a9670f-0184-1000-ffff-ffff96ca55e7
> RouteOnAttribute Running
191c191
< Generate token request body 5b0fe682-685a-3ba7-980f-920848011fb4
GenerateFlowFile Stopped
---
> Generate token request body 5b0fe682-685a-3ba7-980f-920848011fb4
> GenerateFlowFile Running
213c213
< Get field id 17174a2b-57b0-3b51-b7c7-2d11a172e5e8
UpdateAttribute Stopped
---
> Get field id 17174a2b-57b0-3b51-b7c7-2d11a172e5e8
> UpdateAttribute Running
218c218
< Get field start and end date 9c1f3042-b80a-1b98-9cc5-9b9a28c4a73e
EvaluateJsonPath Stopped
---
> Get field start and end date 9c1f3042-b80a-1b98-9cc5-9b9a28c4a73e
> EvaluateJsonPath Running
227c227
< Get OpenStack auth token 1240b168-0d68-3633-9de8-ab166083dc50
InvokeHTTP Stopped
---
> Get OpenStack auth token 1240b168-0d68-3633-9de8-ab166083dc50
> InvokeHTTP Running
232c232
< Get S3 buckets 6e61526b-eb53-365a-bc7a-786a3630d950
InvokeHTTP Stopped
---
> Get S3 buckets 6e61526b-eb53-365a-bc7a-786a3630d950
> InvokeHTTP Running
241c241
< Get WiG Monitor field by ID cb2155b4-d8e3-3b8e-8f7a-42376878bfb9
QueryElasticsearchHttp Stopped
---
> Get WiG Monitor field by ID cb2155b4-d8e3-3b8e-8f7a-42376878bfb9
> QueryElasticsearchHttp Running
244c244
< Ingest to GeoServer by date ra 62075c82-c698-3f83-b1b1-8d6ca3394fba
ExecuteStreamCommand Stopped
---
> Ingest to GeoServer by date ra 62075c82-c698-3f83-b1b1-8d6ca3394fba
> ExecuteStreamCommand Running
248c248
< Insert daily amount 318783f7-2430-35e3-b877-af7556e5e752
PutElasticsearchHttp Stopped
---
> Insert daily amount 318783f7-2430-35e3-b877-af7556e5e752
> PutElasticsearchHttp Running
258c258
< Listen2BioParKafkaTopic 07a044b2-1c69-33d1-b8d1-db4172cb2821
ConsumeKafka_2_0 Stopped
---
> Listen2BioParKafkaTopic 07a044b2-1c69-33d1-b8d1-db4172cb2821
> ConsumeKafka_2_0 Running
266c266
< Listen for requests 768c60d9-7cf6-32fb-82de-6db7873506e5
ListenHTTP Stopped
---
> Listen for requests 768c60d9-7cf6-32fb-82de-6db7873506e5
> ListenHTTP Running
269c269
< Listen for requests (DEV) f4903634-47c6-3f78-bbbd-2806418827da
ListenHTTP Stopped
---
> Listen for requests (DEV) f4903634-47c6-3f78-bbbd-2806418827da
> ListenHTTP Running
294,295c294,295
< Map elasticsearch hits a45d009d-11bd-360d-8d35-699003acc7d1
EvaluateJsonPath Stopped
< Map elasticsearch hits f47e18d2-1f4b-3464-8696-c2084ef2a60b
EvaluateJsonPath Stopped
---
> Map elasticsearch hits a45d009d-11bd-360d-8d35-699003acc7d1
> EvaluateJsonPath Running
> Map elasticsearch hits f47e18d2-1f4b-3464-8696-c2084ef2a60b
> EvaluateJsonPath Running
306c306
< MergeContent e7dc4ed5-0b95-31c5-8869-78d4d73829b4
MergeContent Stopped
---
> MergeContent e7dc4ed5-0b95-31c5-8869-78d4d73829b4
> MergeContent Running
311,312c311,312
< Merge layer updates 51e5faa1-4228-3765-bd6a-31a966e3b0cb
MergeContent Stopped
< Merge layer updates d51522dc-21c6-3692-824d-b3455faeea1a
MergeContent Stopped
---
> Merge layer updates 51e5faa1-4228-3765-bd6a-31a966e3b0cb
> MergeContent Running
> Merge layer updates d51522dc-21c6-3692-824d-b3455faeea1a
> MergeContent Running
332,333c332,333
< NDVI only 01d0fcc6-51af-320a-83f5-b890127aea39
RouteOnAttribute Stopped
< NDVI only 6eee7e37-add5-303d-a2af-9477490df9b3
RouteOnAttribute Stopped
---
> NDVI only 01d0fcc6-51af-320a-83f5-b890127aea39
> RouteOnAttribute Running
> NDVI only 6eee7e37-add5-303d-a2af-9477490df9b3
> RouteOnAttribute Running
338c338
< Nodes JSON array to list 3a6e72ab-04a1-131c-ffff-ffffb6bc29c4
UpdateAttribute Stopped
---
> Nodes JSON array to list 3a6e72ab-04a1-131c-ffff-ffffb6bc29c4
> UpdateAttribute Running
342c342
< Parse date from csv filename 7dcd3ae0-54e6-3154-be63-6af3e150115c
UpdateAttribute Stopped
---
> Parse date from csv filename 7dcd3ae0-54e6-3154-be63-6af3e150115c
> UpdateAttribute Running
346c346
< PenalizeFlowfile cd18f411-0184-1000-0000-0000634de6cb
ExecuteGroovyScript Stopped
---
> PenalizeFlowfile cd18f411-0184-1000-0000-0000634de6cb
> ExecuteGroovyScript Running
349,350c349,350
< PerformESQueryAndSaveResults 8a9fb202-9972-3ec4-8999-25e046e6bb44
ExecuteStreamCommand Stopped
< Perform insert? 9bbd773b-4154-3892-9398-ea464ee5130e
RouteOnAttribute Stopped
---
> PerformESQueryAndSaveResults 8a9fb202-9972-3ec4-8999-25e046e6bb44
> ExecuteStreamCommand Running
> Perform insert? 9bbd773b-4154-3892-9398-ea464ee5130e
> RouteOnAttribute Running
360,363c360,363
< Prepare maintenance mode reque 00b03a51-ef7b-1324-be6c-c9ad6084d3f8
ReplaceText Stopped
< Prepare maintenance mode reque 5e6d3f0d-b307-1340-b9cc-1a9fd93b658e
ReplaceText Stopped
< Prepare request 3a6e6194-04a1-131c-ffff-ffffbd9b0111
ReplaceText Stopped
< PreProcessingFailed 41e8c78a-879e-337f-920c-f0d8827e7ba1
PutEmail Stopped
---
> Prepare maintenance mode reque 00b03a51-ef7b-1324-be6c-c9ad6084d3f8
> ReplaceText Running
> Prepare maintenance mode reque 5e6d3f0d-b307-1340-b9cc-1a9fd93b658e
> ReplaceText Running
> Prepare request 3a6e6194-04a1-131c-ffff-ffffbd9b0111
> ReplaceText Running
> PreProcessingFailed 41e8c78a-879e-337f-920c-f0d8827e7ba1
> PutEmail Running
367,372c367,372
< ProductType BIOPAR dbbb7bd6-7fe4-31ef-a919-59bf63628fe6
ReplaceText Stopped
< ProductTypeNotSupported 8af13111-3b1a-3bd4-a68f-29b7342d3e14
PutEmail Stopped
< ProductType PROBAV 13676455-d74a-334a-b043-f9addc203c93
ReplaceText Stopped
< PublishingToKafkaFailed 28f09816-eb03-3f8e-aa26-86b938278c4c
PutEmail Stopped
< Publish to BIOPAR topic 90cf8266-48cd-3cc2-8367-19549e9ec01c
PublishKafka_2_0 Stopped
< Publish to PROBAV topic 9692f962-fb0d-38ea-9a16-e422d6e29d27
PublishKafka_2_0 Stopped
---
> ProductType BIOPAR dbbb7bd6-7fe4-31ef-a919-59bf63628fe6
> ReplaceText Running
> ProductTypeNotSupported 8af13111-3b1a-3bd4-a68f-29b7342d3e14
> PutEmail Running
> ProductType PROBAV 13676455-d74a-334a-b043-f9addc203c93
> ReplaceText Running
> PublishingToKafkaFailed 28f09816-eb03-3f8e-aa26-86b938278c4c
> PutEmail Running
> Publish to BIOPAR topic 90cf8266-48cd-3cc2-8367-19549e9ec01c
> PublishKafka_2_0 Running
> Publish to PROBAV topic 9692f962-fb0d-38ea-9a16-e422d6e29d27
> PublishKafka_2_0 Running
374c374
< PutElasticsearchHttp 1d825800-2ad8-3b79-ac19-f2df0f14698d
PutElasticsearchHttp Stopped
---
> PutElasticsearchHttp 1d825800-2ad8-3b79-ac19-f2df0f14698d
> PutElasticsearchHttp Running
378c378
< PutEmail 15c86ba7-3455-3d4b-9641-d0af25664038
PutEmail Stopped
---
> PutEmail 15c86ba7-3455-3d4b-9641-d0af25664038
> PutEmail Running
401,402c401,402
< Query provider amounts efd704b0-1dd3-3008-ba3a-c50667161eec
JsonQueryElasticsearch Stopped
< Query provider ogs 4d6c453f-430c-3995-866a-f83f3ab6f7f3
JsonQueryElasticsearch Stopped
---
> Query provider amounts efd704b0-1dd3-3008-ba3a-c50667161eec
> JsonQueryElasticsearch Running
> Query provider ogs 4d6c453f-430c-3995-866a-f83f3ab6f7f3
> JsonQueryElasticsearch Running
414,415c414,415
< Read files to ingest 99101eed-273b-3b4e-be1d-3784952a09a6
SplitJson Stopped
< Read files to ingest edb94501-2c34-3862-bdcc-ed472bb74c72
SplitJson Stopped
---
> Read files to ingest 99101eed-273b-3b4e-be1d-3784952a09a6
> SplitJson Running
> Read files to ingest edb94501-2c34-3862-bdcc-ed472bb74c72
> SplitJson Running
419c419
< Reindex production to staging 195fb31b-0181-1000-ffff-ffffc2cb1e86
PostHTTP Stopped
---
> Reindex production to staging 195fb31b-0181-1000-ffff-ffffc2cb1e86
> PostHTTP Running
423,426c423,426
< ReplaceText 5591266c-1b87-3d0c-b15b-ec30175788ad
ReplaceText Stopped
< ReplaceText f56e80ae-58e1-39ed-9a95-b0648f6fe47c
ReplaceText Stopped
< Request job state f7a83874-3b09-1d47-b9eb-0fabf1f6b746
InvokeHTTP Stopped
< Request soil moisture data bc3830ec-4a88-1b9a-b8c3-4be34bf13934
InvokeHTTP Stopped
---
> ReplaceText 5591266c-1b87-3d0c-b15b-ec30175788ad
> ReplaceText Running
> ReplaceText f56e80ae-58e1-39ed-9a95-b0648f6fe47c
> ReplaceText Running
> Request job state f7a83874-3b09-1d47-b9eb-0fabf1f6b746
> InvokeHTTP Running
> Request soil moisture data bc3830ec-4a88-1b9a-b8c3-4be34bf13934
> InvokeHTTP Running
429c429
< Restart NodeManagers 3a6e66cc-04a1-131c-0000-0000149aab16
InvokeHTTP Stopped
---
> Restart NodeManagers 3a6e66cc-04a1-131c-0000-0000149aab16
> InvokeHTTP Running
432c432
< Retrieve job id ab603922-7294-1aa8-9ed5-e259ba875063
EvaluateJsonPath Stopped
---
> Retrieve job id ab603922-7294-1aa8-9ed5-e259ba875063
> EvaluateJsonPath Running
455,457c455,457
< RouteOnAttribute 0acda8f4-0505-391d-913d-668f00d49af3
RouteOnAttribute Stopped
< RouteOnAttribute 196de661-2947-3f1c-91cd-e71105e5df48
RouteOnAttribute Stopped
< RouteOnAttribute 472e3908-e0c7-1ac4-8368-821ecfa90590
RouteOnAttribute Stopped
---
> RouteOnAttribute 0acda8f4-0505-391d-913d-668f00d49af3
> RouteOnAttribute Running
> RouteOnAttribute 196de661-2947-3f1c-91cd-e71105e5df48
> RouteOnAttribute Running
> RouteOnAttribute 472e3908-e0c7-1ac4-8368-821ecfa90590
> RouteOnAttribute Running
459,460c459,460
< RouteOnAttribute 67b4aa91-afe4-3971-8f41-aa1e1b4b5338
RouteOnAttribute Stopped
< RouteOnAttribute 69470228-c723-3028-83e3-6127846fc566
RouteOnAttribute Stopped
---
> RouteOnAttribute 67b4aa91-afe4-3971-8f41-aa1e1b4b5338
> RouteOnAttribute Running
> RouteOnAttribute 69470228-c723-3028-83e3-6127846fc566
> RouteOnAttribute Running
463,464c463,464
< RouteOnAttribute 90d57b06-2506-3a50-bf75-da6c2084eecf
RouteOnAttribute Stopped
< RouteOnAttribute b32e38ff-7306-3e88-96ff-b7ac1d4ccc1f
RouteOnAttribute Stopped
---
> RouteOnAttribute 90d57b06-2506-3a50-bf75-da6c2084eecf
> RouteOnAttribute Running
> RouteOnAttribute b32e38ff-7306-3e88-96ff-b7ac1d4ccc1f
> RouteOnAttribute Running
472c472
< Save file in working directory 6f2b31be-331a-1b66-98bc-3cf413ac7a96
PutFile Stopped
---
> Save file in working directory 6f2b31be-331a-1b66-98bc-3cf413ac7a96
> PutFile Running
474,475c474,475
< Search daily provider amount 894d09fc-e120-3d2b-ae61-03755886d812
QueryElasticsearchHttp Stopped
< Search provider id 29bea5c0-c649-3a7d-a9de-458e333d9724
QueryElasticsearchHttp Stopped
---
> Search daily provider amount 894d09fc-e120-3d2b-ae61-03755886d812
> QueryElasticsearchHttp Running
> Search provider id 29bea5c0-c649-3a7d-a9de-458e333d9724
> QueryElasticsearchHttp Running
487,488c487,488
< Send email alerts f6be2a1e-061e-3dbf-bc85-1b991c04f0de
PutEmail Stopped
< Send email OK 19723ab9-0181-1000-0000-00006fa722d5
PutEmail Stopped
---
> Send email alerts f6be2a1e-061e-3dbf-bc85-1b991c04f0de
> PutEmail Running
> Send email OK 19723ab9-0181-1000-0000-00006fa722d5
> PutEmail Running
491c491
< Send error email 3b823de9-5356-1197-a2c2-77879bc5a277
PutEmail Stopped
---
> Send error email 3b823de9-5356-1197-a2c2-77879bc5a277
> PutEmail Running
511,512c511,512
< SendQueryFailedMail aa573f8c-caaf-1c1f-a56c-dbbdd562e05c
PutEmail Stopped
< SendQueryFailedMail db5437b8-aae2-1eb9-8d4d-7ff792a340af
PutEmail Stopped
---
> SendQueryFailedMail aa573f8c-caaf-1c1f-a56c-dbbdd562e05c
> PutEmail Running
> SendQueryFailedMail db5437b8-aae2-1eb9-8d4d-7ff792a340af
> PutEmail Running
524c524
< Set ES index 266bd706-0186-1000-ffff-ffffd01c937e
UpdateAttribute Stopped
---
> Set ES index 266bd706-0186-1000-ffff-ffffd01c937e
> UpdateAttribute Running
526,527c526,527
< Set field idx and calculation ce67c4cb-5410-31bf-a1a1-e9d95ccdf121
UpdateAttribute Stopped
< Set field idx and calculation da89e6d9-60eb-38d9-8cee-5766fa3f1a11
UpdateAttribute Stopped
---
> Set field idx and calculation ce67c4cb-5410-31bf-a1a1-e9d95ccdf121
> UpdateAttribute Running
> Set field idx and calculation da89e6d9-60eb-38d9-8cee-5766fa3f1a11
> UpdateAttribute Running
531c531
< Set field start and end date 9f61952f-007c-34f5-a455-f5bb6fd86ed1
UpdateAttribute Stopped
---
> Set field start and end date 9f61952f-007c-34f5-a455-f5bb6fd86ed1
> UpdateAttribute Running
545,548c545,548
< Split buckets 7a44bcf1-0e5c-3990-b4bf-8aeaf8e86991
SplitJson Stopped
< SplitCsv ded2bf3c-fbb1-308a-aebc-c3edbece117c
SplitText Stopped
< Split file urls b3ec6956-9536-1a83-0000-00004c894228
SplitJson Stopped
< SplitJson 0fd1b0b0-2bf0-3152-9933-6752e9e6001e
SplitJson Stopped
---
> Split buckets 7a44bcf1-0e5c-3990-b4bf-8aeaf8e86991
> SplitJson Running
> SplitCsv ded2bf3c-fbb1-308a-aebc-c3edbece117c
> SplitText Running
> Split file urls b3ec6956-9536-1a83-0000-00004c894228
> SplitJson Running
> SplitJson 0fd1b0b0-2bf0-3152-9933-6752e9e6001e
> SplitJson Running
554c554
< Split soil moisture file list 81ff3751-bc2c-16e8-9795-cc2fe8f617fb
SplitJson Stopped
---
> Split soil moisture file list 81ff3751-bc2c-16e8-9795-cc2fe8f617fb
> SplitJson Running
558c558
< Statistics update error mail 62513881-7525-11db-98df-ac3e75f479a5
PutEmail Stopped
---
> Statistics update error mail 62513881-7525-11db-98df-ac3e75f479a5
> PutEmail Running
564,567c564,567
< Store filename and es hosts as 5957378c-ca15-14f3-8d8d-5a8403f6db75
UpdateAttribute Stopped
< Store file url as variable 123e3bee-4990-13aa-8225-dedaf3d785b2
ExtractText Stopped
< Store in ElasticSearch 2b390562-67da-3295-a541-35d257b27ca3
PutElasticsearchHttp Stopped
< Store max, min lat and lon to 0d20360d-456f-1e3c-b1f7-958b2503b7a8
EvaluateJsonPath Stopped
---
> Store filename and es hosts as 5957378c-ca15-14f3-8d8d-5a8403f6db75
> UpdateAttribute Running
> Store file url as variable 123e3bee-4990-13aa-8225-dedaf3d785b2
> ExtractText Running
> Store in ElasticSearch 2b390562-67da-3295-a541-35d257b27ca3
> PutElasticsearchHttp Running
> Store max, min lat and lon to 0d20360d-456f-1e3c-b1f7-958b2503b7a8
> EvaluateJsonPath Running
572c572
< Trigger - Crop data by search 024baac7-d3e7-3215-9486-cbc85822a177
GenerateFlowFile Stopped
---
> Trigger - Crop data by search 024baac7-d3e7-3215-9486-cbc85822a177
> GenerateFlowFile Running
574c574
< Trigger - Crop data by search 2966cd8e-67c8-3d66-b64d-882b0dee16f8
GenerateFlowFile Stopped
---
> Trigger - Crop data by search 2966cd8e-67c8-3d66-b64d-882b0dee16f8
> GenerateFlowFile Running
591,592c591,592
< Trigger - Mask and Crop data b b84b2a20-db63-3a12-85d1-88b5c6dbf8e4
GenerateFlowFile Stopped
< Trigger scan 3a6e59fd-04a1-131c-0000-000073caa6a7
GenerateFlowFile Stopped
---
> Trigger - Mask and Crop data b b84b2a20-db63-3a12-85d1-88b5c6dbf8e4
> GenerateFlowFile Running
> Trigger scan 3a6e59fd-04a1-131c-0000-000073caa6a7
> GenerateFlowFile Running
596c596
< Turn OFF YARN maintenance mode 788638b6-6ad3-15b8-85de-d9b8a8e31ee6
InvokeHTTP Stopped
---
> Turn OFF YARN maintenance mode 788638b6-6ad3-15b8-85de-d9b8a8e31ee6
> InvokeHTTP Running
600c600
< Turn ON YARN maintenance mode cbfe31b8-7495-1c10-82bd-65bb8310d6fd
InvokeHTTP Stopped
---
> Turn ON YARN maintenance mode cbfe31b8-7495-1c10-82bd-65bb8310d6fd
> InvokeHTTP Running
602c602
< UpdateAttribute 1a27d5ac-3d97-3399-8519-2f2bb5b41556
UpdateAttribute Stopped
---
> UpdateAttribute 1a27d5ac-3d97-3399-8519-2f2bb5b41556
> UpdateAttribute Running
604c604
< UpdateAttribute 4a24f512-a7ae-3097-8138-24059fab39b1
UpdateAttribute Stopped
---
> UpdateAttribute 4a24f512-a7ae-3097-8138-24059fab39b1
> UpdateAttribute Running
609c609
< UpdateAttribute b2f34bc2-f240-3163-9990-0ac4ed3e5277
UpdateAttribute Stopped
---
> UpdateAttribute b2f34bc2-f240-3163-9990-0ac4ed3e5277
> UpdateAttribute Running
612c612
< UpdateAttribute d79ca8d6-5d91-30bc-9858-a34b99f2b7a0
UpdateAttribute Stopped
---
> UpdateAttribute d79ca8d6-5d91-30bc-9858-a34b99f2b7a0
> UpdateAttribute Running
614c614
< UpdateAttribute f4f80131-f8ac-3201-b31c-93b1d703b02b
UpdateAttribute Stopped
---
> UpdateAttribute f4f80131-f8ac-3201-b31c-93b1d703b02b
> UpdateAttribute Running
617c617
< Update daily amount 0013d7e4-2f54-3a2a-8279-0fbb482196a6
PutElasticsearchHttp Stopped
---
> Update daily amount 0013d7e4-2f54-3a2a-8279-0fbb482196a6
> PutElasticsearchHttp Running
621,625c621,625
< Update file url part 1 b3ec6a39-9536-1a83-0000-0000324656bb
ReplaceText Stopped
< Update file url part 2 264e5e26-0186-1000-ffff-ffffc8723624
ReplaceText Stopped
< Update layer name 36a2b94f-005e-38fb-8894-06864130d063
UpdateAttribute Stopped
< Update layer name d83ad513-802a-376b-a70d-2ce22ec0f22b
UpdateAttribute Stopped
< Update max-min amounts ab6bd12e-335b-31fc-be44-e8f05cd57575
PutElasticsearchHttp Stopped
---
> Update file url part 1 b3ec6a39-9536-1a83-0000-0000324656bb
> ReplaceText Running
> Update file url part 2 264e5e26-0186-1000-ffff-ffffc8723624
> ReplaceText Running
> Update layer name 36a2b94f-005e-38fb-8894-06864130d063
> UpdateAttribute Running
> Update layer name d83ad513-802a-376b-a70d-2ce22ec0f22b
> UpdateAttribute Running
> Update max-min amounts ab6bd12e-335b-31fc-be44-e8f05cd57575
> PutElasticsearchHttp Running
629c629
< ValidateCsv 1969b8a9-0895-3297-80d3-49642462ced6
ValidateCsv Stopped
---
> ValidateCsv 1969b8a9-0895-3297-80d3-49642462ced6
> ValidateCsv Running
{code}
In the nifi-app logs we don't see issues indicating why component states get
out of sync. Typically, two nodes are in sync why the state of the third node
is different.
Stopping the Nifi service and deleting the flow.json.gz and flow.xml.gz files,
and starting the Nifi service again seems to align state again.
Does somebody have a clue what could be wrong or what is causing this?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
