I wanted to see if anyone else has experienced performance issues with the newest version of nifi and MergeRecord? We have been running on nifi 1.9.2 for awhile now, and recently upgraded to nifi 1.11.4. Once upgraded, our identical flows were no longer able to keep up with our data mainly at MergeRecord processors.
We ended up downgrading back to nifi 1.9.2. Once we downgraded, all was keeping up again. There were no errors to speak of when we were running the flow with 1.11.4. We did see higher load on the OS, but this may have been caused by the fact there was such a tremendous backlog built up in the flow. Another side note, we saw one UpdateRecord processor producing errors when I tested the flow with nifi 1.11.4 with a small test flow. I was able to fix this issue by changing some parameters in my RecordWriter. So perhaps some underlying ways records are being handled since 1.9.2 caused the performance issue we saw? Any insight anyone has would be greatly appreciated, as we very much would like to upgrade to nifi 1.11.4. One thought was switching the MergeRecord processors to MergeContent since I've been told MergeContent seems to perform better, but not sure if this is actually true. We are using the pattern of chaining a few MergeRecord processors together to help with performance. Thanks in advance!
