[GitHub] [nifi] mattyb149 commented on a diff in pull request #6791: NIFI-6501: Refactor CaptureChangeMySQL to not use an unbounded event queue

2023-01-17 Thread GitBox 


mattyb149 commented on code in PR #6791:
URL: https://github.com/apache/nifi/pull/6791#discussion_r1072612630


##
nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQL.java:
##
@@ -433,7 +434,7 @@ public class CaptureChangeMySQL extends 
AbstractSessionFactoryProcessor {
 private BinlogLifecycleListener lifecycleListener;
 private GtidSet gtidSet;
 
-private final LinkedBlockingQueue queue = new 
LinkedBlockingQueue<>();
+private final BlockingQueue queue = new 
LinkedBlockingQueue<>(100);

Review Comment:
   I'll make it `1000` to help handle bursts of events,  but since it's now a 
true blocking queue, we shouldn't miss events and the processor should be 
draining the buffer as fast as the library is populating it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [nifi] mattyb149 commented on a diff in pull request #6791: NIFI-6501: Refactor CaptureChangeMySQL to not use an unbounded event queue

2022-12-22 Thread GitBox 


mattyb149 commented on code in PR #6791:
URL: https://github.com/apache/nifi/pull/6791#discussion_r1055910980


##
nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/processors/CaptureChangeMySQL.java:
##
@@ -433,7 +434,7 @@ public class CaptureChangeMySQL extends 
AbstractSessionFactoryProcessor {
 private BinlogLifecycleListener lifecycleListener;
 private GtidSet gtidSet;
 
-private final LinkedBlockingQueue queue = new 
LinkedBlockingQueue<>();
+private final BlockingQueue queue = new 
LinkedBlockingQueue<>(100);

Review Comment:
   Since we're blocking waiting on the event to be added to the queue, 
@markap14 suggested we wouldn't need a very large buffer at all. I'm thinking 
maybe a larger buffer size would be for times when the processor is scheduled 
for a longer period than it should be? I welcome your and Mark's comments here, 
just went with his suggestion :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org