Antti-Kaikkonen opened a new issue #6317: A source connector created with --processing-guarantees EFFECTIVELY_ONCE produces messages one at a time. URL: https://github.com/apache/pulsar/issues/6317 **Describe the bug** Creating a source connector will make pulsar always wait for the previous message to be acknowledged before producing the next message. This is caused by this [future.join()](https://github.com/apache/pulsar/blob/e1f3409fd45b26bafb44c321f99f853ebbc50109/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/sink/PulsarSink.java#L237). **To Reproduce** Steps to reproduce the behavior: 1. Run [this](https://github.com/Antti-Kaikkonen/PulsarSequenceSource/blob/master/src/main/java/SequenceSource.java) test souce connector with `--processing-guarantees EFFECTIVELY_ONCE` 2. Run the same source connector with `--processing-guarantees ATLEAST_ONCE` 3. Read the logs to see how much higher the throughput is with ATLEAST_ONCE. For me the throughput with EFFECTIVELY_ONCE was only 90 ack/s and over 300 000 ack/s with ATLEAST_ONCE. **Expected behavior** Throughput with `--processing-guarantees EFFECTIVELY_ONCE` should be almost as high as with `--processing-guarantees ATLEAST_ONCE`. **Desktop (please complete the following information):** - OS: Ubuntu 18.04 - pulsar 2.5.0 standalone **Additional context** I haven't tested but I think that this issue should affect regular pulsar functions as well. I have asked @merlimat why the [future.join()](https://github.com/apache/pulsar/blob/e1f3409fd45b26bafb44c321f99f853ebbc50109/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/sink/PulsarSink.java#L237) is required, but he didn't remember why it was added and asked me to open an issue.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
