Hi Devs, I'm working on an experiment, in which I have a socket feed receives data at a very low rate (1 record/second). In data feeds, when data is not coming fast enough, the flow controller will try to force a flush before it heads into waiting. However, I found the data was not flushed as I expected in my experiment.
After some investigation, I pinpointed a "potential suspect". In the "flushAndReset" method of the AbstractOneInputOneOutputOneFramePushRuntime class [1], the record appender writes data records into its writer, but that writer is not flushed. Thus, the data records are not flushed as expected, even when the "flushFramesRapidly" flag is on. After adding a writer.flush() call, everything runs as expected. This feels like a bug to me. Since this class touches many runtime factories, and it may have impacts on the performance, I want to double check with you guys before I submit the fix. Thoughts? [1] org/apache/hyracks/algebricks/runtime/operators/base/AbstractOneInputOneOutputOneFramePushRuntime.java:74 Best, Xikui
