Thanks, David. I've created a ticket [1] to update javadoc for flush().
Behavior you describe looks rather natural. flush() should not track all previously completed add operations. So, one of the possible ways to ensure all data were added is to check whether flush() finishes without an exception, and then check if get() doesn't throw for each addData future (this part won't block). [1] https://issues.apache.org/jira/browse/IGNITE-8406 2018-04-26 20:47 GMT+03:00 David Harvey <[email protected]>: > "Could you please refine what kind of flaw do you suspect? " > I think the basic flaw is that it is unclear how to write a simple task to > stream in some data, and then confirm that all of that data was > successfully stored before taking an action to record that. This may be > simply a documentation issue for flush(), where I can't tell what the > intended design would be for such code. > > We ran into this issue because we assumed that we needed to test the > status of all of the futures returned by addData, and we found that the > listener was not always called before flush() returned. > > As I dig deeper into the code, I see an attempt to cause flush() to throw > an exception if any exception was thrown on the server for *any* prior > record. If that is the intent (which is not stated but would be > convenient), then I think there is a race: > > - DataStreamerImpl.flush() calls EnterBusy while activeFuts is non > empty. This seems to be the last test of "cancelled". If there were > failed requests before this, flush() would throw. > - Before doFlush() looks at activeFuts, activeFuts becomes empty, > because the requests failed. > - flush() returns without throwing an exception. > > > -DH > > -- Best regards, Andrey Kuznetsov.
