David, I believe that flush() should throw an exception if any of the futures has been completed with error. You can also call fut.get() after assert isDone();
In this case get() should return immediately. --Yakov 2018-04-26 16:37 GMT+03:00 David Harvey <dhar...@jobcase.com>: > Thanks Yakov. I think this is more subtle. > > Our loading via IgniteDatastreamer is idempotent, but this depends on > being certain that a batch of work has successfully completed. It is > *not* sufficient for us to listen to the futures returned by addData, > then to call flush(), and then to record success if there have been no > exceptions. We must wait until get() is called on every future before > recording that the operation was successful. The fact that the future is > done is not sufficient, we need to know that it is done and there is no > exception. We can call flush and then do a future.get() on the incomplete > futures, but not as an assert. (It is valid to assert that fut.isDone(), > but that is not sufficient.) > > Based on by current understanding, I think this is a flaw in Ignite, even > if the fix might only be to clarify the comments for flush() to make this > behavior clear. > > -DH > > On Wed, Apr 25, 2018 at 5:43 PM, Yakov Zhdanov <yzhda...@apache.org> > wrote: > >> David, can you try adding all the futures to a collection and change >> assert condition to check that for all futures in that collection >> fut.isDone() is true? >> >> This should be a proper assertion. Please let me know if it works. >> >> Yakov Zhdanov >> >> чт, 26 апр. 2018 г., 0:30 David Harvey <dhar...@jobcase.com>: >> >>> We had assumed that if we did, in one thread: >>> IgniteFuture<?> f = streamer.addData(...); >>> f.listen(...) >>> >>> streamer.flush(); >>> >>> assert( all the prior futures have completed); << this triggered. >>> >>> >>> >>> I can't determine if this a bug, or just that the description could use >>> improvement. >>> >>> Inspecting the code it looks like the thread issuing the flush can be >>> awoken prior to the apply function for the addData future is called, since >>> GridFutureAdapter.unblockAll() will unpark or notify in a arbitrary >>> order. >>> >>> >>> *Disclaimer* >>> >>> The information contained in this communication from the sender is >>> confidential. It is intended solely for use by the recipient and others >>> authorized to receive it. If you are not the recipient, you are hereby >>> notified that any disclosure, copying, distribution or taking action in >>> relation of the contents of this information is strictly prohibited and may >>> be unlawful. >>> >>> This email has been scanned for viruses and malware, and may have been >>> automatically archived by *Mimecast Ltd*, an innovator in Software as a >>> Service (SaaS) for business. Providing a *safer* and *more useful* >>> place for your human generated data. Specializing in; Security, archiving >>> and compliance. To find out more Click Here >>> <http://www.mimecast.com/products/>. >>> >> > > > *Disclaimer* > > The information contained in this communication from the sender is > confidential. It is intended solely for use by the recipient and others > authorized to receive it. If you are not the recipient, you are hereby > notified that any disclosure, copying, distribution or taking action in > relation of the contents of this information is strictly prohibited and may > be unlawful. > > This email has been scanned for viruses and malware, and may have been > automatically archived by *Mimecast Ltd*, an innovator in Software as a > Service (SaaS) for business. Providing a *safer* and *more useful* place > for your human generated data. Specializing in; Security, archiving and > compliance. To find out more Click Here > <http://www.mimecast.com/products/>. >
