Hello! Contrary to its name, data streamer is not actually suitable for long-lived, low-intensity streaming. What it's good for is burst load of large number of data in a short period of time.
If your data arrives in large batches, you can use Data Streamer for each batch. If not, better use Cache API. If you are worried that plain Cache API is slow, but also want failure resilience, there's catch-22. The only way to make something resilient is to put it into cache :) Regards, -- Ilya Kasnacheev пн, 3 февр. 2020 г. в 14:34, narges saleh <[email protected]>: > Hi, > But services are by definition long lived, right? Here is my layout: The > data is continuously generated and sent to the streamer services (via JDBC > connection with set streaming on option), deployed, say, as node singleton > (actually deployed also as microservices) to load the data into the caches. > The streamers do flush data based on some timers. > If the streamer crashes before the buffer is flushed, the client catches > the exception and resends the batch. Any issue with this layout? > > thanks. > > On Mon, Feb 3, 2020 at 5:02 AM Ilya Kasnacheev <[email protected]> > wrote: > >> Hello! >> >> It is not recommended to have long-lived data streamers, it's best to >> acquire it when it is needed. >> >> If you have to keep data streamer around, don't forget to flush() it. >> This way you don't have to worry about its queue. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> пн, 3 февр. 2020 г. в 13:24, narges saleh <[email protected]>: >> >>> Hi, >>> My specific question/concern is with regard to the state of the streamer >>> when it run as a service, i.e. when it crashes and it gets redeployed. >>> Specifically, what happens to the data? >>> I have a similar question with regard to the state of a continuous query >>> when it is deployed as a service, what happens to the data in the >>> listener's queue? >>> >>> thanks. >>> >>> On Sun, Feb 2, 2020 at 4:18 PM Mikael <[email protected]> wrote: >>> >>>> Hi! >>>> >>>> Not as far as I know, I have a number of services using streamers >>>> without any problems, do you have any specific problem with it ? >>>> >>>> Mikael >>>> >>>> >>>> Den 2020-02-02 kl. 22:33, skrev narges saleh: >>>> > Hi All, >>>> > >>>> > Is there a problem with running the datastreamer as a service, being >>>> > instantiated in init method? Or loading the data via JDBC connection >>>> > with streaming mode enabled? >>>> > In either case, the deployment is affinity based. >>>> > >>>> > thanks. >>>> >>>
