Hello! I think these benefits are imaginary. You will have to worry about service more, rather about data streamer which may be recreated at any time.
Regards, -- Ilya Kasnacheev пн, 3 февр. 2020 г. в 16:58, narges saleh <[email protected]>: > Thanks Ilya. > I have to listen to these burst of data which arrive every few seconds > meaning an almost constant bursts of data from different data sources. > The main reason that the services grid is appealing to me is its > resiliency; I don't have to worry about it. With the client side streamer, > I will have to deploy it and keep it up running, and load/re balance it. > > On Mon, Feb 3, 2020 at 7:17 AM Ilya Kasnacheev <[email protected]> > wrote: > >> Hello! >> >> I don't see why you would deploy it as a service, sounds like you will >> have to send more data over network. If you have to pull batches in, then >> service should work. I recommend re-acquiring data streamer for each batch. >> >> Please note that Data Streamer is very scalable, so it is preferred to >> tune it than trying to use more than one streamer. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> пн, 3 февр. 2020 г. в 16:11, narges saleh <[email protected]>: >> >>> Hi Ilya >>> The data comes in huge batches of records (each burst can be up to >>> 50-100 MB, which I plan to spread across multiple streamers) so, the >>> streamer seems to be the way to go. Also, I don't want to establish a JDBC >>> connection each time. >>> So, if the streamer is the way to go, is it feasible to deploy it as a >>> service? >>> thanks. >>> >>> On Mon, Feb 3, 2020 at 6:51 AM Ilya Kasnacheev < >>> [email protected]> wrote: >>> >>>> Hello! >>>> >>>> Contrary to its name, data streamer is not actually suitable for >>>> long-lived, low-intensity streaming. What it's good for is burst load of >>>> large number of data in a short period of time. >>>> >>>> If your data arrives in large batches, you can use Data Streamer for >>>> each batch. If not, better use Cache API. >>>> >>>> If you are worried that plain Cache API is slow, but also want failure >>>> resilience, there's catch-22. The only way to make something resilient is >>>> to put it into cache :) >>>> >>>> Regards, >>>> -- >>>> Ilya Kasnacheev >>>> >>>> >>>> пн, 3 февр. 2020 г. в 14:34, narges saleh <[email protected]>: >>>> >>>>> Hi, >>>>> But services are by definition long lived, right? Here is my layout: >>>>> The data is continuously generated and sent to the streamer services (via >>>>> JDBC connection with set streaming on option), deployed, say, as node >>>>> singleton (actually deployed also as microservices) to load the data into >>>>> the caches. The streamers do flush data based on some timers. >>>>> If the streamer crashes before the buffer is flushed, the client >>>>> catches the exception and resends the batch. Any issue with this layout? >>>>> >>>>> thanks. >>>>> >>>>> On Mon, Feb 3, 2020 at 5:02 AM Ilya Kasnacheev < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello! >>>>>> >>>>>> It is not recommended to have long-lived data streamers, it's best to >>>>>> acquire it when it is needed. >>>>>> >>>>>> If you have to keep data streamer around, don't forget to flush() it. >>>>>> This way you don't have to worry about its queue. >>>>>> >>>>>> Regards, >>>>>> -- >>>>>> Ilya Kasnacheev >>>>>> >>>>>> >>>>>> пн, 3 февр. 2020 г. в 13:24, narges saleh <[email protected]>: >>>>>> >>>>>>> Hi, >>>>>>> My specific question/concern is with regard to the state of the >>>>>>> streamer when it run as a service, i.e. when it crashes and it gets >>>>>>> redeployed. Specifically, what happens to the data? >>>>>>> I have a similar question with regard to the state of a continuous >>>>>>> query when it is deployed as a service, what happens to the data in the >>>>>>> listener's queue? >>>>>>> >>>>>>> thanks. >>>>>>> >>>>>>> On Sun, Feb 2, 2020 at 4:18 PM Mikael <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi! >>>>>>>> >>>>>>>> Not as far as I know, I have a number of services using streamers >>>>>>>> without any problems, do you have any specific problem with it ? >>>>>>>> >>>>>>>> Mikael >>>>>>>> >>>>>>>> >>>>>>>> Den 2020-02-02 kl. 22:33, skrev narges saleh: >>>>>>>> > Hi All, >>>>>>>> > >>>>>>>> > Is there a problem with running the datastreamer as a service, >>>>>>>> being >>>>>>>> > instantiated in init method? Or loading the data via JDBC >>>>>>>> connection >>>>>>>> > with streaming mode enabled? >>>>>>>> > In either case, the deployment is affinity based. >>>>>>>> > >>>>>>>> > thanks. >>>>>>>> >>>>>>>
