I am sorry for mixing these two up. I am asking if I were to use binaryobject builder with datastreamer in place on jdbc connection would/should I create a pool of the streamer objects. From your answers, the answer seems to be yes. thank you.
On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <[email protected]> wrote: > You said that you use Binary Object Builder, so, I thought that you use > key value API and data streamers. I don't really understand now you use > BinaryObjectBuilder with thin JDBC client. > > >What if I have a persistent connection that sends data continuously? > Should I hold on the instance of the streamer (for a particular cache), or > recreate a new one once a new load of data arrives? > If you're loading data continuously, it makes sense to store data streamer > instance somewhere and just reuse it, avoiding recreating it each time. I > > >Are you saying have the data streamed to the streamer via multiple > connections, across multiple threads? > If you use just a simple IgniteDataStreamer, you can use it from multiple > threads(use addData from multiple threads) to increase the throughput. > Evgenii > > ср, 15 апр. 2020 г. в 12:07, narges saleh <[email protected]>: > >> Hello Evgenii, >> >> I am not sure what you mean by reuse a data streamer from multiple >> threads. I have data being constantly "streamed" to the streamer via a >> connection. Are you saying have the data streamed to the streamer via >> multiple connections, across multiple threads? >> What if I have a persistent connection that sends data continuously? >> Should I hold on the instance of the streamer (for a particular cache), or >> recreate a new one once a new load of data arrives? >> >> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev < >> [email protected]> wrote: >> >>> > Should I create a pool of data streamers (a few for each cache)? >>> If you use just KV API, it's better to have only one data streamer per >>> cache and reuse it from multiple threads - it will give you the best >>> performance. >>> >>> Evgenii >>> >>> ср, 15 апр. 2020 г. в 04:53, narges saleh <[email protected]>: >>> >>>> Please note that in my case, the streamers are running on the server >>>> side (as part of different services). >>>> >>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <[email protected]> >>>> wrote: >>>> >>>>> So, in effect, I'll be having a pool of streamers, right? >>>>> Would this still be the case if I am using BinaryObjectBuilder to >>>>> build objects to stream the data to a few caches? Should I create a pool >>>>> of >>>>> data streamers (a few for each cache)? >>>>> I don't want to have to create a new object builder and data streamer >>>>> if I am inserting to the same cache over and over. >>>>> >>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev < >>>>> [email protected]> wrote: >>>>> >>>>>> For each connection, on node side will be created its own >>>>>> datastreamer. I think it makes sense to try pooling for data load, but >>>>>> you >>>>>> will need to measure everything, since the pool size depends on the lot >>>>>> of >>>>>> things >>>>>> >>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <[email protected]>: >>>>>> >>>>>>> Yes, Evgenii. >>>>>>> >>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Do you use STREAMING MODE for thin JDBC driver? >>>>>>>> >>>>>>>> Evgenii >>>>>>>> >>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <[email protected]>: >>>>>>>> >>>>>>>>> Thanks Alex. I will study the links you provided. >>>>>>>>> >>>>>>>>> I read somewhere that jdbc datasource is available via Ignite >>>>>>>>> JDBC, (which should provide connection pooling). >>>>>>>>> >>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> At this point you need to implement connection pooling yourself. >>>>>>>>>> Use >>>>>>>>>> >>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int- >>>>>>>>>> to specify number of threads Ignite creates to service >>>>>>>>>> connection >>>>>>>>>> requests. >>>>>>>>>> >>>>>>>>>> Each new connection will be handled by a separate thread inside >>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above) >>>>>>>>>> >>>>>>>>>> ClientConnectorConfiguration is set inside >>>>>>>>>> IgniteConfiguration: >>>>>>>>>> >>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration- >>>>>>>>>> >>>>>>>>>> More info: >>>>>>>>>> >>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration >>>>>>>>>> >>>>>>>>>> Thanks, Alex >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>>>>>>>> >>>>>>>>>
