Hello! In streaming mode, INSERTs should not block, so you may reuse the connection. Still, it would not hurt to benchmark both approaches, if you can.
Regards, -- Ilya Kasnacheev сб, 2 мая 2020 г. в 01:42, narges saleh <[email protected]>: > Hi All, > If I use client connection configuration to set the number of threads for > a JDBC connection, and use the connection with multiple insert statements > (with streaming set to true), to multiple different caches, are the inserts > stacked because the connection is shared? Should I create multiple JDBC > connections and switch them across the statements? > thanks. > > On Thu, Apr 16, 2020 at 12:33 PM Evgenii Zhuravlev < > [email protected]> wrote: > >> As I said, if you use only DataStreamer, without jdbc, just a plain, >> key-value IgniteDataStreamer, then, you should have only one instance per >> cache. It will give you the better performance. This one streamer can be >> used from multiple threads. >> >> чт, 16 апр. 2020 г. в 09:54, narges saleh <[email protected]>: >> >>> I am sorry for mixing these two up. >>> I am asking if I were to use binaryobject builder with datastreamer in >>> place on jdbc connection would/should I create a pool of the streamer >>> objects. From your answers, the answer seems to be yes. thank you. >>> >>> On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev < >>> [email protected]> wrote: >>> >>>> You said that you use Binary Object Builder, so, I thought that you use >>>> key value API and data streamers. I don't really understand now you use >>>> BinaryObjectBuilder with thin JDBC client. >>>> >>>> >What if I have a persistent connection that sends data continuously? >>>> Should I hold on the instance of the streamer (for a particular cache), or >>>> recreate a new one once a new load of data arrives? >>>> If you're loading data continuously, it makes sense to store data >>>> streamer instance somewhere and just reuse it, avoiding recreating it each >>>> time. I >>>> >>>> >Are you saying have the data streamed to the streamer via multiple >>>> connections, across multiple threads? >>>> If you use just a simple IgniteDataStreamer, you can use it from >>>> multiple threads(use addData from multiple threads) to increase the >>>> throughput. >>>> Evgenii >>>> >>>> ср, 15 апр. 2020 г. в 12:07, narges saleh <[email protected]>: >>>> >>>>> Hello Evgenii, >>>>> >>>>> I am not sure what you mean by reuse a data streamer from multiple >>>>> threads. I have data being constantly "streamed" to the streamer via a >>>>> connection. Are you saying have the data streamed to the streamer via >>>>> multiple connections, across multiple threads? >>>>> What if I have a persistent connection that sends data continuously? >>>>> Should I hold on the instance of the streamer (for a particular cache), or >>>>> recreate a new one once a new load of data arrives? >>>>> >>>>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev < >>>>> [email protected]> wrote: >>>>> >>>>>> > Should I create a pool of data streamers (a few for each cache)? >>>>>> If you use just KV API, it's better to have only one data streamer >>>>>> per cache and reuse it from multiple threads - it will give you the best >>>>>> performance. >>>>>> >>>>>> Evgenii >>>>>> >>>>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <[email protected]>: >>>>>> >>>>>>> Please note that in my case, the streamers are running on the server >>>>>>> side (as part of different services). >>>>>>> >>>>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> So, in effect, I'll be having a pool of streamers, right? >>>>>>>> Would this still be the case if I am using BinaryObjectBuilder to >>>>>>>> build objects to stream the data to a few caches? Should I create a >>>>>>>> pool of >>>>>>>> data streamers (a few for each cache)? >>>>>>>> I don't want to have to create a new object builder and data >>>>>>>> streamer if I am inserting to the same cache over and over. >>>>>>>> >>>>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> For each connection, on node side will be created its own >>>>>>>>> datastreamer. I think it makes sense to try pooling for data load, >>>>>>>>> but you >>>>>>>>> will need to measure everything, since the pool size depends on the >>>>>>>>> lot of >>>>>>>>> things >>>>>>>>> >>>>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <[email protected]>: >>>>>>>>> >>>>>>>>>> Yes, Evgenii. >>>>>>>>>> >>>>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Do you use STREAMING MODE for thin JDBC driver? >>>>>>>>>>> >>>>>>>>>>> Evgenii >>>>>>>>>>> >>>>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <[email protected] >>>>>>>>>>> >: >>>>>>>>>>> >>>>>>>>>>>> Thanks Alex. I will study the links you provided. >>>>>>>>>>>> >>>>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite >>>>>>>>>>>> JDBC, (which should provide connection pooling). >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> At this point you need to implement connection pooling >>>>>>>>>>>>> yourself. >>>>>>>>>>>>> Use >>>>>>>>>>>>> >>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int- >>>>>>>>>>>>> to specify number of threads Ignite creates to service >>>>>>>>>>>>> connection >>>>>>>>>>>>> requests. >>>>>>>>>>>>> >>>>>>>>>>>>> Each new connection will be handled by a separate thread >>>>>>>>>>>>> inside >>>>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above) >>>>>>>>>>>>> >>>>>>>>>>>>> ClientConnectorConfiguration is set inside >>>>>>>>>>>>> IgniteConfiguration: >>>>>>>>>>>>> >>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration- >>>>>>>>>>>>> >>>>>>>>>>>>> More info: >>>>>>>>>>>>> >>>>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, Alex >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>>>>>>>>>>> >>>>>>>>>>>>
