Hi Dennis , Again thanks for you feedback and links these are very helpful.
Also it looks like prior to 2.8.0 the Java thin Client was thread safe but single threaded , but since 2.8.0 the Java thin client is Multithreaded, (I have a book that mentioned it was not multithread, which confused me). Is this correct and why you suggest I only need one connection per service? As a rule of thumb is it better more preformanent to use gets and puts as opposed to SQL , only use the SQL API if you need to do something that requires SQL? Regarding the 3 different caches problem , I did a bit of research on this and think I am good with this essentially by not using the try and autoclose with the igniteClient I can control when the igniteClient object is closed and therefore use it for all 3 Cache checks. Regards Eugene On Thu, Jun 11, 2020 at 10:57 PM Denis Magda <[email protected]> wrote: > Eugene, > > In such a case, I would go for the thin client for several reasons: > > - With the thick client (aka. standard client), you will come across > several limitations if the servers are deployed on VMs, but the thick > clients are running in a K8S environment. Those limitations are to be > addressed soon [1]. > - With the partition-awareness feature [2], the thin client will be > able to send requests to those server nodes that store the primary copy of > a requested record. The functionality should be released in Ignite 2.9 and > already exists in GridGain. > > I don't think that you need to be bothered about a connection pool. Just > have a single thin client connection per service instead and enable the > partition-awareness. If you don't use the latter feature, then ensure that > the thin clients on those service instances are connected to random proxy > servers (you need to pass different IPs of the server nodes in the > connection string parameter [3]). > > One thing I am not clear on is in the scenario where we need to check 3 >> different caches, for a single call to our service, is this going to be an >> expensive operation? >> Will we have to initiate a connection to Ignite for every call to the >> cache? > > > To help with this, I need to understand your business use case better. > There is a chance you can do all the checks with one network hop. > > [1] > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html > [2] > https://www.gridgain.com/docs/latest/developers-guide/thin-clients/getting-started-with-thin-clients#partition-awareness > [3] > https://apacheignite.readme.io/docs/java-thin-client-initialization-and-configuration#initialization > > - > Denis > > > On Thu, Jun 11, 2020 at 12:12 AM Eugene McGowan <[email protected]> > wrote: > >> Thanks for getting back to me on this Dennis, >> The company's Ignite Cluster is already up and running on VMs on our >> company network. The service my team is standing up will run in our Private >> cloud K8S environment. >> >> Regarding our usecase we will have say 2 or 3 caches, our service will be >> passed 3 parameters , if it gets a match to parameter 1 in cache 1 no >> further action is required , if not it needs to check cache 2 for parameter >> 2 ,again if it gets a match it uses data from cache 2 and no further >> action is required , otherwise it needs to check cache 3 . So cache 1 has >> most relevant data. >> We would have somewhere between 2 and 10 instances of our service running >> servicing a max of 3000+ request per second >> >> Our preference if possible would be to use a thin client. One thing I am >> not clear on is in the scenario where we need to check 3 different caches, >> for a single call to our service, is this going to be an expensive >> operation? >> Will we have to initiate a connection to Ignite for every call to the >> cache? From reading connection pooling does not come out of the box with >> key value operations over thin client, but its possible to implement your >> own connection pooling, is it therefore possible to have long lived >> connections to avoid the expense of constantly connecting and disconnecting >> from Ignite caches. >> >> Regarding the possibility of using the SQL API the background that is: >> (1) We have a rarely used usecase where we want to get a >> combination of data from all 3 caches , so was thinking the SQL API would >> be one way of doing that. >> (2) Looking at the docs it seems that connection pooling is >> possible via JDBC over thin client , so was wondering would that be a more >> performant approach, if it's not possible to have a connection pool for key >> value operations over the thin client. >> >> Regards Eugene >> >> On Thu, Jun 11, 2020 at 12:28 AM Denis Magda <[email protected]> wrote: >> >>> Hi Eugene, >>> >>> Let me help you with that as much as I can. Please help me understand >>> the following: >>> >>> - Is the cluster supposed to be deployed outside of Kubernetes (on >>> VMs or bare-metal)? Is this a private cloud (OpenShift, PCF) or a public >>> environment (Azure, AWS, etc.)? >>> - If you use primary keys to access the records, do you really need >>> SQL for this type of operations? Or do you perform additional filtering >>> of >>> records with SQL operands? >>> >>> In the meantime, please check this page to understand the pros and cons >>> of thick and thin clients: >>> https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#thick-vs-thin-clients >>> >>> - >>> Denis >>> >>> >>> On Wed, Jun 10, 2020 at 1:07 PM Eugene McGowan <[email protected]> >>> wrote: >>> >>>> Hi Igniters, >>>> Our company has recently started using Ignite and our team are now to >>>> it. >>>> We would like to use Ignite as a Cache layer for an upcoming project >>>> >>>> Our use case is that we have an Ignite cluster that we want to interact >>>> with , our client service is Springboot based and running in a Kubernetes >>>> Cluster but not co located with our Ignite cluster , we anticipate our >>>> service will receive *3000+ request per second* , the service will >>>> scale based on the volume of requests, and we will need to query Ignite for >>>> each of these requests. >>>> The data we need to query is fairly straight forward , a single row >>>> from either 1 or 2 caches based on primary key. >>>> We are at the point where we are beginning to design the architecture. >>>> >>>> I have done some initial reading on Ignite and was wondering if someone >>>> could >>>> Verify my understanding is correct. >>>> >>>> I would also be interested to know what architectures other folks are >>>> using for similar use cases >>>> >>>> *=======* >>>> *Thin Client* >>>> ======== >>>> JDBC connection are not thread safe >>>> https://apacheignite-sql.readme.io/docs/jdbc-driver >>>> To use this approach we need to ensure non cuncurrent use of >>>> Connections, Statements, and ResultSet perhaps using a locking mechanism. >>>> >>>> Key value operations or SQL Queries via the API over the thin client >>>> are thread safe >>>> Connection Pooling is not provided via the API , but it is possible >>>> to implement your own connection pool. >>>> If you implement your own connection pool you could keep connections >>>> alive to reduce >>>> the cost of establishing a connection to Ignite and reusing >>>> the connections for various requests. >>>> >>>> >>>> ========== >>>> *Thick Client * >>>> ========== >>>> >>>> Does not have a concept of Connection pooling >>>> >>>> When using either JDBC or the IGNITE API on the Thick Client >>>> It is thread safe and can handle multiple concurrent threads >>>> >>>> Is part of the Ignite cluster >>>> Ignite Servers can iniatate TCP connection needs to be reachable via >>>> a TCP connection >>>> If the Client is behind a firewall,NAT or Load Balancer it makes >>>> using ,thick client challenging >>>> >>>> >>>> >>>> ========= >>>> *Question on GRPC* >>>> ============= >>>> >>>> The Thin Client TCP socket connection uses a Binary protocol >>>> https://apacheignite.readme.io/docs/binary-client-protocol >>>> >>>> I may be mixing up network levels here but , is it possible to run this >>>> over a GRPC connection >>>> >>>
