Re: Apache Ignite Sink Connector

[Alexandr Shapkin]

Hello!

Please share the exception and attach Ignite logs. 

> On 27 Aug 2021, at 15:12, Shubham Shirur  wrote:
> 
> Hi, 
> 
> Connector is giving me error while running in distributed mode. But running 
> in standalone.
> 
> Can you please highlight?
> 
> Thanks
> Shubham

Re: ignite dump

[Alexandr Shapkin]

Hello!

What Ignite version are you running? 

I can see that a similar issue has been fixed in 2.8 
https://issues.apache.org/jira/browse/IGNITE-11755

> On 31 Aug 2021, at 19:22, DK  wrote:
> 
> Hi Team,
> 
> Any pointers on this, please?
> 
> Seems internally H2StatementCache contains a huge set of data that were 
> queried over a period of time.
> 
> Thanks & Regards,
> Devakumar J
> 
>  
> 
> Virus-free. www.avast.com 
> 
>  
> On Sun, Aug 29, 2021 at 7:37 PM DK  > wrote:
> Hi,
> 
> One of our server node's heap usage keeps increasing by uptime and finally 
> ends up in node restart.
> 
> When i checked the heap dump detachedConns holds up huge memory around 12GB 
> out of 16GB and GC also couldn't reclaim. This happens randomly in one of the 
> server node only (3 nodes running).
> 
> Is this related to releasing or closing the thin client connection?
> 
> 
> 
> Thanks & Regards,
> Devakumar J
> 
> 
> -- 
> Thanks & Regards
> 
> Devakumar J

Re: JvmClasspath value in IgniteConfiguration

[Alexandr Shapkin]

Hello!

That’s correct, Ignite .NET server/thick client is JVM-dependent and requires 
*.jar modules to be present in addition to the .NET dependencies. For console 
and test deployments usually, it’s ok to rely on internal $IGNITE_HOME 
resolution. By default, Ignite tries to locate it starting from the current 
application folder (from where your application is running) moving towards the 
disk root up the folders hierarchy until it managed to find “/libs” folder or 
throws an error.

But this won’t work with IIS deployment because of a different startup logic 
thus the most convenient way in this setup is just to set $IGNITE_HOME 
explicitly. Refer to the following docs for more IIS-related details:
https://ignite.apache.org/docs/latest/net-specific/net-deployment-options#asp-net-deployment



> On 21 Aug 2021, at 01:56, Josh Katz  
> wrote:
> 
> When using Ignite.Start from a Web client in .NET - What should be the value 
> of the JvmClasspath in the configuration?
> It looks like IIS cannot find the JVM so the log stops at ‘JVM Started’ after 
> trying to resolve the JVM Classpath.
> When used in a unit test or executable we are not facing this issue so it 
> seems specific to IIS and JVM.
>  
> Thanks,
> Josh
>  
> Please follow the hyperlink to important disclosures.
> https://www.dodgeandcox.com/disclosures/email_disclosure_funds.html 
> 

Re: rebalancing & K8

[Alexandr Shapkin]

Hello!

It's recommended to set liveness and readiness probes, though the defaults
should be fine. 

As per rebalancing, you might want to set
-DIGNITE_WAIT_FOR_BACKUPS_ON_SHUTDOWN=true meaning, that k8s can't restart a
pod if this could lead to a data loss.

You might check the Apache Ignite operator as well here
https://www.gridgain.com/docs/latest/installation-guide/operator/overview



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: slow node discovering on Kubernetes

[Alexandr Shapkin]

Hi Maxim, Not really, I haven’t used flannel plugin and I’m not quite sure about network recommendations as well. Well, besides something basic, like avoid stretching between multiple availability zones and some persistence tuning,Like disabling MMAP for WAL (IGNITE_WAL_MMAP=false) > The problem still actual, we deployed same 10 nodes on google kubernetes and>got a normal time. Do you mean, that switching on GKE makes it working or it was the initial setup and nothing has changed since that? > For now we did not define any limits or requests, our ignite is the onlydeployment on the kubernetes. I was wondering cause if I remember correctly there might be some issues if your pods have insufficient resources, but unfortunately, no direct numbers. Well, default GKE instances should works ok on a default cluster, anyway. > Do you have any idea how to estimate network or hardware performance to find>possible bottlenecks? I think enabling DEBUG logs for discovery might help to see what’s really happening to the grid.      category> Btw, is it a persistent cluster or pure in-memory one?  From: mvolkomorovSent: Friday, April 9, 2021 10:26 AMTo: user@ignite.apache.orgSubject: RE: slow node discovering on Kubernetes Hello, Alexandr! The problem still actual, we deployed same 10 nodes on google kubernetes andgot a normal time.For now we did not define any limits or requests, our ignite is the onlydeployment on the kubernetes.We use flannel network plugin (host-gw), are there any recommendations forthe network plugin?Do you have any idea how to estimate network or hardware performance to findpossible bottlenecks?   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: slow node discovering on Kubernetes

[Alexandr Shapkin]

Hello Maxim, Could you please share the current state of this issue? Have you managed to resolve it or it still exists? What is your pods configuration in terms of resource usage? From: Maxim VolkomorovSent: Thursday, March 11, 2021 11:44 PMTo: user@ignite.apache.orgSubject: slow node discovering on Kubernetes Hi Ignite team,We're running an Ignite cluster with 10 server nodes at Kubernetes.Using empty ignite configuration we can't run more than 5 nodes in a normal time.Trying to deploy 10 nodes on our empty config leads to weird discovery problems caused by "IgniteSpiException: Node with the same ID was found".  After increasing the AckTimeout=1 and switching to G1GC, cluster was started, but time still long and getting TcpDiscoverySpi errors. I checked connections and port availability between nodes and found nothing suspicious.I've attached a log with one of 10 nodes deploying.Our ignite config is default-config.xml with only TcpDiscoveryKubernetesIpFinder.Would you like to take a look and give us some suggestions on how to reduce deploy time?Thanks,Maxim 

RE: Ignite best practice for restarting k8s pod

[Alexandr Shapkin]

Hi,  Thanks for the ticket. I’ll check that.  What configuration do you use, is it a persistent cluster or in-memory one?Do you use an official Ignite image or a custom assembly? 'kubectl delete pod' should work fine here and forward KILL SIG to a running node internally. Well, at least starting from 2.9 version.  From: vbmSent: Wednesday, January 13, 2021 11:32 AMTo: user@ignite.apache.orgSubject: Ignite best practice for restarting k8s pod Hi, I had raised this ticket:https://issues.apache.org/jira/browse/IGNITE-13974  Currently I do not see any cleanup functions getting called when we do a'kubectl delete pod'. May I know, what is the best practice for restarting k8s ignite pod ?How do we handle scenario when we need to scale down Ignite pods ? I thinkinternally when we do kubectl scale down it calls kubectl delete pod.  Regards,Vishwas   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Error:data-streamer-stripe....Timed out while waiting for schema update.

[Alexandr Shapkin]

Hi Siva, Nodes started on local machine require more than 80% of physical RAM whatcan lead to significant slowdown due to swapping (please decrease JVM heapsize, data region size or checkpoint buffer size) [required=78152MB,available=65535MB] According to the warning, you don’t have enough heap size (65 MB). Try increasing it and check the result. From: sivaSent: Thursday, January 14, 2021 7:21 AMTo: user@ignite.apache.orgSubject: Error:data-streamer-stripeTimed out while waiting for schema update. Hi,I have .Net ClientServer Ignite Application.I am using DataStreamer to loaddata into ignite caches. while loading data client node getting stop with stackoverflow message .Andon server node displaying message on console likeError:data-streamer-stripeTimed out while waiting for schema update.  And while again starting client node getting the following message onconsole. Nodes started on local machine require more than 80% of physical RAM whatcan lead to significant slowdown due to swapping (please decrease JVM heapsize, data region size or checkpoint buffer size) [required=78152MB,available=65535MB] what might be the issue and how to solve? Any other inform needed please let me know.Thanks.   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

Re: Query execution is too long

[Alexandr Shapkin]

Hi Siva,

Have you checked the SQL tuning documentation?

https://www.gridgain.com/docs/latest/perf-troubleshooting-guide/sql-tuning



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Ignite rebalancing when a server is rebooted w/ persistance enabled.

[Alexandr Shapkin]

Hi, Looks like the error message is truncated. Could you please re-send it or attach the full log file? PartitionsEvictManager is part of rebalancing routine and it clears local data before demanding it from other nodes. Also I see the following JIRA https://issues.apache.org/jira/browse/IGNITE-11974  From: maxi628Sent: Thursday, January 14, 2021 12:16 AMTo: user@ignite.apache.orgSubject: Ignite rebalancing when a server is rebooted w/ persistance enabled. Hello everyone. I have several ignite clusters with version 2.7.6 and persistence enabled.I have a 3 caches on every cluster, with 10M~ records each. Sometimes when I reboot a node, it takes a lot of time to boot, it can behours. With rebooting I mean stopping the container that's running ignite andstarting it again, without ever changing the baseline topology, it can take2 minutes to restart the container.The node joins the topology just fine but takes a long time to start servingtraffic. Checking the logs I've found that there are several lines like this oneshere:   So for some reason after booting it starts a process calledPartitionsEvictManager, which can take a lot of time.What is the intended functionality behind PartitionsEvictManager?It is something that we should expect? This is a problem because a rolling restart of all nodes in a cluster cantake up to a day. Thanks.--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: incorrect partition map exchange behaviour

[Alexandr Shapkin]

Hi, As you correctly pointed to the PME implementation details webpage, this is a process of exchanging information about partition holders. And it’s happening on every topology change, cluster deactivation, etc. The process itself is not about data rebalancing, it’s about what node should store a particular partition.  If you want to check whether the data rebalance happened you need to find something like  [2020-01-15 15:46:57,042][INFO ][sys-#50][GridDhtPartitionDemander] Starting rebalance routine [ignite-sys-cache, topVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], supplier=9e88a103-4465-4e5b-865f-4edaa909fee1, fullPartitions=[0-99], histPartitions=[]] It also depends on whether your cluster is under load during the rolling upgrade, if there are no updates happening then no data rebalance should happen as well.  I’m not pretty sure about the metric and visor. Anyway you can perform the checks explicitly from code: ignite.cache("myCache").localSize(CachePeekMode.BACKUP);ignite.cache("myCache").localSize(CachePeekMode.PRIMARY);   From: tschauenbergSent: Friday, January 8, 2021 3:59 AMTo: user@ignite.apache.orgSubject: incorrect partition map exchange behaviour Hi, We have a cluster of Ignite 2.8.1 server nodes and have recently startedlooking at the individual cache metrics for primary keysorg.apache.ignite.internal.processors.cache.CacheLocalMetricsMXBeanImpl.OffHeapPrimaryEntriesCount In our configuration we have a replicated cache with 2 backups.  Our clusterhas 3 nodes in it so the primaries should be spread equally on the 3 nodesand each node has backups from the other two nodes.  All these server nodesare in the baseline.  Additionally we have some thick clients connected butI don't think they are relevant to the discussion. Whenever we do a rolling restart one node at a time, at the end after thelast node is restarted it always owns zero primaries and owns solelybackups.  The two nodes restarted earlier during the rolling restart own allthe primaries. When our cluster is in this scenario, if we start and stop visor, when visorleaves the cluster it triggers a PME where all keys get balanced on allserver nodes.  Looking at the visor cache stats between the start and stopwe can see a min of 0 keys on the nodes for our cache so visor and the jmxmetrics line up on that front.  After stopping visor, the jmx metrics showthe evenly distributed primaries and then starting visor a second time wecan confirm that again the min, average, max node keys are all evenlydistributed. Every join and leave during the rolling restart and during visor start/stopshows reflects a topology increment and node leave and join events in thelogs.   According tohttps://cwiki.apache.org/confluence/display/IGNITE/%2528Partition+Map%2529+Exchange+-+under+the+hoodeach leave and join should trigger the PME but we only see the keys changingon the leaves. Additionally, we tried waiting longer between the stop and start part of therolling restart to see if that had any effect.  We ensured we waited longenough for a PME to do any moving but waiting longer didn't have any effect. The stop always has the PME move the keys off that node and the start neversees the PME move any primaries back. Why are we only seeing the PME change keys when nodes (server or visor) stopand never when they join?   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Critical Workers Health Check on client side

[Alexandr Shapkin]

Hi, It’s too internal details and it’s not possible to catch *Critical Workers HealthCheck* on client side.  I think you might want to listen for Ignite events on your clients and apply your custom logic accordingly. The most essential candidate - EVT_NODE_FAILED You might check more available events here: https://ignite.apache.org/docs/latest/events/events and here:https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/events/EventType.html   From: ihalilaltunSent: Monday, January 4, 2021 2:52 PMTo: user@ignite.apache.orgSubject: Critical Workers Health Check on client side hi there, I am curious about whether we can manage somehow *Critical Workers HealthCheck*on client side? What i need to do is catch critical workers healthcheck results on client side, can this be done by implementing customStopNodeOrHaltFailureHandler on client side? We are on ignite v2.7.6 thanks   -İbrahim Halil AltunSenior Software Engineer @ Segmentify--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Explicit Communication / Discovery port configuration on the client side needed ?

[Alexandr Shapkin]

Hi, You don’t have to specify the communication settings explicitly, usually it’s ok to leave the defaults. This is indeed your case, if one port is already occupied, the next one is chosenYou can reduce it directly in your node configuration. But it’s static configuration and applicable only for a given node. I.e. if you need to adjust it for server then you should modify your server side configuration.   From: VeenaMithareSent: Tuesday, January 12, 2021 4:50 PMTo: user@ignite.apache.orgSubject: Explicit Communication / Discovery port configuration on the client side needed ? Hi ,  1. If we deploy more than one client on same machine - should we explicitlyconfigure communication and discovery port for the client i.e. should we dothe below :         class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">          2. The localPortRange for communicationport and discovery port is 100 . Whyis this range needed ? 3. Can the port range be reduced to 5 on the client side configuration ? regards,Veena--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Apache Ignite C#/.NET: Asynchronous Execution example failing - facing different errors

[Alexandr Shapkin]

Hi, This is kind of expected, cause you don’t keep your client running.  Try adding something like Console.ReadLline() at the end of your client’s main method. When a task initiator lefts from the grid, there is no way to send the results back, therefore depending on NODE_FAILED timing your either get: #1 – “Topology projection is empty”#2 – “Task cancelled due to stopping of the grid”  From: adumalaganSent: Monday, December 7, 2020 2:14 AMTo: user@ignite.apache.orgSubject: Apache Ignite C#/.NET: Asynchronous Execution example failing - facing different errors Hello, I tried to follow the Asynchronous Execution example in the Apache Ignite docs, however I ran into some errors, and the errors vary to the different implementations I have. Variation #1 I follow the example in the link above exactly - the ServerNode is run first then the ClientNode_1. From the server console, I see that the client connects with the server, then the client immediately disconnects. Finally, a whole slew of errors follow. Variation #2 ServerNode remains the same - instead, I run ClientNode_2. In this variation, I run into these errors. I was wondering what causes these specific errors for these specific variations and how I can resolve them - thank you!  Sent from the Apache Ignite Users mailing list archive at Nabble.com. 

RE: With as syntax does not work in ignite 2.9.0?

[Alexandr Shapkin]

Hi, I’ve filed a JIRA ticket https://issues.apache.org/jira/browse/IGNITE-13822 to track this issue.Full logs or a simple reproducer would help to debug this more quickly, feel free to share them as well. From: yangjiajunSent: Monday, December 7, 2020 5:40 AMTo: user@ignite.apache.orgSubject: With as syntax does not work in ignite 2.9.0? Hello. We use 'with xxx as  (select xxx) ',which works vrey fine in 2.8.1 and otherpast release versions.After we uprade to 2.9.0,such sqls start to throwexception. In the server side,the error looks like: , args=Object[] [], stmtType=SELECT_STATEMENT_TYPE, autoCommit=true,partResReq=false, super=JdbcRequest [type=2, reqId=790418]]]class org.apache.ignite.internal.processors.query.IgniteSQLException: Failedto parse query. General error: "java.lang.NullPointerException" [5-197]  atorg.apache.ignite.internal.processors.query.h2.H2Connection.prepareStatementNoCache(H2Connection.java:194)  atorg.apache.ignite.internal.processors.query.h2.H2PooledConnection.prepareStatementNoCache(H2PooledConnection.java:109)  atorg.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:355)  atorg.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:222)  atorg.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:138)  atorg.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1071)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2779)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2775)  atorg.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:3338)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$2(GridQueryProcessor.java:2795)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2833)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2769)  atorg.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2727)  atorg.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:647)  atorg.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:320)  atorg.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:257)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:202)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:56)  atorg.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)  atorg.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)  atorg.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)  atorg.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)  atorg.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)  atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:745)Caused by: org.h2.jdbc.JdbcSQLException: General error:"java.lang.NullPointerException" [5-197]  at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)  at org.h2.message.DbException.get(DbException.java:168)  at org.h2.message.DbException.convert(DbException.java:307)  at org.h2.message.DbException.toSQLException(DbException.java:280)  at org.h2.message.TraceObject.logAndConvert(TraceObject.java:357)  at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:697)  atorg.apache.ignite.internal.processors.query.h2.H2Connection.prepareStatementNoCache(H2Connection.java:191)  ... 26 moreCaused by: java.lang.NullPointerException  at org.h2.table.Table.removeChildrenAndResources(Table.java:545)  at org.h2.table.TableView.removeChildrenAndResources(TableView.java:439)  at org.h2.engine.Session.removeLocalTempTable(Session.java:409)  at org.h2.command.Parser.parseSingleCommonTableExpression(Parser.java:5233) 

RE: Implementing HA among Ignite Clusters

[Alexandr Shapkin]

Hi, I don’t think that ignite support this out of the box, cause data replication is something beyond the default distribution. The usage depends on how your data is replicated, whether you want active-active synchronization, and so on.  But if it’s about switching from one cluster to another, and always using a single cluster at a time, something like dynamic DNS or a custom load balancer should work here. I.e. you can configure your consumers to access domain my.cluster.com that resolves to the primary cluster, IP=192.168.1.1. When you want to switch the load, you can adjust your DNS to resolve the nameto, say, IP=192.168.1.13 which means the secondary one. This approach has some drawbacks but is quite simple to use and straightforward. From: NaveenSent: Sunday, December 6, 2020 2:47 PMTo: user@ignite.apache.orgSubject: Implementing HA among Ignite Clusters Hi All We have 2 clusters (Primary and secondary)  running and both are always insync from data perspective. Always consumers are using Primary only, but when primary is not reachable,it should get redirected to secondary cluster How can this be achieved ? 1. Can we do anything on the cluster side which will take care of this, likewith config changes on the primary cluster like changing the connectonstring etc.Like some of the messaging products uses connection URLs liketcp://server1||tco://server2something like this, if server1 is not reachable, it will get routed toserver2.Somehing like this can be done here ?2. DOing this from consumer side only, as and when they get exceptions likedisconnect etc, try connecting it to secondary and vice versa What are the options we have for this scenario ThanksNaveen   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: WAL is growing exponentially

[Alexandr Shapkin]

Hi, What version do you use? Is it possible to share the full log files?  From: rihan123Sent: Friday, December 4, 2020 2:32 PMTo: user@ignite.apache.orgSubject: WAL is growing exponentially HiI am doing POC to see if Ignite is good for my usecase and i achievedrequired data ingestion rate but after ingesting huge data i am seeing WALdisk is growing exponentially.To check the number of records ingested i just connected to visor after thisi saw WAL is growing exponentially.There are lot of messeges like this --  Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map sizeis 4Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map sizeis 5Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map sizeis 6Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=c1f85a50-c218-4e38-a692-0aa87380d295, timestamp=1606979161926,ptr=FileWALPointer [idx=1427, fileOff=51216143, len=9667]], history map sizeis 7Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history mapsize is 2Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history mapsize is 4Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=5b4cfe31-99e6-4c00-b306-9c590df8fdfb, timestamp=1606979174328,ptr=FileWALPointer [idx=1520, fileOff=59350658, len=10066]], history mapsize is 5  Any suggestions on this issue is greatly appreciated   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Could not clear historyMap due to WAL reservation on cp

[Alexandr Shapkin]

Hi, Could you share the full logs with the exception? There is an interesting  JIRA ticket that could be the reason for this [1] [1] - https://issues.apache.org/jira/browse/IGNITE-13373   From: shivakumarSent: Thursday, December 3, 2020 10:51 AMTo: user@ignite.apache.orgSubject: Could not clear historyMap due to WAL reservation on cp Hi I have deployed 5 node Ignite cluster on K8S with persistence enabled(version 2.9.0 on java 11)I started ingesting data to 3 tables and after ingesting large amount ofdata using JDBC batch insertion (around 20 million records to each of 3tables with backup set to 1), now i connected to visor shell (from one ofthe pod which i deployed just to use as visor shell) using the same igniteconfig file which is used for ignite servers and after visor shell connectsto ignite cluster the unwanted wal record cleanup stopped (which should runpost checkpoint ) and WAL started growing linearly as there is continuousdata ingestion. This is making WAL disk run out of space and pods crashes. when i see logs there are continuous warning messages saying Could not clear historyMap due to WAL reservation on cp: CheckpointEntry[id=e8bb9c22-0709-416f-88d6-16c5ca534024, timestamp=1606979158669,ptr=FileWALPointer [idx=1468, fileOff=45255321, len=9857]], history map sizeis 4  and if i see checkpoint finish messages  Checkpoint finished [cpId=ca254956-5550-45d6-87c5-892b7e07b13b,pages=494933, markPos=FileWALPointer [idx=1472, fileOff=72673736, len=9857],walSegmentsCleared=0, walSegmentsCovered=[1470 - 1471], markDuration=464ms,pagesWrite=8558ms, fsync=3597ms, total=14027ms] here you can see walSegmentsCleared=0   means there are no WAl segmentscleared even after checkpoint, not sure what is causing this behaviour. we are ingesting very large data (~25mb/s)please someone help in this issue   regards,Shiva --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Failed to prepare update plan error while inserting data, when select is used aong with insert

[Alexandr Shapkin]

Hi, That’s odd, could you please share the logs with  the error?   From: harinathSent: Wednesday, November 18, 2020 6:52 PMTo: user@ignite.apache.orgSubject: RE: Failed to prepare update plan error while inserting data, when select is used aong with insert Hi Alexandar, I have tried without parentheses, still i see the same error Thanks,Harinath   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Failed to prepare update plan error while inserting data, when select is used aong with insert

[Alexandr Shapkin]

Hi,  Seems like there is a syntax error and Apache Ignite failed to produce an execution plan Could you try to run it without the parentheses? Like: INSERT INTO desttable (name, lastname, age) SELECT name, lastname, age from srctable  From: Alexandr ShapkinSent: Wednesday, November 18, 2020 5:10 PMTo: user@ignite.apache.orgSubject: RE: Failed to prepare update plan error while inserting data, when select is used aong with insert Hi, Seems like there is a syntax error. From: harinathSent: Wednesday, November 18, 2020 10:28 AMTo: user@ignite.apache.orgSubject: Failed to prepare update plan error while inserting data, when select is used aong with insert Hi, Using ignite thick client, i'm trying to copy data from one table intoanother table using following sql statement "INSERT INTO desttable (name, lastname, age) (SELECT name, lastname, agefrom srctable)" I'm getting the following error while i run sql query."Failed to prepare update plan." Not sure of the reason for the error.Any help here will be much appreciated. Thanks in advance Thanks & Reagrds,Harinath   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/  

RE: Failed to prepare update plan error while inserting data, when select is used aong with insert

[Alexandr Shapkin]

Hi, Seems like there is a syntax error. From: harinathSent: Wednesday, November 18, 2020 10:28 AMTo: user@ignite.apache.orgSubject: Failed to prepare update plan error while inserting data, when select is used aong with insert Hi, Using ignite thick client, i'm trying to copy data from one table intoanother table using following sql statement "INSERT INTO desttable (name, lastname, age) (SELECT name, lastname, agefrom srctable)" I'm getting the following error while i run sql query."Failed to prepare update plan." Not sure of the reason for the error.Any help here will be much appreciated. Thanks in advance Thanks & Reagrds,Harinath   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Query on Ignite K8s Operator

[Alexandr Shapkin]

Hi, In short, yes, it’s licensed and yes, you can use it for free. The project is under development and should be publicly announced in the near future. It’s licensed under the default GridGain community edition license and basically, It means that it’s free to use, you might find more information about the license at the official website, though. It also should works with both versions: Apache Ignite and GridGain editions, but It’s better to wait for Apache Ignite 2.9 release for about a week or so, cause it should come with an improved image that works better under k8s management (with 2.8.1 it’s cumbersome to scale the cluster down).  The last warning – please, remember that it’s a beta version and use it on your own risk for now. Please, share your thought if you’d like to try it, any feedback is welcomed! From: Vishwas BmSent: Friday, October 9, 2020 5:23 PMTo: user@ignite.apache.orgSubject: Query on Ignite K8s Operator Hi,I found the below link on Ignite K8s operator docker image:https://hub.docker.com/r/gridgain/apache-ignite-operatorIs this licensed ? Can this be used ?  Thanks & Regards,Vishwas 

RE: Unable to perform handshake within timeout..

[Alexandr Shapkin]

Hi, Am I right that you are using the Azure Kubernetes cluster? If yes, it’s most likely is about the k8s kube-proxy health check probe pinging all open ports including the default thin client one (port 10800). Because of that, Ignite can’t read the message correctly and the ClientListenerNioListener states that it’s unable to perform a handshake with the remote address within 10 seconds. I’d suggest you make the following changes for healthCheckNodePort in your LoadBalancer configuration: externalTrafficPolicy: LocalhealthCheckNodePort: 30001 (a custom port that’s not in use by Apache Ignite and other apps) More details:https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip   From: SanjayaSent: Sunday, September 13, 2020 9:08 AMTo: user@ignite.apache.orgSubject: Re: Unable to perform handshake within timeout.. We are experencing same, We are using apache ignite 2.8.1. We are experiencing bunch of logs as below, The good things the applicationworks as expected. But we did not undetstand why the logs are coming, Request to help on this.  STACK TRACE==[05:08:52,928][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.66:60529][05:08:54,051][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.128:59150][05:08:55,729][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.190:60622][05:08:57,502][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.35:51873][05:08:57,929][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.66:60649][05:08:59,050][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.128:59209][05:09:00,741][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.190:60733][05:09:02,517][WARNING][grid-timeout-worker-#23][ClientListenerNioListener]Unable to perform handshake within timeout [timeout=1,remoteAddr=/10.208.16.35:51931]--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Unable to read fields value from BinaryObject

[Alexandr Shapkin]

Hi, Just for the record, I noticed that the following tickets has been filed: https://issues.apache.org/jira/browse/IGNITE-13415 https://issues.apache.org/jira/browse/IGNITE-13436 Also seems like the consumer would work if you do at least one cache#put operation. From: Alexandr ShapkinSent: Thursday, September 10, 2020 3:04 PMTo: user@ignite.apache.orgSubject: Re: Unable to read fields value from BinaryObject Hi, Yes, I can reproduce the behavior. It could be an issue with the custombinary mappers.  The weird thing is that the CustomBinaryIdMapper works fine with a singlenode, i.e. if it reads and writes the value from a cache, then the mapper isworking. But fails on the other node. Meanwhile, it's quite interesting, why do you want to replace the defaultmapper with a custom one? What tasks are you trying to solve?   -Alex Shapkin--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Null Pointer exception error caught during grid runnable execution

[Alexandr Shapkin]

Hi, It looks like a known issue when a server node is not fully initialized but clients are trying to connect to it [1] I’d suggest you try the latest version (2.8.1) and check if the error will be gone. [1] - https://issues.apache.org/jira/browse/IGNITE-11671 From: SanjayaSent: Friday, September 11, 2020 10:06 AMTo: user@ignite.apache.orgSubject: Null Pointer exception error caught during grid runnable execution Hi All, we are using apache ignite 2.7.6 with pesistence enable inside AzureKubernetes cluster in production.we have 2 pods having persistence enable, The pods have below resourcesassgined to it CPU : 3 coreheap: 3 GBPeristence : SSD 15 GBWAL : SSD 5 GB   We are reading / writting throigh JDBC thin clinet. The client are outsideof  kubernetes, so they connect through kubernetes LoadBlancer IP. The loading data part (from PostGres to Ignite) is quite OK, and it runs asscheduled. But we are facing below issue(Null pointer exception) when the clients aretrying to connect through jdbc thin client. Once the issue occurres, ignitetriggers a complete restart, so clients are not able to connect that timeand everything goes for toss. We never faces this issue in lower enviornment. We are stuck with the issue,request help on thios.  STACK TRACE[05:18:39,803][WARNING][main][GridCacheDatabaseSharedManager] Ignite nodestopped in the middle of checkpoint. Will restore memory state and finishcheckpoint on node start.[05:18:42,276][SEVERE][client-connector-#40][ClientListenerProcessor]Runtime error caught during grid runnable execution: GridWorker[name=message-received-notify, igniteInstanceName=null, finished=false,heartbeatTs=1599801522271, hashCode=1044207064, interrupted=false,runner=client-connector-#40]java.lang.NullPointerException  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.nextConnectionId(ClientListenerNioListener.java:306)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.prepareContext(ClientListenerNioListener.java:285)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onHandshake(ClientListenerNioListener.java:223)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:129)  atorg.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:45)  atorg.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)  atorg.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)  atorg.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)  atorg.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)  atorg.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)  atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

Re: Preload all indexes to memory

[Alexandr Shapkin]

Hi,

Looks like this is the part of the upcoming warmup feature [1] that should
be published with the next Ignite release (2.10) or the upcoming GridGain
edition (v8.7.26?) [2].

Please, check the IEP-40 for the detailed explanation [3]

[1] - https://issues.apache.org/jira/browse/IGNITE-13345
[2] -
https://www.gridgain.com/docs/latest/release-notes/8.7.25/release-notes_8.7.25
[3] -
https://cwiki.apache.org/confluence/display/IGNITE/IEP-40+Cache+warm-up




-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unable to read fields value from BinaryObject

[Alexandr Shapkin]

Hi,

Yes, I can reproduce the behavior. It could be an issue with the custom
binary mappers. 

The weird thing is that the CustomBinaryIdMapper works fine with a single
node, i.e. if it reads and writes the value from a cache, then the mapper is
working. But fails on the other node.

Meanwhile, it's quite interesting, why do you want to replace the default
mapper with a custom one? What tasks are you trying to solve?



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite NPE when try to loadCache from persistent mysql database

[Alexandr Shapkin]

Hi,

Is it possible to share a simple reproducer or to get the full logs from the
failed node?




-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: ClusterGroupEmptyException: Topology projection is empty.

[Alexandr Shapkin]

Hi, [33m[2020.09.08 21:19:28.898] [WARN ] [o.a.i.i.p.r.h.t.GridTaskCommandHandler] Failed to execute task due to topology issues (are all mapped nodes alive?) [name=o.a.i.i.v.compute.VisorGatewayTask, clientId=null, err=class o.a.i.IgniteCheckedException: Failed to reduce job results due to undeclared user exception [task=o.a.i.i.v.compute.VisorGatewayTask@6eb42aa, err=class o.a.i.cluster.ClusterGroupEmptyException: Topology projection is empty.]]  Well, it’s the VisorNodeConfigurationCollectorTask that’s trying to execute, but failed cause the server is not ready.Do you use the Visor for cluster monitoring? I think it’s fine to ignore it, but it’s worth understanding what’s launching it. From: kaySent: Wednesday, September 9, 2020 2:48 AMTo: user@ignite.apache.orgSubject: Re: ClusterGroupEmptyException: Topology projection is empty. Hello Here is my full logNexusCache1-1-1.out   I'm using 2.8.0 version and I don't use any clustergroup configuration. Thank you!   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

Re: Countdown latch issue with 2.6.0

[Alexandr Shapkin]

Hi,

Have you tried to reproduce this behaviour with the latest release - 2.8.1?
If I remember correctly, there were some fixes regarding the partition
counters mismatches, for sample:
https://issues.apache.org/jira/browse/IGNITE-10078



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Messages being Missed on Node Start

[Alexandr Shapkin]

Hi,

Yes, seems like the https://issues.apache.org/jira/browse/IGNITE-1410 is not
resolved, unfortunately.

Is it possible to listen for predefined events like
LifecycleEventType.AFTER_NODE_START or EVT_CLUSTER_ACTIVATED?

Alternatively, you might want to utilize Ignite data structures, like
AtomicLong to signal other nodes that the client is ready for processing.

If none of the above suggestions work for you, can you provide an example of
why do you need to register a custom event? And what you are trying to
achieve with it?



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Question regarding topology

[Alexandr Shapkin]

Hello!

It's not clear, whether the code snippet came from a client or a server. If
it's a client, then I'd recommend you try sending the keys and additional
data directly to a server node and perform the insertion directly on the
node. The Compute API could be an option here [1]

You might also want to check the thread dump or JFR and check where the
possible starvation is happening. It's also useful to check the resource
utilization prior to adding additional nodes.

[1] - https://apacheignite.readme.io/docs/compute-grid#ignitecompute



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite memory memory-architecture with cache partitioned mode

[Alexandr Shapkin]

I just wanted to highlight that there is the Partition Awareness feature
available for thin clients.
When enabled, a client will know to witch node a partition belongs and send
a request directly to that node.

[1] -
https://apacheignite-net.readme.io/docs/thin-client#partition-awareness
[2] -
https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients



-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: connection refused

[Alexandr Shapkin]

Hi, As Wesley mentioned, It’s quite difficult to say something specific about without the logs.  As for the additional questions, can you explain, what Ignite API is being used? Is it a Compute API call or a regular cache GETs? It’s totally fine to spawn several user threads and access the server simultaneously.Regarding the node connectivity, by default there is a 1 connection per node, you can control it using the TcpCommunicationSpi#setConnectionsPerNode [1]  You can check the pool and adjust the sizes using the following docs [2].If you perform much compute API calls, you might be interested in increasing the server public pool size, striped pool for cache accesses and so on. In any case, it’s better to check your system resources utilization (CPU/memory/etc) if it’s about 100% on a client and 20% on a server, you might want to add several more clients to a cluster. [1] - https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.html#setConnectionsPerNode-int- [2] - https://apacheignite.readme.io/docs/thread-pools  From: Clay TeahouseSent: Tuesday, June 2, 2020 5:20 PMTo: user@ignite.apache.orgSubject: Re: connection refused It turned out if I inject a pause of 4-5 seconds, then the client connection goes through.Why the service grid requests go through  immediately but the regular cache access requires a pause from the time an ignite instance is acquired? Subsequent calls are fast. The caches are replicated.Another question, probably trivial:Can you instantiate multiple concurrent connections from a client to the compute nodes? Is the number of connections limited by the number of threads? Or do I need to create separate client instances for each request?   On Tue, Jun 2, 2020 at 5:46 AM Wesley Peng  wrote:Does either client or server have any logs? Thanks  On Tue, Jun 2, 2020 at 6:37 PM Clay Teahouse  wrote:I'd appreciate your help with this issue.I have a server and a client node, the latter running in tomcat. I get connection refused if I try to connect to the server node from the client, although the server shows the client has joined the cluster. I don't have a problem running a service on the server's service grid from the client node.I don't have any issue connecting from any client anywhere, if the client is not running in tomcat.what could be the problem?  

RE: Re: Backups not being done for SQL caches

[Alexandr Shapkin]

Hi,  I believe that you need to find the following message: 2020-04-30 16:57:44.8141|INFO|Test|Topology snapshot [ver=53, locNode=26887aac, servers=3, clients=0, state=ACTIVE, CPUs=16, offheap=8.3GB, heap=15.0GB]2020-04-30 16:57:44.8141|INFO|Test|  ^-- Baseline [id=0, size=3, _online_=3, offline=0] Metrics don’t tell you l the actual topology and baseline snapshot.A node might be running, but not included into the baseline, that might be the reason in your case. Also, what Ignite version do you use? From: Courtney RobinsonSent: Thursday, April 30, 2020 7:58 PMTo: user@ignite.apache.orgSubject: Re: Backups not being done for SQL caches Hi Illya,Yes we have persistence enabled in this cluster. This is also change from our current production deployment where we have our own CacheStore with read and write through enabled. In this test cluster Ignite's native persistence is being used without any external or custom CacheStore implementation. From the Ignite logs it says all 3 nodes are present: 2020-04-30 16:53:20.468  INFO 9 --- [orker-#23%hypi%] o.a.ignite.internal.IgniteKernal%hypi    :Metrics for local node (to disable set 'metricsLogFrequency' to 0)    ^-- Node [id=e0b6889f, name=hypi, uptime=19:15:06.473]    ^-- H/N/C [hosts=3, nodes=3, CPUs=3]    ^-- CPU [cur=-100%, avg=-100%, GC=0%]    ^-- PageMemory [pages=975]    ^-- Heap [used=781MB, free=92.37%, comm=4912MB]    ^-- Off-heap [used=3MB, free=99.91%, comm=4296MB]    ^--   sysMemPlc region [used=0MB, free=99.98%, comm=100MB]    ^--   metastoreMemPlc region [used=0MB, free=99.95%, comm=0MB]    ^--   TxLog region [used=0MB, free=100%, comm=100MB]    ^--   hypi region [used=3MB, free=99.91%, comm=4096MB]    ^-- Ignite persistence [used=3MB]    ^--   sysMemPlc region [used=0MB]    ^--   metastoreMemPlc region [used=0MB]    ^--   TxLog region [used=0MB]    ^--   hypi region [used=3MB]    ^-- Outbound messages queue [size=0]    ^-- Public thread pool [active=0, idle=0, qSize=0]    ^-- System thread pool [active=0, idle=6, qSize=0]Regards,Courtney RobinsonFounder and CEO, HypiTel: ++44 208 123 2413 (GMT+0)https://hypi.io  On Thu, Apr 30, 2020 at 3:12 PM Ilya Kasnacheev  wrote:Hello! Do you have persistence? If so, are you sure that all 3 of your nodes are in baseline topology? Regards,-- Ilya Kasnacheev  чт, 30 апр. 2020 г. в 16:09, Courtney Robinson :We're continuing migration from using the Java API to purley SQL and have encountered a situation on our development cluster where even though ALL tables are created with backups=2, as intemplate=partitioned,backups=2,affinity_key=instanceId,atomicity=ATOMIC,cache_name=In the logs, with 3 nodes in this test environment we have: 2020-04-29 22:55:50.083 INFO 9 --- [orker-#40%hypi%] o.apache.ignite.internal.exchange.time : Started exchange init [topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=e0b6889f-219b-4686-ab52-725bfe7848b2, customEvt=DynamicCacheChangeBatch [id=a81a0e7c171-3f0fbbc0-b996-448c-98f7-119d7e485f04, reqs=ArrayList [DynamicCacheChangeRequest [cacheName=hypi_whatsapp_Item, hasCfg=true, nodeId=e0b6889f-219b-4686-ab52-725bfe7848b2, clientStartOnly=false, stop=false, destroy=false, disabledAfterStartfalse]], exchangeActions=ExchangeActions [startCaches=[hypi_whatsapp_Item], stopCaches=null, startGrps=[hypi_whatsapp_Item], stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], allowMerge=false, exchangeFreeSwitch=false]2020-04-29 22:55:50.280 INFO 9 --- [orker-#40%hypi%] o.a.i.i.p.cache.GridCacheProcessor : Started cache [name=hypi_whatsapp_Item, id=1391701259, dataRegionName=hypi, mode=PARTITIONED, atomicity=ATOMIC, backups=2, mvcc=false]2020-04-29 22:55:50.289 INFO 9 --- [ sys-#648%hypi%] o.a.i.i.p.a.GridAffinityAssignmentCache : Local node affinity assignment distribution is not ideal [cache=hypi_whatsapp_Item, expectedPrimary=1024.00, actualPrimary=0, expectedBackups=2048.00, actualBackups=0, warningThreshold=50.00%]2020-04-29 22:55:50.293 INFO 9 --- [orker-#40%hypi%] .c.d.d.p.GridDhtPartitionsExchangeFuture : Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED]2020-04-29 22:55:50.330 INFO 9 --- [orker-#40%hypi%] .c.d.d.p.GridDhtPartitionsExchangeFuture : Finished waiting for partitions release latch: ServerLatch [permits=0, pendingAcks=HashSet [], super=CompletableLatch [id=CompletableLatchUid [id=exchange, topVer=AffinityTopologyVersion [topVer=27, minorTopVer=1 You can see the line Local node affinity assignment distribution is not ideal but it's clear they the backup = 2 is there. To verify, I stoped 2 of the three nodes and sure enough I get the exception Failed to find data nodes for cache: InstanceMapping Is there some additional configuration needed for partitioned SQL caches to have the backups as configured?Until now we used the Java API with put/get and didn't 

RE: Re: Query performance varying with LIMIT keyword

[Alexandr Shapkin]

Hello, Your query doesn’t use indices and performs a full scan on your tables.That requires more time in order to execute a join and get records. Please, consider adding the corresponding index:https://www.gridgain.com/docs/latest/sql-reference/ddl#create-index From: nunobSent: Tuesday, January 21, 2020 7:17 PMTo: user@ignite.apache.orgSubject: Re: Query performance varying with LIMIT keyword Hello, Thanks for responding. Well, it takes 50ms to fetch the first line. What I don't get is why it takes 8 seconds to fetch the 50th line. In both cases I'm only fetching ONE row. I suppose query execution time should be roughly the same whether i want the1st or the 50th line. I mean, aren't OFFSET and LIMIT just discarding rows from the resultingdataset? Isn't it a question of  looping through the cursor until it findsthe Nth row? Why does it look like it's responding with exponential time asI increase the offset? Looking at explain it does look like it's doing a full table scan. In thefollowing query I have indexes over the joining columns on both tables: SELECT    SE__Z1.ISINCODE AS __C0_0,    M__Z0.SYMBOL AS __C0_1,    NULL AS __C0_2,    CURRENT_TIMESTAMP() AS __C0_3,    M__Z0.LASTTIME AS __C0_4,    M__Z0.OPENPRICE AS __C0_5,    M__Z0.HIGHPRICE AS __C0_6,    M__Z0.LOWPRICE AS __C0_7,    M__Z0.CLOSEPRICE AS __C0_8,    M__Z0.LASTPRICE AS __C0_9,    M__Z0.LASTQUANTITY AS __C0_10,    M__Z0.BIDPRICE AS __C0_11,    M__Z0.ASKPRICE AS __C0_12,    M__Z0.BIDSIZE AS __C0_13,    M__Z0.ASKSIZE AS __C0_14FROM SERVERDEAL.MARKETDATA M__Z0    /* SERVERDEAL.MARKETDATA.__SCAN_ */LEFT OUTER JOIN SIMS.SECURITYEXCHANGE SE__Z1    /* SIMS.SECURITYEXCHANGE.__SCAN_ */    ON (SE__Z1.SECURITYTYPE = M__Z0.MARKET)    AND ((SE__Z1.EXCHANGEID = M__Z0.EXCHANGE)    AND (SE__Z1.SYMBOL = M__Z0.SYMBOL))LIMIT 11--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Get Added CacheConfiguration template from .Net Client Ignite Node

[Alexandr Shapkin]

Hello! Normally, a cache configuration should be transferred between the nodes, once it gets created.There are a couple of methods for accessing it:ICache#GetConfiguration(cacheName)igniteInstance.GetConfiguration().CacheConfiguration.Single(x => x.Name == CacheName) From: sivaSent: Tuesday, January 21, 2020 6:48 PMTo: user@ignite.apache.orgSubject: Get Added CacheConfiguration template from .Net Client Ignite Node Hi,I am using .Net Client and Server App using Ignite v2.7.6. Node1:All cache created and Added configuration into Node1.Node2:Started and some of cache configuration added into Node2.  for example:let's say for "test1" cache no cache template added into ignitenode.--IgniteClientInstance.GetOrCreateCache("test1").CacheConfiguration//throwing error   so how to get from node whether configuration is exists or not for theparticular cache name? --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

Re: Average get time not supported for getAll

[Alexandr Shapkin]

Hello!

Roughly speaking, the getAll() internally being transformed into a sequence
of an individual get() operations. That's why Ignite doesn't provide a
special metric for that and will utilize the same #getAverageGetTime(). At
the same time, Ignite does care about what caches' API is being used. I.e.
it will calculate the duration of the #getAll() method and divide it by the
actual value reads count. 

So, you should consider listening for the same #getAverageGetTime() metrics.






-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Average get time not supported for getAll

[Alexandr Shapkin]

Hello!

Roughly speaking, the getAll() internally being transformed into a sequence
of an individual get() operations. That's why Ignite doesn't provide a
special metric for that and will utilize the same #getAverageGetTime(). At
the same time, Ignite does care about what caches' API is being used. I.e.
it will calculate the duration of the #getAll() method and divide it by the
actual value reads count. 

So, you should consider listening for the same #getAverageGetTime() metrics.






-
Alex Shapkin
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Re: How best to push continuously updating ordered data on Apache ignite cache

[Alexandr Shapkin]

Hello! You could take a look at putAll method that sends all updates to the server right away. > *What is best way out of above to push continuously updating data like> market data?*For me it seems that the DataStreamer is more suitable for continuous data processing where you are not interested in getting acks from a server.Use StreamReceiver/StreamTransformer for additional data modificationbefore the updating the values. As Mikael suggested, you could implement some kind of an update counter. It feels that DateTime should work here. Atomic sequences for the valid stock price could be helpful as well [1]. [1] - https://apacheignite.readme.io/docs/atomic-types From: MikaelSent: Friday, January 17, 2020 12:07 PMTo: user@ignite.apache.orgSubject: Re: How best to push continuously updating ordered data on Apache ignite cache Hi! Trusting that things come in the correct order could be tricky, the only way to do it I guess it to use put(), if you need fast writes you will have to use multiple threads and any ordering will be lost, can't you include some ordering data when you write them, a counter or whatever and use that to sort them in client 2 ? that would be uch safer. You might need to include some kind of waiting process in case one entry has not arrived yet, but if you use a counter to write them you know that after 1 there will always ba a 2 and so on without any gaps so it should not be that messy. Mikael Den 2020-01-17 kl. 04:51, skrev trans:> *Usecase*> > Here is the topology we are working on> > *Server - 1* --> marketData cache, holds different share prices information> > *Client - 1* --> Pushing data on the cache> > *Client - 2* --> Continuous query, listening updates coming on marketData> cache per key basis> > I want data to follow the order in which it was received and pushed on> queue. Reason is that client - 2 should not get old data. For example last> price data for an instrument moved from 100 to 101 then 102. Client - 2> should get in the same order and to **not** get in order like 100 to 102> then 101.> So for a key in my cache i want the messages to be pushed in order.> > Ways of pushing data on cache:> >   1. *put* seems to be safest way but looks like slow and full cache update> is happening and then thread moves to next statement. This might not be> suitable for pushing 2 updates per second.> >   2. *putAsync *seems to be good way, my understanding is that it uses> striped pool to push data on the cache. As the  striped pool> > uses max(8, no of cores) so it should be faster. But as the multiple threads> are being processed in parallel so does it confirm the data ordering as it> was pushed in?> >   3. *DataStreamer *seems to be best way as it also processes things in> parallel but again the problem is with the order of putting data into cache.> API documentation> > also mention that ordering of data is not guaranteed.> > > Can someone please clarify above? I could not find document giving clear> deeper understanding for these ways.> *What is best way out of above to push continuously updating data like> market data?*> > > > --> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Populate ddl (e.g., create table) to third party persistence

[Alexandr Shapkin]

Hello! I don’t think it could be done out of the box. You could try to start from the opposite side and generate a schema from a RDBMS instead using the WebConsole [1]. [1] - https://www.gridgain.com/docs/web-console/2019.12.02/automatic-rdbms-integration  From: crypto_rickleeSent: Friday, January 17, 2020 10:19 AMTo: user@ignite.apache.orgSubject: Populate ddl (e.g., create table) to third party persistence Hi all, I'm using Postgres as the third party persistence for Ignite 2.7.6. Justwant to know if I create a table on ignite cluster, is there any way topopulate the schema back to the Postgres automatically? Thanks!   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: Cache entries expiring in pageEvictionMode=DISABLED

[Alexandr Shapkin]

>What's worse - they are evicted on one node, and eviction>is not propagated on the other two nodes (so now every node has different>count of entries for that cache, even though it's replicated) Data region configuration defines a local machine’s memory settings, it doesn’t propagate to other nodes.  Have you managed to get the eviction work properly? From: akorenshSent: Wednesday, January 15, 2020 11:36 PMTo: user@ignite.apache.orgSubject: Re: Cache entries expiring in pageEvictionMode=DISABLED Hi, Are you not seeing cache entries and therefore making the assumption thateviction has kicked in?You could inadvertently be using a different config w/persistence turned on? search in ignite logs for the words "persistenceEnabled=false"  If you've not enabled onHeapCaching or persistence, and page eviction isdisabled and no memory is left, Ignite will throwIgniteOutOfMemoryException. docs for your version: https://apacheignite.readme.io/v2.3/docs/evictions Can you try with only one cache(NonEvictable_256MB_Region) and see whetherthe problem repeats itself? Also, you have an old version, and if possible, can you do a small testusing a newer version(2.7.6) https://ignite.apache.org/download.cgi  and seewhether the behavior repeats itself. If you are able to reproduce this on 2.7.6, then send us your entire config,a reproducer project (or the code that you use to populate the cache) and wewill take a look. Thanks, Alex   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: C# client not able to connect to cluster using IP address

[Alexandr Shapkin]

Hello! Thin clients don’t change topology version, you may think of it as a different API for performing cluster operations alongside withREST API, WebConsole and others.[1] When you call Ignition.StartClient,  your application establishes a new TCP connection to a server node using default 10800 port and utilizes it for performing cluster operations. On the other side, calling Ignition.Start will start a new node instance.The process of starting a new NODE is much complicated than opening a single TCP connection. It involves the Discovery API – a new node needs to find others in order to form a cluster(47500 port), as well as the Communication SPI – nodes, should send messages toeach other (47100 port). That’s true, a node could be either a client or a server [2],but you should distinguish a regular client node that forms a topology and a thin client connection that doesn’t. [1] - https://apacheignite-net.readme.io/docs/thin-client[2] - https://apacheignite.readme.io/docs/clients-vs-servers  From: SidPSent: Thursday, December 26, 2019 1:35 PMTo: user@ignite.apache.orgSubject: Re: C# client not able to connect to cluster using IP address Hi, Do you mean i should try this? Keep Server config as it is, In TcpDiscoverySpi node keep value as  MyHostName:47500..47504 C# thin Client tries to connect with below code private static IgniteClientConfiguration GetIgniteClientConfiguration()    {    return new IgniteClientConfiguration    {    Host = "MyHostName",    SocketTimeout = TimeSpan.FromSeconds(15) };    } * IgniteThinClient = Ignition.StartClient(GetIgniteClientConfiguration());* Code pass through above line without any exception, but in server log,Topology snapshot as belowTopology snapshot [ver=1, locNode=f19c8896, servers=1, *clients=0*,state=ACTIVE, CPUs=8, offheap=3.2GB, heap=1.0GB] Please let me know if i am doing anything wrong? Also in case, if ignite server config has some different ports specified forexample (46000..46505) then how thin client should connect to one of thenode by specified one of the port from this (46000..46505) range, right? --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: Unable to connect to a remote apache ignite node.

[Alexandr Shapkin]

Have you tried to remove ` joinTimeout` value?  From: Ilya KasnacheevSent: Tuesday, November 26, 2019 1:48 PMTo: user@ignite.apache.orgSubject: Re: Unable to connect to a remote apache ignite node. Hello! I think you have some kind of connectivity problems between two nodes so the second node can never finish the join process. I recommend trying to specify localHost on both nodes and only keep a single non-loopback address in discovery section. Regards,-- Ilya Kasnacheev  пт, 22 нояб. 2019 г. в 13:50, xabush :I am running a simple code to test Apache Ignite distributed cache. One nodeis on a server and the other one is run locally when I start a java program.public class HelloIgnite {    public static void main(String[] args) {        System.out.println("Hello Ignite");        Ignite ignite = Ignition.start("example-cache.xml");        // get or create cache        IgniteCache cache =ignite.getOrCreateCache("testCache");        // put some cache elements        for(int i = 1; i <= 100; i++){            cache.put(i, Integer.toString(i));        }        // get them from the cache and write to the console        for(int i =1; i<= 100; i++){            System.out.println("Cache get:"+ cache.get(i));        }        ignite.close();    }} The above is the simple code I am testing. And here is my ignite confighttp://www.springframework.org/schema/beans"       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"       xsi:schemaLocation="        http://www.springframework.org/schema/beans        http://www.springframework.org/schema/beans/spring-beans.xsd">    class="org.apache.ignite.configuration.IgniteConfiguration">                            class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">                                    class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">                                                                                    127.0.0.1:47500..47509                                SERVER_IP:47500..47509                                                                                                                                I start node on the server using ignite.sh -p example-cache.xml first andthen run the above code locally. However, instead of putting values to thecache and printing them, when I run the code, it is stuck showing thismessage:/ [13:40:46] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler[tryStop=false, timeout=0, super=AbstractFailureHandler[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED,SYSTEM_CRITICAL_OPERATION_TIMEOUT[13:40:48] Message queue limit is set to 0 which may lead to potential OOMEswhen running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due tomessage queues growth on sender and receiver sides.[13:40:49] Security status [authentication=off, tls/ssl=off]/On the node I started on the server, It is repeatedly printing the followingto the console:/[11:48:28] Topology snapshot [ver=56, locNode=cd55466e, servers=2,clients=0, state=ACTIVE, CPUs=40, offheap=28.0GB, heap=34.0GB][11:48:28] Topology snapshot [ver=57, locNode=cd55466e, servers=1,clients=0, state=ACTIVE, CPUs=32, offheap=25.0GB, heap=30.0GB][11:48:34] Joining node doesn't have encryption data[node=02e2f398-b5b8-4044-83dd-53755e28f94e]/However, when I stop the server node, the local node starts to run properlyand print the values.How can I resolve this? what I am doing wrong here?--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: Ignite data loss

[Alexandr Shapkin]

Hello! Is it possible that you had disabled WAL before restarting a cluster? Doing this may cause data loss in some scenarios. From: MikaelSent: Saturday, November 16, 2019 9:58 AMTo: user@ignite.apache.orgSubject: Re: Ignite data loss Hi!So it does not matter what node you restart ? it's always that one that keeps the data ?Are all 4 nodes part of the baseline topology ?I pretty much have the same setup but with 3 nodes and have not had any problems at all, not sure what it could be ?If you turn on all logging you will see all it does at startup and can see if there is something weird going on, it's lots of information but it usually gives a good indication if there is any problem, nothing else in the logs ? if the nodes clear everything at startup there should be something in the logs.MikaelDen 2019-11-16 kl. 04:16, skrev KR Kumar:Hi Guys - I have a four node cluster with native persistence enabled. Its a partitioned cache and sync rebalance is enabled. When I restart the cluster the first node that starts retain the data and all the other nodes data is deleted and all the ignite data files are turned into 4096 byte files. Am I missing something? or some configuration that I missing.  Following is the cache configuration:    CacheConfigurationbyte[]> cacheConfig = new CacheConfigurationbyte[]>();      cacheConfig.setCacheMode(CacheMode.PARTITIONED);      cacheConfig.setRebalanceMode(CacheRebalanceMode.SYNC);//      cacheConfig.setRebalanceDelay(30);   cacheConfig.setName("eventCache-" + tenantRunId + "-" + tenantId);   cacheConfig.setBackups(1);      cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);      cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);   IgniteCachebyte[]> cache = IgniteContextWrapper.getInstance().getEngine()      .getOrCreateCache(cacheConfig);  Here is the configuration of ignite                                                                Any quick pointers ?? Thanx and Regards,KR Kumar  

RE: Re: How to perform distributed compute in similar way to Spark vector UDF

[Alexandr Shapkin]

Hello! Have you tried the ML package for Apache Ignite [1]? It definitely should use data frames internally. > So cache keys [0...9] have affinity key 0, keys [10...19] have>affinity key 1 etc? Sure, the easiest one would be to add a new field to your model with[AffinityKeyMapped] public string AffKey { get; set; } = rowNumber / itemsPerPartition  Also, you may refer to GridGain’s documentation as well [2]. It’s compatible with Apache Ignite.  [1] - https://apacheignite.readme.io/docs/ml-partition-based-dataset [2] - https://www.gridgain.com/docs/latest/developers-guide/data-modeling/affinity-collocation#configuring-affinity-key From: camer314Sent: Monday, November 18, 2019 6:43 AMTo: user@ignite.apache.orgSubject: Re: How to perform distributed compute in similar way to Spark vector UDF Reading a little more in the Java docs about AffinityKey, I am thinking that,much like vector UDF batch sizing, one way I could easily achieve my resultis to batch my rows into affinity keys. That is, for every 100,000 rows theaffinity key changes for example. So cache keys [0...9] have affinity key 0, keys [10...19] haveaffinity key 1 etc? If that is the case, may I suggest you update the .NET documentation forData Grid regarding Affinity Colocation as it does not mention the use ofAffinityKey or go into anywhere near as much detail as the Java docs.  --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: RE: IgniteSQLException

[Alexandr Shapkin]

Jira ticket https://issues.apache.org/jira/browse/IGNITE-12381   From: Alexandr ShapkinSent: Monday, November 18, 2019 4:44 PMTo: user@ignite.apache.orgSubject: RE: IgniteSQLException If you take a look at QueryParser#parseH2 then you can see the following statement: catch (IgniteCheckedException | SQLException e) {    throw new IgniteSQLException("Failed to parse query. " + e.getMessage(), IgniteQueryErrorCode.PARSING, e);} So any exception on this stage will be considered as a Parsing.I think it’s ok to translate an h2 exception, say TABLE_OR_VIEW_NOT_FOUND_1 = 42102 into SqlStateCode.I will check JIRA for that. From: DSSent: Monday, November 18, 2019 3:42 PMTo: user@ignite.apache.orgSubject: IgniteSQLException catch (Exception e) {  ///System.out.println(superCache.err);  if (e instanceof CacheException) {    IgniteSQLException sqLEx = X.cause(e, IgniteSQLException.class);    System.out.println("> statusCode " + sqLEx.statusCode()); }  I am trying to fetch the exact string for the sql statusCode.But i am getting statusCode :1001 (parsing error ) for both wrong SQL syntax and table does not exist(3001).  P.SI am trying to understand how SQL error works. and how to caputure precisestring for particular statusCode.eg : CACHE_NOT_FOUND      4006COLUMN_ALREADY_EXISTS    3009--Sent from: http://apache-ignite-users.70518.x6.nabble.com/  

RE: IgniteSQLException

[Alexandr Shapkin]

If you take a look at QueryParser#parseH2 then you can see the following statement: catch (IgniteCheckedException | SQLException e) {    throw new IgniteSQLException("Failed to parse query. " + e.getMessage(), IgniteQueryErrorCode.PARSING, e);} So any exception on this stage will be considered as a Parsing.I think it’s ok to translate an h2 exception, say TABLE_OR_VIEW_NOT_FOUND_1 = 42102 into SqlStateCode.I will check JIRA for that. From: DSSent: Monday, November 18, 2019 3:42 PMTo: user@ignite.apache.orgSubject: IgniteSQLException catch (Exception e) {  ///System.out.println(superCache.err);  if (e instanceof CacheException) {    IgniteSQLException sqLEx = X.cause(e, IgniteSQLException.class);    System.out.println("> statusCode " + sqLEx.statusCode()); }  I am trying to fetch the exact string for the sql statusCode.But i am getting statusCode :1001 (parsing error ) for both wrong SQL syntax and table does not exist(3001).  P.SI am trying to understand how SQL error works. and how to caputure precisestring for particular statusCode.eg : CACHE_NOT_FOUND      4006COLUMN_ALREADY_EXISTS    3009--Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: Ignite.Net Read through approach - error : could not create .Net cachestore

[Alexandr Shapkin]

Sudhir, If I configure a cacheStore for a clientNode, then using 2.7.6 it becomes initialized twice, for both server and a client.Whereas I checked the same scenario against the master-branch and only a single cache store had created. Also, I have found the IGNITE-1903 that may fix the issue for the master. > What I am trying is call get operation on a key in cache for 1st time from client node as per my requirements. i.e. server node don't >have that key yet. Is this possible or correct??? I think yes because it worked without any issues when I used thin client instead of >normal client node. All cache query operations are routed to the server. Local cacheStore could be required for some transactional scenarios as well as for near caching. So, I would suggest you configure a cacheStore only for server nodes. From: Sudhir PatilSent: Thursday, November 14, 2019 8:38 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Mikael, The client should not have to pass any cache configuration, once the cache is created in the server, you should just be able to get the cache Yes exactly this is my understanding also. What I am trying is call get operation on a key in cache for 1st time from client node as per my requirements. i.e. server node don't have that key yet. Is this possible or correct??? I think yes because it worked without any issues when I used thin client instead of normal client node.Server & client node communication works fine (i.e. client retrieves key value stored from cache etc) without cachestore enabled on server. But it throws that exception when cachestore is enabled. On Thursday, November 14, 2019, Mikael <mikael-arons...@telia.com> wrote:Hi!The client should not have to pass any cache configuration, once the cache is created in the server, you should just be able to get the cache, I do not see any reason that should not work but I have never used .NET so it is possible there is some hickup there that I am not seeing.You do not have any other Ignite servers running on the same subnet or something ? verify that the client connect to the correct server (sounds silly but I managed to mess up a number of times ;).Is it just that cache ? could you create another cache as simple as possible (no cache store) and see if it will find that one ?It sounds like it cannot create the actual cache store but in that case I would think you would get the exception in the server and not the client, is there any more useful information from the exception ?Mikael Den 2019-11-14 kl. 02:41, skrev Sudhir Patil:Hi All,  Server node is started & cache is created with configuration likevar cacheCfg = new CacheConfiguration    {    Name = "cars",    CacheStoreFactory = new AdoNetCacheStoreFactory(),    KeepBinaryInStore = true,    ReadThrough = true,    WriteThrough = true    }; ICache cars = ignite.GetOrCreateCache(cacheCfg).WithKeepBinary();Now another node with clientmode = true is started and when I access same cache likeICache carsFromCache = igniteClient.GetCache().WithKeepBinary();This line is throwing IgniteException with message - could not create .Net cachestore Question is - while accessing cache like above from client node do I also need to pass cachestore configuration like below var cacheCfg = new CacheConfiguration    {    Name = "cars",    CacheStoreFactory = new AdoNetCacheStoreFactory(),    KeepBinaryInStore = true,    ReadThrough = true,    WriteThrough = true    };ICache carsFromCache = igniteClient.GetCache(cacheCfg).WithKeepBinary(); Regards,SudhirOn Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Hi All,  I am really struggling with this exception / error message. I am using same configurations on client & server node but still it is giving same error / exception - could not create .Net CacheStore. Can someone please help / provide what is issue here?? Regards,SudhirOn Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Alexandr,  Can you share the code u used + configuration ? I am not able to send snippets as  i have to type in and can't send those from my work.On Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:I configured 1 server node with : Name = "cars",CacheStoreFactory = new AdoNetCacheStoreFactory(),KeepBinaryInStore = true,ReadThrough = true,WriteThrough = true The second one with ClientMode = true just accesses the cache by its name and performs all operations. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:56 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Ok. But

RE: Re: Store objects as IBinaryobject in cadche

[Alexandr Shapkin]

You can also refer to the BinaryModeExample.cs example and the docs [1] [1] - https://apacheignite-net.readme.io/docs/binary-mode#section-creating-binary-objects From: Ilya KasnacheevSent: Friday, November 15, 2019 4:13 PMTo: user@ignite.apache.orgSubject: Re: Store objects as IBinaryobject in cadche Hello! You can use BinaryObjectBuilder:https://apacheignite.readme.io/docs/binary-marshaller#section-modifying-binary-objects-using-binaryobjectbuilder However, each variation of its composition is stored in schema cache, which means you should not have too many of those. It is recommended to use Map to store all variable/non-essential fields if you have a lot of them. Regards,-- Ilya Kasnacheev  пт, 15 нояб. 2019 г. в 16:08, Manan Joshi :I would like to store any time of object into cache as IBinaryobject, so i can store cache stored in loosely bind with object. I am working with Ignite .Net. Can someone please suggest something. I need something like MyObject obj = new MyObject();  obj.setFieldA("A"); obj.setFieldB(123);  BinaryObject binaryObj = Ignition.ignite().binary().toBinary(obj); ThanksManan Joshi 

RE: Re: query if queue exist and get it like getOrCreateCache

[Alexandr Shapkin]

Yes, that true in most cases unless you try to call the queue method with a different collection configuration.In such case, an exception will be thrown. From: Narsi Reddy NallamilliSent: Friday, November 15, 2019 10:47 AMTo: user@ignite.apache.orgSubject: Re: query if queue exist and get it like getOrCreateCache Thank you Mikael. On Thu, Nov 14, 2019 at 7:05 PM Mikael  wrote:Hi!If there is a queue created already with the name "abc" it will just return that queue, if it cannot find it and you have included a configuration it will create it.MikaelDen 2019-11-14 kl. 13:30, skrev Narsi Reddy Nallamilli:Hi Mikael, Do you mean the 4th statement will not create new Queue? and would point to igCollectionConfiguration colCfg = new CollectionConfiguration();colCfg.setCollocated(true);IgniteQueue ig = ignite.queue("abc",0,colCfg);IgniteQueue ig1 = ignite.queue("abc",0,colCfg);  On Thu, Nov 14, 2019 at 5:42 PM Mikael  wrote:What is wrong with Ignite.queue() ?queue(String name, int cap, CollectionConfiguration cfg) Will get a named queue from cache and create one if it has not been created yet and cfg is not null. Mikael Den 2019-11-14 kl. 13:02, skrev Narsi Reddy Nallamilli:Hi!  I am looking for some method just like getOrCreateCache available for queue. Any help! 

RE: unsubscribe

[Alexandr Shapkin]

Please, send this email to user-unsubscr...@ignite.apache.org   From: Kurt SembaSent: Friday, November 15, 2019 11:44 AMTo: user@ignite.apache.orgSubject: unsubscribe unsubscribe  

RE: Re: Ignite.net get grid / node metrics

[Alexandr Shapkin]

> What are API ways to get details for individual nodes ?Please, inspect public API [1] and the Apache Ignite documentation.For sample, you can set a custom attribute to a node and request it later [2]. >is gridgain a licensed versionYes, it’s free to use. [1] https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cluster.IClusterGroup.html[2] https://apacheignite-net.readme.io/docs/cluster#section-cluster-node-attributes From: Sudhir PatilSent: Friday, November 15, 2019 4:31 AMTo: user@ignite.apache.orgSubject: Re: Ignite.net get grid / node metrics Alexandr thank you. I will explore on 1)What are API ways to get details for individual nodes ? For 2) is gridgain a licensed version ?On Friday, November 15, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote: It depends on what kind of information are you interested in. For sample, you can use the Cluster API to inspect a node’s metrics manually [1].Or take a look at some monitoring tools, like WebConsole [2]. [1] - https://apacheignite-net.readme.io/docs/cluster-groups#section-cluster-group-metrics[2] - https://www.gridgain.com/docs/web-console/latest/web-console-getting-started From: Sudhir PatilSent: Thursday, November 14, 2019 8:26 PMTo: user@ignite.apache.orgSubject: Ignite.net get grid / node metrics Hi All, How to get grid / node metrics (e.g. memory used, etc) in Ignite.net? Regards,Sudhir-- Thanks & Regards,Sudhir Patil,+91 9881095647. -- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: Re: How do we delete caches?

[Alexandr Shapkin]

> How do we delete caches?  What about DROP TABLE [1]? This should remove both a table and a cache. > Is there a way to run the SQL query that doesn't require us to first get orcreate an existing cache You can use a thin client for that, please, review the JDBC example.With this approach, the default schema would be PUBLIC. > which results in two caches, only one of which is deleted when we drop the table.Ignite allows you to use a single cache for multiple tables, but we do not recommend this approach. You can set your schema manually using cache API with SqlQuery.setSchema("PUBLIC")  [2] [1] - https://apacheignite-sql.readme.io/docs/drop-table[2] - https://apacheignite-sql.readme.io/docs/schema-and-indexes#section-overview  From: devinbostSent: Friday, November 15, 2019 2:55 AMTo: user@ignite.apache.orgSubject: Re: How do we delete caches? devinbost wrote> When we create a new table via SQL, it creates the cache for the table (as> expected). However, if we drop the table, the cache is not deleted. We> can't> figure out how to delete the cache. So, we have many caches accumulating> that we can't delete but can't use for SQL because we've deleted their> tables.> > How do we delete caches? > > --> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ For example, to run a SQL script to create a table, we can run:    clientCache = ignite.getOrCreateCache(tableName);   clientCache.query(IgniteHelpers.createFieldsQuery("sql/createExampleTable.sql")).getAll(); Is there a way to run the SQL query that doesn't require us to first get orcreate an existing cache? If we run the two lines above, it takes the cachename and uses it for the Table's schema name and then creates another cachein the form of SQL__, which results in two caches,only one of which is deleted when we drop the table.    --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Re: How to query existing cache that was configured via XML

[Alexandr Shapkin]

Please, have a look at examples/datagrid/CacheQueryExample.java  file. From: devinbostSent: Friday, November 15, 2019 1:31 AMTo: user@ignite.apache.orgSubject: Re: How to query existing cache that was configured via XML ilya.kasnacheev wrote> It's not possible to add table to cache.> > It is possible to define cache with table(s) from XML, but you will have> to> populate it with data from scratch:> https://apacheignite.readme.io/docs/indexes Am I able to use the QueryCursor on a cache that was created without atable? We're trying to figure out how to easily iterate through the cachevalues to migrate them into a cache that has a table.   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Set the partitions when create table in the code

[Alexandr Shapkin]

Have you tried a cache template [1]? After you designed your custom affinity function, you can configure it for a new cache template.You could use it with the CREATE TABLE statement [2].Please, note, that a template should be registered in advance. [1] - https://apacheignite.readme.io/docs/cache-template[2] - https://apacheignite-sql.readme.io/docs/create-table#section-parameters From: codeboyyongSent: Friday, November 15, 2019 1:29 AMTo: user@ignite.apache.orgSubject: Set the partitions when create table in the code Hi Dear ignite friendsI have a uniq use case here . I have to create a table in a big ignitecluster and since the columns and table name are dynamically generated. Ican not config the cache config in the xml.So I use the SQL to create the table .The thing is , seems there is no way to config the partition numbers for thetable. The default value 1024 is quite small and not suit on our use case. Can some one pls let me know if there is a solution for this ?Please note that I can not configure the cache config in the xml, becausethe table columns and names are generated on runtime.  Thank You   --Sent from: http://apache-ignite-users.70518.x6.nabble.com/ 

RE: Ignite.net get grid / node metrics

[Alexandr Shapkin]

 It depends on what kind of information are you interested in. For sample, you can use the Cluster API to inspect a node’s metrics manually [1].Or take a look at some monitoring tools, like WebConsole [2]. [1] - https://apacheignite-net.readme.io/docs/cluster-groups#section-cluster-group-metrics[2] - https://www.gridgain.com/docs/web-console/latest/web-console-getting-started From: Sudhir PatilSent: Thursday, November 14, 2019 8:26 PMTo: user@ignite.apache.orgSubject: Ignite.net get grid / node metrics Hi All, How to get grid / node metrics (e.g. memory used, etc) in Ignite.net? Regards,Sudhir-- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: Re: Ignite.Net Read through approach - error : could not create .Net cachestore

[Alexandr Shapkin]

Sudhir, Can you share a stacktrace? It’s difficult to locate the problem without the details.According to source code, the exception is thrown when .NET side is unable to create a CacheStore instance.Since you have configured a factory, please, check that it returns a correct instance and the class is present for required nodes. From: MikaelSent: Thursday, November 14, 2019 10:37 AMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Hi!The client should not have to pass any cache configuration, once the cache is created in the server, you should just be able to get the cache, I do not see any reason that should not work but I have never used .NET so it is possible there is some hickup there that I am not seeing.You do not have any other Ignite servers running on the same subnet or something ? verify that the client connect to the correct server (sounds silly but I managed to mess up a number of times ;).Is it just that cache ? could you create another cache as simple as possible (no cache store) and see if it will find that one ?It sounds like it cannot create the actual cache store but in that case I would think you would get the exception in the server and not the client, is there any more useful information from the exception ?Mikael Den 2019-11-14 kl. 02:41, skrev Sudhir Patil:Hi All,  Server node is started & cache is created with configuration likevar cacheCfg = new CacheConfiguration    {    Name = "cars",    CacheStoreFactory = new AdoNetCacheStoreFactory(),    KeepBinaryInStore = true,    ReadThrough = true,    WriteThrough = true    }; ICache cars = ignite.GetOrCreateCache(cacheCfg).WithKeepBinary(); Now another node with clientmode = true is started and when I access same cache likeICache carsFromCache = igniteClient.GetCache().WithKeepBinary();This line is throwing IgniteException with message - could not create .Net cachestore Question is - while accessing cache like above from client node do I also need to pass cachestore configuration like below var cacheCfg = new CacheConfiguration    {    Name = "cars",    CacheStoreFactory = new AdoNetCacheStoreFactory(),    KeepBinaryInStore = true,    ReadThrough = true,    WriteThrough = true    };ICache carsFromCache = igniteClient.GetCache(cacheCfg).WithKeepBinary(); Regards,SudhirOn Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Hi All,  I am really struggling with this exception / error message. I am using same configurations on client & server node but still it is giving same error / exception - could not create .Net CacheStore. Can someone please help / provide what is issue here?? Regards,SudhirOn Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Alexandr,  Can you share the code u used + configuration ? I am not able to send snippets as  i have to type in and can't send those from my work.On Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:I configured 1 server node with : Name = "cars",CacheStoreFactory = new AdoNetCacheStoreFactory(),KeepBinaryInStore = true,ReadThrough = true,WriteThrough = true The second one with ClientMode = true just accesses the cache by its name and performs all operations. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:56 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Ok. But what is type of those 2 nodes ???On Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:Sudhir, > Are there any samples for ReadThrough operation done from normal ignite client node? I tried the AspNetCachestore example with two nodes and it worked well for me. > I can not share snippet now but I will try to type it later...:(That’s ok. Feel free to add details whenever you get ready. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:30 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Thanks for details. I can not share snippet now but I will try to type it later...:( I am getting cacheexception with message as - could not create .Net CacheStore. From ignite samples i rendered same sample code e.g. AspNetCachestore one. This works when used with thin client but when I use it with normal ignite node with ClientMode =true, it is throwing above exception with that message when I call GetCache function call from client node for a key from cache created in server node. Are there any samples for ReadThrough operation done from normal ignite client node?  Regards,SudhirOn Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.

RE: Re: Ignite.Net Read through approach - error : could not create .Net cachestore

[Alexandr Shapkin]

I configured 1 server node with : Name = "cars",CacheStoreFactory = new AdoNetCacheStoreFactory(),KeepBinaryInStore = true,ReadThrough = true,WriteThrough = true The second one with ClientMode = true just accesses the cache by its name and performs all operations. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:56 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Ok. But what is type of those 2 nodes ???On Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:Sudhir, > Are there any samples for ReadThrough operation done from normal ignite client node? I tried the AspNetCachestore example with two nodes and it worked well for me. > I can not share snippet now but I will try to type it later...:(That’s ok. Feel free to add details whenever you get ready. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:30 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Thanks for details. I can not share snippet now but I will try to type it later...:( I am getting cacheexception with message as - could not create .Net CacheStore. From ignite samples i rendered same sample code e.g. AspNetCachestore one. This works when used with thin client but when I use it with normal ignite node with ClientMode =true, it is throwing above exception with that message when I call GetCache function call from client node for a key from cache created in server node. Are there any samples for ReadThrough operation done from normal ignite client node?  Regards,SudhirOn Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:Sudhir, What exception are you getting? Can you share a full stacktrace/code snippet?Most likely you have an issue with the underlying DB configuration. You could have a look at Pavel’s examples for 3rd party persistence: https://github.com/ptupitsyn/ignite-net-examples From: Sudhir PatilSent: Wednesday, November 13, 2019 3:07 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore I am extremely sorry for typing mistakes. Here is correct details Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call for same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am getting error as - could not create .Net CacheStore. Can you please help me on this?On Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call, on same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am setting terrorist as could not create .Net CacheStore. Can you please help me on this?  -- Thanks & Regards,Sudhir Patil,+91 9881095647.-- Thanks & Regards,Sudhir Patil,+91 9881095647. -- Thanks & Regards,Sudhir Patil,+91 9881095647. -- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: Re: Ignite.Net Read through approach - error : could not create .Net cachestore

[Alexandr Shapkin]

Sudhir, > Are there any samples for ReadThrough operation done from normal ignite client node? I tried the AspNetCachestore example with two nodes and it worked well for me. > I can not share snippet now but I will try to type it later...:(That’s ok. Feel free to add details whenever you get ready. From: Sudhir PatilSent: Wednesday, November 13, 2019 4:30 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore Alexandr, Thanks for details. I can not share snippet now but I will try to type it later...:( I am getting cacheexception with message as - could not create .Net CacheStore. From ignite samples i rendered same sample code e.g. AspNetCachestore one. This works when used with thin client but when I use it with normal ignite node with ClientMode =true, it is throwing above exception with that message when I call GetCache function call from client node for a key from cache created in server node. Are there any samples for ReadThrough operation done from normal ignite client node?  Regards,SudhirOn Wednesday, November 13, 2019, Alexandr Shapkin <lexw...@gmail.com> wrote:Sudhir, What exception are you getting? Can you share a full stacktrace/code snippet?Most likely you have an issue with the underlying DB configuration. You could have a look at Pavel’s examples for 3rd party persistence: https://github.com/ptupitsyn/ignite-net-examples From: Sudhir PatilSent: Wednesday, November 13, 2019 3:07 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore I am extremely sorry for typing mistakes. Here is correct details Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call for same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am getting error as - could not create .Net CacheStore. Can you please help me on this?On Wednesday, November 13, 2019, Sudhir Patil <patilsudhi...@gmail.com> wrote:Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call, on same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am setting terrorist as could not create .Net CacheStore. Can you please help me on this?  -- Thanks & Regards,Sudhir Patil,+91 9881095647.-- Thanks & Regards,Sudhir Patil,+91 9881095647. -- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: Re: Ignite.Net Read through approach - error : could not create .Net cachestore

[Alexandr Shapkin]

Sudhir, What exception are you getting? Can you share a full stacktrace/code snippet?Most likely you have an issue with the underlying DB configuration. You could have a look at Pavel’s examples for 3rd party persistence: https://github.com/ptupitsyn/ignite-net-examples From: Sudhir PatilSent: Wednesday, November 13, 2019 3:07 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net Read through approach - error : could not create .Net cachestore I am extremely sorry for typing mistakes. Here is correct details Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call for same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am getting error as - could not create .Net CacheStore. Can you please help me on this?On Wednesday, November 13, 2019, Sudhir Patil  wrote:Hi All, I have started 2 nodes - 1) Server node - using Ignition.Start. On Server node cache is created with ReadThrough = true with CacheStoreFactory value set to property implementation. 2) Client one - using Ignition.Start (for client with Ignition.ClientMode = true).  With this when i do 1st GetCache call, on same cache from client mode, hoping it will do a readthrough operation connecting db & filling the data in cache, i am setting terrorist as could not create .Net CacheStore. Can you please help me on this?  -- Thanks & Regards,Sudhir Patil,+91 9881095647.-- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: Re: Ignite.Net configuration for storing large cache data

[Alexandr Shapkin]

Thin client does not store data and is not a part of a cluster, instead, you can think of it as an interface of accessing and manipulating your data.Please, note, that you can not configure these properties for IgniteClientConfiguration.The data region configuration makes sense only for server nodes. From: Sudhir PatilSent: Wednesday, November 13, 2019 1:17 PMTo: user@ignite.apache.orgSubject: Re: Ignite.Net configuration for storing large cache data Outside this question, such large data will not work with thin client??? Regards,SudhirOn Wednesday, November 13, 2019, Pavel Tupitsyn  wrote:Yes, region sizes are in bytes (thanks for pointing this out, I'll update the XMLDoc)Yes, MaxSize is the right property. Docs say it should not be less than 10 mb.Here is an example of C# config with multiple regions:var cfg = new IgniteConfiguration{    DataStorageConfiguration = new DataStorageConfiguration    {    DefaultDataRegionConfiguration = new DataRegionConfiguration    {    MaxSize = 16L * 1024 * 1024 * 1024,    Name = "default_region"    },    DataRegionConfigurations = new[]    {    new DataRegionConfiguration    {    MaxSize = 8L * 1024 * 1024 * 1024,    Name = "custom_region"    }    }    },    CacheConfiguration = new []    {    new CacheConfiguration    {    Name = "cache_in_default_region"    },    new CacheConfiguration    {    Name = "cache_in_custom_region",    DataRegionName = "custom_region"    },    }};var ignite = Ignition.Start(cfg); As a result, in the Ignite log we can see:[12:27:23] Data Regions Configured:[12:27:23]   ^-- default_region [initSize=256.0 MiB, maxSize=16.0 GiB, persistence=false][12:27:23]   ^-- custom_region [initSize=256.0 MiB, maxSize=8.0 GiB, persistence=false] On Wed, Nov 13, 2019 at 11:25 AM Sudhir Patil  wrote:Thanks Pavel. 1) i will look into this. 2) ok. 3) DefaultDataRegionConfiguration - What is default unit used  for different properties to set size values ? Is it byte?? For - By default, only one region is configured with max size set to 20% of available RAM. What is the way to set max size ? MaxSize property? But documentation says it can be set maximum to 10 mb ?? How to set multiple regions? RegardsSudhirOn Wednesday, November 13, 2019, Pavel Tupitsyn  wrote:Hi Sudhir, There are 3 things to consider: 1) Capacity planning - how much memory do you need for your data?Depending on type, 30-50 properties can take very different amount of memory to store.- n+4+1 for strings, where n is size in utf8 encoding- 4+1 bytes for int- 8+1 bytes for longand so on(every field takes extra byte for type code) More details: https://apacheignite.readme.io/docs/capacity-planning 2) JVM settingsIgnite does not store your data in Java Heap, but some memory is still required for processing See https://apacheignite.readme.io/docs/jvm-and-system-tuning#section-garbage-collection-tuning 3) Ignite DataRegionConfigurationIgnite stores cache data in so-called Data Regions, in unmanaged heap (aka "offheap").By default, only one region is configured with max size set to 20% of available RAM.You may want to increase this by changing IgniteConfiguration.DataStorageConfiguration.DefaultDataRegionConfiguration More details: https://apacheignite.readme.io/docs/durable-memory-tuning On Wed, Nov 13, 2019 at 4:13 AM Sudhir Patil  wrote:Hi All, What should be configuration changes required for storing large data e.g. 10 million records of custom class objects having around 30- 50 properties? I am using basic default configuration for Ignite.net.JvmOptions used are like-Djava.net.preferIPv4Stack=true-Xms512m-XX:+UseG1GC-XX:+DisabkeExplicitGC Regards,Sudhir -- Thanks & Regards,Sudhir Patil,+91 9881095647.-- Thanks & Regards,Sudhir Patil,+91 9881095647.-- Thanks & Regards,Sudhir Patil,+91 9881095647. 

RE: javax.cache.CacheException: Failed to find SQL table for type: XXXXXX

[Alexandr Shapkin]

Hello!

In general, when you want to use SQL API you need to configure a table first.
There are several options for doing it:
- you can use create table statement  [1]
- configuring QueryEntities [2]

When you create cache with Create statement, the DEFAULT SQL schema is used, 
otherwise, it would be a CacheName [3]

[1] – 
https://www.gridgain.com/docs/8.7.6/developers-guide/SQL/sql-key-value-storage
[2] - https://www.gridgain.com/docs/8.7.6/developers-guide/SQL/sql-api
[3] -  
https://apacheignite-sql.readme.io/docs/schemas#section-cache-and-schema-names

From: vinod.jv
Sent: Thursday, October 31, 2019 8:36 AM
To: user@ignite.apache.org
Subject: javax.cache.CacheException: Failed to find SQL table for type: XX

I am using ignite 2.7 version and getting the below exception while querying
the ignite cache:
19/10/30 05:33:14 ERROR yarn.ApplicationMaster: User class threw exception:
javax.cache.CacheException: Failed to find SQL table for type:
CanonicalNameOAVO
javax.cache.CacheException: Failed to find SQL table for type:
CanonicalNameOAVO

This is my code to start ignite:
def getIgnite(): Ignite = {
   val clusterName = clusterProperties.getProperty("IGNITE_CLUSTER_NAME")

// https://apacheignite.readme.io/v2.7/docs/clients-vs-servers
Ignition.setClientMode(true)

try {
   val ignite = Ignition.ignite(clusterName);
   if ( clusterName == ignite.name() ) {
  logInfo(" Found and returning client for cluster: " +
clusterName)
  return ignite
   }
}
catch {
   case e: Exception => e.printStackTrace()
}

val configFilePath = clusterProperties.getProperty("IGNITE_XML_CONFIG")
logInfo(" configFilePath: " + configFilePath)
val configInputStream = FileSystem.get(new Configuration()).open(new
Path(configFilePath));

logInfo(" Starting Ignite Client")
return Ignition.start(configInputStream)
}

I am loading data into the cache and data get loaded successfully and cache
size also get printed as below:
INFO dataloader.IgniteDataLoader:  OA_ALTERNATE_NAME_CACHE SIZE =>
51016471

Below code is for accessing the cache:
def createOfficialNameOACache: IgniteCache[String, CanonicalNameOAVO] = {
val orgCacheCfg: CacheConfiguration[String, CanonicalNameOAVO] =
  new CacheConfiguration[String,
CanonicalNameOAVO](OA_OFFICIAL_NAME_CACHE)
orgCacheCfg.setIndexedTypes(classOf[String], classOf[CanonicalNameOAVO])
orgCacheCfg.setCacheMode(CacheMode.PARTITIONED)
orgCacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC)
orgCacheCfg.setBackups(3)
getIgnite().getOrCreateCache(orgCacheCfg)
}

But while trying to query the cache I am getting the exception. Below code
is for querying the cache:

val companyName = "SUFFOLK CONSTRUCTION CO INC"
val queryString = "orgName = '" + companyName + "'"
val companyNameQuery = new SqlQuery[String,
CanonicalNameOAVO](classOf[CanonicalNameOAVO], queryString)
val queryCursor =
igniteDataLoader.createOfficialNameOACache.query(companyNameQuery)
val queryResults = Future {
  queryCursor.getAll()
}
try {
  val companyResults = extractCananicalNameOAVO(queryResults)
  logInfo(s"Ignite Results returned - $companyResults")
} catch {
  case exe: Exception => logInfo(s"Official Name exact search timed out
for companyName")
  queryCursor.close()
  Vector.empty
}

The code throwing exception at query() method for the below line:
val queryCursor =
igniteDataLoader.createOfficialNameOACache.query(companyNameQuery)

19/10/30 05:33:14 INFO dataloader.IgniteServerDataLoader:
getExactOfficialName1 - 'SqlQuery [type=CanonicalNameOAVO, alias=null,
sql=orgName = 'SUFFOLK CONSTRUCTION CO INC', args=null, timeout=0,
distributedJoins=false, replicatedOnly=false]'
19/10/30 05:33:14 INFO dataloader.IgniteDataLoader:  Found and returning
client for cluster: JalaaDaala
19/10/30 05:33:14 ERROR yarn.ApplicationMaster: User class threw exception:
javax.cache.CacheException: Failed to find SQL table for type:
CanonicalNameOAVO
javax.cache.CacheException: Failed to find SQL table for type:
CanonicalNameOAVO
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:697)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:376)
at
un.api.dataloader.IgniteServerDataLoader$.loadOAData(IgniteServerDataLoader.scala:70)
at
un.api.StartStandalone$.startIgniteAndDataloading(StartStandalone.scala:49)
at
un.api.StartStandalone$delayedInit$body.apply(StartStandalone.scala:13)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at
scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.collection.immutable.List.foreach(List.scala:318)
at

RE: Ignite.Net exception on Ignition.Start

[Alexandr Shapkin]

Hello!
 
Please, check that you are using the right JDK version (8+).
The proper version is set for JAVA_HOME, and you do not use the JvmDllPath 
configuration.
 
Is it possible that you have multiple Java installations?

From: Sudhir Patil
Sent: Monday, October 21, 2019 2:25 PM
To: user@ignite.apache.org
Subject: Re: Ignite.Net exception on Ignition.Start

I am using nuget package Apache.Ignite 2.7.5

Regards,
Sudhir

On Monday, October 21, 2019, Sudhir Patil  wrote:
Hi All,

I am getting this error when Ignition.Start method is called. Exception message 
says - JNI_CreateJavaVM failed: VersionError.

I don't have any log file for this or pleas kelet me know where to get log file.
Please help me on this issue.

Regards,
Sudhir


-- 
Thanks & Regards,
Sudhir Patil,
+91 9881095647.


-- 
Thanks & Regards,
Sudhir Patil,
+91 9881095647.

RE: Running a C++ Thick Node compute func from a Java Thick Node

[Alexandr Shapkin]

Thank you for clarification.

I agree, that in some situation it would be nice to have an ability to trigger 
a platform-specific task by it’s name with some default args.
But platform agnostic compute requires a separate discussion, I think.

Initially I wanted to highlight a couple of possible workarounds like:

- Executing a native java task by it’s name. The task itself could hook a c++ 
library and perform it’s business [1].
- Calling a Java service [2] with the same logic.

But later I realized that unfortunately both of these options are missing for 
C++.  I created a new ticket in Apache Ignite Jira [3].


As a further workaround you may take a look into ContiniousQueries API, in 
order to trigger some internal logic based on a cache value.

[1] - 
https://apacheignite-net.readme.io/docs/platform-interoperability#section-compute-in-mixed-platform-clusters
[2] - https://apacheignite-net.readme.io/docs/calling-java-services
[3] - https://issues.apache.org/jira/browse/IGNITE-12177

From: codie
Sent: Tuesday, September 17, 2019 10:00 AM
To: user@ignite.apache.org
Subject: Re: Running a C++ Thick Node compute func from a Java Thick Node

We have a polyglot environment, thus people choose a different language 
depending on their use-case. In consequence some algorithms are programmed in 
C++ or Fortran. We usually implement a "task" and these tasks can be 
used/called by others (using the name in ignite compute) without having to care 
what's behind (C++, Scala or anything else). Polyglot is a rising topic, thus I 
was surprised I couldn't cross languages barriers when integrating operations 
in Ignite and C++ thick nodes. 
Am 9/16/2019 um 6:38 PM schrieb Alexandr Shapkin:
Json, 
 
Could you share an example of why do you need this feature?
 
In general, platform-specific nodes should be able to execute only native code. 
I.e. C++ -> C++, C#-> C# etc.
 
But since a thick client is kind of a wrapper around Java one it’s quite easy 
to call a Java code on a remote node.
 
 
From: Igor Sapego
Sent: Monday, September 16, 2019 6:14 PM
To: user
Subject: Re: Running a C++ Thick Node compute func from a Java Thick Node
 
Hello,
 
Such function is not supported for now. You can raise a ticket
if it's something you'd like to have in Ignite.


Best Regards,
Igor
 
 
On Mon, Sep 16, 2019 at 6:00 PM codie  wrote:
Hello,
I have two nodes running with a TCP Discovery Spi, one is a C++ Thick 
node and one is a Java Thick node. The C++ node has the examples compute 
func "CountWords" registered. How can I trigger this function from the 
Java thick nodes? Does "RegisterComputeFunc" not inform other nodes, 
that the compute function can be found on this specific c++ node?

Thanks,
Json
 

RE: Data region LRU offheap algo not working

[Alexandr Shapkin]

Hello!

As far as my understanding goes, I see that eviction works ok.

>2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%] 
>IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider 
>increasing 'maxSize' on Data Region configuration: 1G_Region

This message tells that now eviction process is started and will free pages 
according to the algorithm.
it means that when Ignite requests for a new page and there are no available 
ones, then Ignite will apply eviction first and then use those pages for a new 
values.
So you will not see any significant memory changes during eviction. 
I.e. Ignite will show that the data region is full, but still allows you to 
write new values.

There are no any special messages log messages regarding evicted pages. Only 
the first one, when eviction comes to the play.
I think you may try to look into getEvictionRate() metrics [1] for details. 
Also you should be able to see it in Visor GUI.

[1] - 
https://apacheignite.readme.io/docs/memory-metrics#section-data-region-metrics

From: rick_tem
Sent: Tuesday, September 10, 2019 10:12 AM
To: user@ignite.apache.org
Subject: Data region LRU offheap algo not working

Hi,

I am trying to find out why it appears the RANDOM_LRU algo doesn't seem to
work with the following config.  Log attached, as well...after the log of
the below...

2019-09-09 11:04:03.557 WARN  [sys-stripe-5-#6%TemenosGrid%]
IgniteCacheDatabaseSharedManager - Page-based evictions started. Consider
increasing 'maxSize' on Data Region configuration: 1G_Region

after a few minutes you see memory steadily decrease.  What information in
the log will help me determine how many pages are freed, etc?

Thanks,
Rick

dateRepo1.out
  
dataRepo2.out
  

























--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Running a C++ Thick Node compute func from a Java Thick Node

[Alexandr Shapkin]

Json, 

Could you share an example of why do you need this feature?

In general, platform-specific nodes should be able to execute only native code. 
I.e. C++ -> C++, C#-> C# etc.

But since a thick client is kind of a wrapper around Java one it’s quite easy 
to call a Java code on a remote node.


From: Igor Sapego
Sent: Monday, September 16, 2019 6:14 PM
To: user
Subject: Re: Running a C++ Thick Node compute func from a Java Thick Node

Hello,

Such function is not supported for now. You can raise a ticket
if it's something you'd like to have in Ignite.


Best Regards,
Igor


On Mon, Sep 16, 2019 at 6:00 PM codie  wrote:
Hello,
I have two nodes running with a TCP Discovery Spi, one is a C++ Thick 
node and one is a Java Thick node. The C++ node has the examples compute 
func "CountWords" registered. How can I trigger this function from the 
Java thick nodes? Does "RegisterComputeFunc" not inform other nodes, 
that the compute function can be found on this specific c++ node?

Thanks,
Json

RE: Unsubscribe

[Alexandr Shapkin]

Hello,  

Please,  send an email to user-unsubscr...@ignite.apache.org and 
dev-unsubscr...@ignite.apache.org

More details https://ignite.apache.org/community/resources.html#mail-lists

From: Deepa Kolwalkar
Sent: Monday, September 16, 2019 12:37 PM
To: user@ignite.apache.org; d...@ignite.apache.org
Subject: Unsubscribe

=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

RE: Topology snapshot explanation

[Alexandr Shapkin]

Hello!

Client nodes still can keep some utility data that might be required for 
application.
I think the most suitable example is near cache [1].

[1] - https://apacheignite.readme.io/docs/near-caches

From: rick_tem
Sent: Friday, September 13, 2019 7:10 PM
To: user@ignite.apache.org
Subject: Topology snapshot explanation

Hello,

I am wondering why offheap/heap memory continues to grow as the topology
shapshot evolves.  It must be client and server included.  Why is it being
reported for client?  Why would a client use offheap?  

Thanks,
Rick

event-processor_1| [16:01:08] Topology snapshot [ver=5,
locNode=91131045, servers=1, clients=4, state=ACTIVE, CPUs=20,
offheap=11.0GB, heap=9.0GB]
event-processor_1| [16:01:08] Topology snapshot [ver=6,
locNode=91131045, servers=1, clients=5, state=ACTIVE, CPUs=24,
offheap=13.0GB, heap=10.0GB]
event-processor_1| [16:01:08] Topology snapshot [ver=7,
locNode=91131045, servers=1, clients=6, state=ACTIVE, CPUs=28,
offheap=15.0GB, heap=11.0GB]
event-processor_1| [16:01:08] Topology snapshot [ver=8,
locNode=91131045, servers=1, clients=7, state=ACTIVE, CPUs=32,
offheap=17.0GB, heap=12.0GB]
event-processor_1| [16:01:11] Performance suggestions for grid
'TemenosGrid' (fix if possible)
event-processor_1| [16:01:11] To disable, set
-DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
event-processor_1| [16:01:11]   ^-- Disable grid events (remove
'includeEventTypes' from configuration)
event-processor_1| [16:01:11]   ^-- Set max direct memory size if
getting 'OOME: Direct buffer memory' (add
'-XX:MaxDirectMemorySize=[g|G|m|M|k|K]' to JVM options)
event-processor_1| [16:01:11]   ^-- Disable processing of calls to
System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
event-processor_1| [16:01:11]   ^-- Decrease number of backups (set
'backups' to 0)
event-processor_1| [16:01:11] Refer to this page for more
performance suggestions:
https://apacheignite.readme.io/docs/jvm-and-system-tuning
event-processor_1| [16:01:11]
event-processor_1| [16:01:11] To start Console Management &
Monitoring run ignitevisorcmd.{sh|bat}
event-processor_1| [16:01:11] Data Regions Configured:
event-processor_1| [16:01:11]   ^-- default [initSize=256.0 MiB,
maxSize=1.7 GiB, persistence=false]
event-processor_1| [16:01:11]   ^-- 1G_Region [initSize=100.0 MiB,
maxSize=500.0 MiB, persistence=false]
event-processor_1| [16:01:11]
event-processor_1| [16:01:11] Ignite node started OK (id=91131045,
instance name=TemenosGrid)
event-processor_1| [16:01:11] Topology snapshot [ver=4,
locNode=91131045, servers=1, clients=3, state=ACTIVE, CPUs=16,
offheap=8.7GB, heap=8.0GB]
event-processor_1| [16:01:26] Topology snapshot [ver=9,
locNode=91131045, servers=1, clients=8, state=ACTIVE, CPUs=36,
offheap=20.0GB, heap=13.0GB]
event-processor_1| [16:01:28] Topology snapshot [ver=10,
locNode=91131045, servers=1, clients=9, state=ACTIVE, CPUs=40,
offheap=22.0GB, heap=14.0GB]



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Iterator for keys only ?

[Alexandr Shapkin]

Hello!

You can try to use SqlFieldsQuery with “select _key from [CacheTable]” for that.

From: Mikael
Sent: Wednesday, September 11, 2019 3:47 PM
To: user@ignite.apache.org
Subject: Iterator for keys only ?

Hi!

What is the best way to iterate over the keys only from a cache ? I 
don't want the values because these are long text strings so will slow 
down the iterator a lot.

Pretty much the Ignite way of doing map.keySet().iterator();

Mikael

RE: exceptions in Ignite node when a thin client process ends

[Alexandr Shapkin]

Hello,

There is an open ticket https://issues.apache.org/jira/browse/IGNITE-12032 with 
more details.
In general, It’s ok to ignore this error. 
You could try to follow Lev’s recommendations with explicit connection closing.

From: Matan Levy
Sent: Thursday, August 1, 2019 9:33 AM
To: user@ignite.apache.org
Subject: exceptions in Ignite node when a thin client process ends

Hi, 

In my application I am using ODBC + Java thin clients to get data from the 
cache(stores in Ignite nodes). 
whenever a process using this thin clients is finished, I am getting in the 
node logs this exception : 

*[07:45:19,559][SEVERE][grid-nio-worker-client-listener-1-#30][ClientListenerProcessor]
 
Failed to process selector key [s 
es=GridSelectorNioSessionImpl [worker=ByteBufferNioClientWorker 
[readBuf=java.nio.HeapByteBuffer[pos=0 lim=8192 cap=8192 
], super=AbstractNioClientWorker [idx=1, bytesRcvd=0, bytesSent=0, 
bytesRcvd0=0, bytesSent0=0, select=true, super=GridWo 
rker [name=grid-nio-worker-client-listener-1, igniteInstanceName=null, 
finished=false, heartbeatTs=1564289118230, hashCo 
de=1829856117, interrupted=false, 
runner=grid-nio-worker-client-listener-1-#30]]], writeBuf=null, 
readBuf=null, inRecove 
ry=null, outRecovery=null, super=GridNioSessionImpl 
[locAddr=/0:0:0:0:0:0:0:1:10800, rmtAddr=/0:0:0:0:0:0:0:1:63697, cre 
ateTime=1564289116225, closeTime=0, bytesSent=1346, bytesRcvd=588, 
bytesSent0=0, bytesRcvd0=0, sndSchedTime=156428911623 
5, lastSndTime=1564289116235, lastRcvTime=1564289116235, readsPaused=false, 
filterChain=FilterChain[filters=[GridNioAsyn 
cNotifyFilter, GridNioCodecFilter [parser=ClientListenerBufferedParser, 
directMode=false]], accepted=true, markedForClos 
e=false]]] 
java.io.IOException: An existing connection was forcibly closed by the 
remote host 
        at sun.nio.ch.SocketDispatcher.read0(Native Method) 
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) 
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) 
        at sun.nio.ch.IOUtil.read(IOUtil.java:197) 
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) 
        at 
org.apache.ignite.internal.util.nio.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:11
 
04) 
        at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNi
 
oServer.java:2389) 
        at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:215
 
6) 
        at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
 
        at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) 
        at java.lang.Thread.run(Thread.java:748)* 



is there a soemthing I can do to avoid those exceptions? 
Thanks!  

RE: ignite.net when started as service not reading spring config

[Alexandr Shapkin]

Hi!

How do you run or install your service? Can you share a cmd?

I believe you need to use IgniteConfiguration.SpringConfigUrl settings for this 
[1]
I.e.  path 

Also get a try with CMD args [2]

[1] - https://apacheignite-net.readme.io/docs/configuration#section-spring-xml
[2] - 
https://apacheignite-net.readme.io/docs/standalone-nodes#section-configuring-standalone-node-via-command-line

From: vladtovbin
Sent: Tuesday, September 3, 2019 9:31 PM
To: user@ignite.apache.org
Subject: ignite.net when started as service not reading spring config

On latest version of apache.ignite. Installed as Windows service on Win 2016.
Added this entry to AppSettings section in Apache.Ignite.exe.config:



If I launch apache.ignite.exe via command prompt, it reads the above config
correctly. If I launch from the same location as service, the config file
above is ignored. Confirmed also via looking at the log files, it's not
reading data from the spring config file.

Any ideas?
Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Tuning client performance.

[Alexandr Shapkin]

Hello,

Not a thin-client tuning in general, but you can check a serialization settings 
in order to make sure you do not use a default one.

From: John Smith
Sent: Friday, August 30, 2019 6:04 PM
To: user@ignite.apache.org
Subject: Tuning client performance.

Hi, is there any specific client settings we can set to tune the client 
performance, maybe some thread pools or any stuff like that?

RE: Apache Ignite distributed cache for .Net core web app

[Alexandr Shapkin]

Meanwhile, I think it’s true that we are lacking of complete .NET examples for 
modern apps.
It would be perfect if anyone would share their Ignite .NET experience so that 
we could collect them in a single place.

Incidentally, Pavel’s GitHub page is a good starting point.


From: Pavel Tupitsyn
Sent: Friday, August 30, 2019 5:34 PM
To: user@ignite.apache.org
Subject: Re: Apache Ignite distributed cache for .Net core web app

Consuming Ignite in .NET Core is not really different from classic .NET.

As for web application usage, there are many scenarios, I can suggest any of 
the following:

1) Start Ignite Server node right in your web app
Pros:
- Simplest development and deployment
- Best performance in cases of small deployments
Cons:
- Slow web app startup, increased resource usage on single machine/process

2) Start standalone Ignite nodes either on a separate cluster or on every 
server with web app, connect from web app with Thin Client
Pros:
- Fast web app startup
- Web app process is dedicated to serving web requests and does not share 
resources with Ignite, can be faster in large deployments with high load
Cons:
- Thin Client API is less capable than "thick" in-process mode


On Fri, Aug 30, 2019 at 5:19 PM Ilya Kasnacheev  
wrote:
Hello!

The link that I have provided has dotnetcore/ dir which hopefully contains 
those.

Regards,
-- 
Ilya Kasnacheev


пт, 30 авг. 2019 г. в 17:06, Sudhir Patil :
Thanks Ilya, yes I had gone through those git samples.
I was asking if any other repo for .Net core.

Regards
Sudhir

On Thursday, August 29, 2019, Ilya Kasnacheev  wrote:
Hello!

1. They are usually ran as stand alone processes on the designated machines. 
You can probably convert them to windows services with some effort.
2. Client nodes are actually fully functional Ignite distributed topology 
nodes. You should start a single client per web application. use it for all 
Ignite operations.
3. Have you seen .Net examples? 
https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples

Regards,
-- 
Ilya Kasnacheev


ср, 28 авг. 2019 г. в 20:16, Sudhir Patil :

Hi All,

We are planning to use Apache ignite for distributed cache for .Net core web 
application. 
I had gone through the basics of ignite.
Any suggestions on implementation of ignite with below points??
1. How/where to host server nodes? Eg separate web application / windows 
service.
2. Client nodes is nothing but main web application consuming cache from server 
nodes?
3. Any sample git repository for consuming ignite in .Net core??

Thanks,
Sudhir


-- 
Thanks & Regards,
Sudhir Patil,
+91 9881095647.


-- 
Thanks & Regards,
Sudhir Patil,
+91 9881095647.

RE: [IgniteKernal%ServerNode] Exception during start processors, nodewill be stopped and close connections java.lang.NullPointerException:null......cache.GridCacheUtils.affinityNode

[Alexandr Shapkin]

Hi, Siva

I have no idea whether you have subscribed to dev list or not, but there is an 
active discussion regarding 2.7.6 release and estimated dates.

http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-2-7-6-Time-Scope-and-Release-manager-td42944.html

Note, that original issue is included into the scope.

Alexandr

From: siva
Sent: Friday, August 2, 2019 7:15 AM
To: user@ignite.apache.org
Subject: Re: [IgniteKernal%ServerNode] Exception during start processors, 
nodewill be stopped and close connections 
java.lang.NullPointerException:null..cache.GridCacheUtils.affinityNode

Hi,
sorry, there is a typo mistake. I mean to tell,
To fix this issue instead of I fix this issue..
To fix this issue, I am testing same scenario with 2.8 nightly build
version. but while restarting server I am facing null NullPointerException.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Running Ignite Cluster using Docker containers

[Alexandr Shapkin]

Most likely you need to get them aware of each other.
I.e. configure DiscoverySpi addresses range to 49500..49509.

Please, share you Ignite config for containerized nodes.

Alexandr

From: vitalys
Sent: Thursday, August 8, 2019 9:00 PM
To: user@ignite.apache.org
Subject: RE: Running Ignite Cluster using Docker containers

Here is the problem. I can start one Server Node in the Docker container and
I can connect to that Node from my client application. However When I run
another Docker Containers they are not joining the Cluster. Here is an
example : 

Node 1 : 

docker run --rm --name myignite1 -p 47501:47500 -p 47101:47100 -p
10801:10800 -it myignite/ignite:1.0

[17:50:48] Topology snapshot [ver=1, locNode=d290d944, servers=1, clients=0,
state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]

Node 2 : 
docker run --rm --name myignite3 -p 47503:47500 -p 47103:47100 -p
10803:10800 -it myignite/ignite:1.0

[17:54:22] Topology snapshot [ver=1, locNode=fe47c5c7, servers=1, clients=0,
state=INACTIVE, CPUs=2, offheap=0.39GB, heap=0.43GB]



They Nodes can't discover one another. When I am starting my Nodes there is
no such issue since they are using the same Configuration where
TcpDiscoveryIpVmFimder is defined : 

 
   
  
  

  localhost:47500..47504 
   
  
  


So my question is : how do I make multiple nodes running in different Docker
Containers discover each other?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Running Ignite Cluster using Docker containers

[Alexandr Shapkin]

Hello, 

Please, refer to the docker port forwarding documentation [1]
In other words, instead of  -p 47503:47503 you need to write -p 47503:47500, 
unless you have not configured the default discovery spi port to 47503 
explicitly.

Please, recall that Ignite requires a minimum set of open ports for a general 
communication: TcpDiscoverySpi port - 47500 and TcpCommunicationSpi port - 
47100.

If you want to enable a thin clients communication, then 10800 default port is 
required as well.

To summarize, your docker cmd should look like:
docker run --rm --name myignite1 -p 47503:47500 -p 47103:47100 -p 10803:10800 
-it ignite-image

Now your host machine should be able to connect to the container with 47103, 
47503, 10803

[1] - 
https://docs.docker.com/config/containers/container-networking/#published-ports


From: vitalys
Sent: Wednesday, August 7, 2019 6:11 PM
To: user@ignite.apache.org
Subject: RE: Running Ignite Cluster using Docker containers

I am getting different issue now. I configured my Ignite Server to use Unicast 
: 


              
            
                             
                 
                                   
                                  
                                  
                                        
                                          localhost:47500..47504 
                                       
                                        
                                  
                
          
        
        

When I start my first Docker Container it looks good and I am able to connect 
to the Ignite Server using my client code. 

 docker run --rm --name myignite1 -p 47500:47500 -p 47501:47501 -p 10800:10800 
-it myignite/ignite:1.0 


However when I am trying to bring up another server Node : 

 docker run --rm --name myignite2 -p 47502:47502 -p 47503:47503 -p 10801:10800 
-it myignite/ignite:1.0 


it's not forming a Cluster with the previously started Node. 

If I run my Ignite Docker image like that : 

 docker run --rm --net=host --name myignite2 -p 47502:47502 -p 47503:47503 -p 
10801:10800 -it myignite/ignite:1.0 

 then Cluster is created and I can see it if I open "ignitevisor" console. 
However Client application is unable to join the Cluster then 
-- 
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Cache replication with two sites or clusters

[Alexandr Shapkin]

Hello,

If I get it right, you are talking about data replication feature.

Ignite does not support it out of the box, but you can take a look at GridGain 
edition [1]

[1] - 
https://www.gridgain.com/products/software/enterprise-edition/data-center-replication

From: raja24
Sent: Monday, August 5, 2019 7:54 PM
To: user@ignite.apache.org
Subject: Cache replication with two sites or clusters

Hi,

Apache Ignite will support for cache replication between two clusters or
sites?


Thanks,
Raja




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: [IgniteKernal%ServerNode] Exception during start processors, nodewill be stopped and close connections java.lang.NullPointerException:null......cache.GridCacheUtils.affinityNode

[Alexandr Shapkin]

Hello, 

So, you have fixed IGNITE-10451 for 2.7.5 and now you are good with that? 

I see that IGNITE-10451 is included into next Apache Ignite release. 
Not necessary 2.8, since community may decide to release some minor update, 
just like 2.7.5.
Please, follow @dev mailing list for latest updates.

2.8 requires pre-release stabilization process and may contain some bugs.
I would not recommend to use this version in your production env. 

In any case, could you create a simple reproducer, please?
We can fire a Jira ticket and community will check it prior to 2.8 release.

From: siva
Sent: Thursday, August 1, 2019 3:56 PM
To: user@ignite.apache.org
Subject: Re: [IgniteKernal%ServerNode] Exception during start processors, 
nodewill be stopped and close connections 
java.lang.NullPointerException:null..cache.GridCacheUtils.affinityNode

Hi,
Currently I am using 2.7.5 version, due to this issue
https://issues.apache.org/jira/browse/IGNITE-10451 whenever restarting
server we were loading into ignite from database.
I fix this issue, I am testing same scenario with 2.8 nightly build version.
but while restarting server I am facing null NullPointerException
http://apache-ignite-users.70518.x6.nabble.com/file/t1379/ignite.log

Any suggestion how to fix this null pointer exception?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: C++ Thin Client can't consume complex objects published by a Javathick client - IgniteError: Unsupported binary protocol flag

[Alexandr Shapkin]

Hello,

I assume that the main difference is how thin and thick clients handles class 
names metadata.

For thick client you definitely need to register such information manually.

I made your solution work by addition this configuration to the Java part (no 
changes in c++ consumer):
BinaryConfiguration bc = new BinaryConfiguration()
   .setCompactFooter(false)
   .setIdMapper(new BinaryBasicIdMapper(true))
   .setNameMapper(new BinaryBasicNameMapper(true))
   .setClassNames(Arrays.asList("yourPackage.ComplexValue"));
cfg.setBinaryConfiguration(bc);

This is a java counterpart of xml config at the bottom “Spring Configuration” 
section [1]

Also, I would suggest you to implement Binarylizable interface for Java part as 
well.
Just to be sure, that fields ordering and naming stays the same during 
serialization for both sides.

Please, let me know if it helps.

[1] - 
https://apacheignite-cpp.readme.io/docs/cross-platform-interoperability#section-defining-cross-platform-type


From: samer.mas...@murex.com
Sent: Tuesday, July 30, 2019 5:53 PM
To: user@ignite.apache.org
Subject: C++ Thin Client can't consume complex objects published by a Javathick 
client - IgniteError: Unsupported binary protocol flag

Hello everyone,

In short, I am producing complex data from a Java Thick Client, and consume
it from a C++ Thin Client.
In the C++ consumer, after calling cache.GetAll(keys, results); I am getting
this error: Unsupported binary protocol flag:
IGNITE_BINARY_FLAG_COMPACT_FOOTER: 32

However, it works fine when I switch the Java producer to a Thin Client, and
I found no problems consuming the same data from Java Thin and Thick
Clients.
It looks like a serialization incompatibility issue between the Java Thick
Clients and the C++ Thin Clients.
So, I tried these scenarios:

Java thick -> C++ thin = Error
Java thin -> C++ thin = Works
Java thick -> Java thin = Works
Java thick -> Java  thin = Works

I attached the code that reproduces the error:
1. Producer.java starts an ignite node and pushes 3 ComplexValues to Ignite
and successfully consumes them.
2. Consumer.cpp starts an ignite client node and consumes. (Fails)

Publisher.java
  
Consumer.cpp
  

Please ask for any additional information if needed.
Thanks in advance.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: [IgniteKernal%ServerNode] Exception during start processors, nodewill be stopped and close connections java.lang.NullPointerException:null......cache.GridCacheUtils.affinityNode

[Alexandr Shapkin]

Hi,

As far as I can see, [1] is included into 2.8 nightly build and most likely 
will be included into next minor release.

Please, clarify, following questions:
1) What is the point of using 2.8 nightly build, since it could be unstable for 
now?
2) What was the original issue you’ve been trying to solve for now?

[1] and logs from your initial post looks quite different. So I’m wondering why 
do you think they are the same?

Is the original issue (Null Pointer exception) still reproducible for latest 
stable release (2.7.5)?

[1] - https://issues.apache.org/jira/browse/IGNITE-10451

From: siva
Sent: Wednesday, July 31, 2019 2:44 PM
To: user@ignite.apache.org
Subject: Re: [IgniteKernal%ServerNode] Exception during start processors, 
nodewill be stopped and close connections 
java.lang.NullPointerException:null..cache.GridCacheUtils.affinityNode

Hi,
could you provide some suggestion on above issue.
1.whether this issue is fixed or still need to wait for next release?
2.Or Any other alternative solution for the above issue?

Thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Partition loss event

[Alexandr Shapkin]

Hi,

You need to configure includeEventTypes configuration property before starting 
the grid [1]

Xml:
 
 

 
 


Code:
cfg.setIncludeEventTypes(EventType.EVT_CACHE_REBALANCE_PART_DATA_LOST);

[1] - https://apacheignite.readme.io/docs/events#section-configuration


From: balazspeterfi
Sent: Wednesday, July 31, 2019 4:20 PM
To: user@ignite.apache.org
Subject: Partition loss event

Hi All,

I'm using Ignite 2.7.5 and am trying to add some logic to my service that
would trigger a full data reload in case there was a partition loss event. I
followed the docs and added a listener on the
*EVT_CACHE_REBALANCE_PART_DATA_LOST* event, but my code never gets called. I
was testing it by setting the backups to 0 and cache mode to *PARTITIONED*.
After killing a node I can see errors from Ignite when accessing one of the
keys on the lost partition but for some reason the event listener doesn't
get executed.
This is the error I receive on accessing lost data:

Does anyone have any idea what I can possibly do wrong?

Thanks,
Balazs



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Running Ignite Cluster using Docker containers

[Alexandr Shapkin]

Hello,

With default configuration Ignite uses MulticastIpFinder that will try to scan 
for available Ignite instances within your network automatically.
The default required ports are: 47500 and 47100 for discovery and 
communications and localhost(127.0.0.1) is the default alias for docker machine.

I would suggest you to set port forwarding for 47100 as well.
With that default configuration you can connect from you local machine to 
Docker at 127.0.0.1:47500 for thick and 127.0.0.1:10800 for thin clients.

You can change default configuration with CONFIG_URL parameter [1] that could 
point to file:///my-config.xml as well. [2]

You can also try to replace the whole “config” folder with your own 
configuration files:
docker run -v /myconfigFolder:/opt/ignite/apache-ignite/config ….

[1] - https://apacheignite.readme.io/docs/docker-deployment
[2] - 
https://stackoverflow.com/questions/54714911/run-ignite-docker-with-custom-config-file

From: vitalys
Sent: Wednesday, July 31, 2019 5:12 PM
To: user@ignite.apache.org
Subject: Running Ignite Cluster using Docker containers

Hi,

I am trying to run Ignite Cluster using Docker containers. I have Docker for
Desktop running on my Windows box and I downloaded Apache Ignite 2.7.0 image
from Docker Hub. I am able to start Ignite server using docker command :

docker run --rm --name myignite -p 47500:47500 -p 47501:47501 -p
10800:10800 -it apacheignite/ignite

I am trying to connect to the Server Node from my client application and
here I hit a snag. How do I connect to the server? I opened an image and
went to the "config" folder and I found "default-config.xml" there with no
"IpFinder" section. What ports am I suppose to connect to from the client?
What hostname do I need to specify on the client side, "localhost"?  

I looked for the documentation in Apache Ignite tutorial but there are very
few things there designate to this topic. Is there any example or some kind
of tutorial on how to run Ignite Cluster using Docker containers?

Thank you in advance.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Error compiling query: entire LINQ expression shouldbespecifiedwithin lambda passed to Compile method. Part of the query can'tbeoutsidethe Compile method call.

[Alexandr Shapkin]

Hello Siva,

DateTime is a struct and thus should work the expected way.

Please, make sure, that you re-created your cache after some development 
modifications.

I checked this with the following example from your sources:

public class Employee
{
…
[QuerySqlField]
public DateTime? DateTime { get; set; }
}


Func>> issueqry =
CompiledQuery.Compile((DateTime dt) => queryable.Where(emp 
=> emp.Value.DateTime < dt));

   foreach (var entry in issueqry(DateTime.UtcNow))
Console.WriteLine(">>>" + entry.Value.Name);



From: siva
Sent: Tuesday, July 30, 2019 10:43 AM
To: user@ignite.apache.org
Subject: Re: Error compiling query: entire LINQ expression 
shouldbespecifiedwithin lambda passed to Compile method. Part of the query 
can'tbeoutsidethe Compile method call.


Hi, 

when i tried with DateTime type using where clause, then also getting the
following 
exception 

*Method not supported: System.DateTime.(Int32 CompareTo(System.DateTime))
*


Please suggest if any other way to query with DateTime type.

Thanks 
siva 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Ignite .NET 2.7.5 - IEvents.RemoteQuery - Failed to unmarshal eventsquery response

[Alexandr Shapkin]

Hello Simon, 

Can you, please, share a full log? There should be an internal Java exception 
with that message.

It would be perfect if you can share a little reproducer as well.


From: simonatelier
Sent: Thursday, July 25, 2019 3:20 PM
To: user@ignite.apache.org
Subject: Ignite .NET 2.7.5 - IEvents.RemoteQuery - Failed to unmarshal 
eventsquery response

Hi there

We have a Client node-Server node setup where the Client node is running a
RemoteQuery on the cluster for CacheEvents.

When the IEventFilter runs for the RemoteQuery it returns true
but once that happens, the following error is raised in the Ignite logs: 

Failed to unmarshal events query response: GridEventStorageMessage [ex=null,
clsLdrId=null, depMode=null, filterClsName=null, userVer=null,
ldrParties=null]

We are able to query the cache objects normally and they are returned fine,
and the above isn't intuitively giving a pointer to what we need to resolve
to fix the problem.

Any help appreciated.

Simon



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Threadpools and .WithExecute() for C# clients

[Alexandr Shapkin]

Hi,

Can you share a more detailed use case, please? 

Right now it’s not clear why do you need a messaging fabric.
If you are interesting in a progress tracking, then you could try a CacheAPI or 
QueryContinious, for example.

What are the sources of inbound requests? Is it a client requests?

What is your cluster config? How many nodes do you have for your distributed 
computations?

From: Raymond Wilson
Sent: Wednesday, July 17, 2019 1:49 PM
To: user
Subject: Re: Threadpools and .WithExecute() for C# clients

Hi Alexandr,

To summarise from the original thread, say I have server A that accepts 
requests. It contacts server B in order to help processing those requests. 
Server B sends in-progress results to server A using the Ignite messaging 
fabric. If the thread pool in server A is saturated with inbound requests, then 
there are no available threads to service the messaging fabric traffic from 
server B to server A resulting in a deadlock condition.

In the original discussion it was suggested creating a custom thread pool to 
handle the Server B to Server A traffic would resolve it.

Thanks,
Raymond.

On Wed, Jul 17, 2019 at 9:48 PM Alexandr Shapkin  wrote:
Hi, Raymond!
 
As far as I can see, there are no plans for porting custom executors 
configuration in .NET client right now [1].
 
Please, remind, why do you need a separate pool instead of a default PublicPool?
 
[1] - https://issues.apache.org/jira/browse/IGNITE-6566
 
 
 
From: Raymond Wilson
Sent: Wednesday, July 17, 2019 10:58 AM
To: user
Subject: Threadpools and .WithExecute() for C# clients
 
Some time ago I ran into and issue with thread pool exhaustion and deadlocking 
in AI 2.2.
 
This is the original thread: 
http://apache-ignite-users.70518.x6.nabble.com/Possible-dead-lock-when-number-of-jobs-exceeds-thread-pool-td17262.html
  
 
At the time .WithExecutor() was not implemented in the C# client so there was 
little option but to expand the size of the public thread pool sufficiently to 
prevent the deadlocking.
 
We have been revisiting this issue and see that .WithExecutor() is not 
supported in the AI 2.7.5 client.
 
Can this be supported in the C# client, or is there a workaround in the .Net 
environment? that does not require this capability?
 
Thanks,
Raymond.
 
 

RE: Threadpools and .WithExecute() for C# clients

[Alexandr Shapkin]

Hi, Raymond!

As far as I can see, there are no plans for porting custom executors 
configuration in .NET client right now [1].

Please, remind, why do you need a separate pool instead of a default PublicPool?

[1] - https://issues.apache.org/jira/browse/IGNITE-6566



From: Raymond Wilson
Sent: Wednesday, July 17, 2019 10:58 AM
To: user
Subject: Threadpools and .WithExecute() for C# clients

Some time ago I ran into and issue with thread pool exhaustion and deadlocking 
in AI 2.2.

This is the original thread: 
http://apache-ignite-users.70518.x6.nabble.com/Possible-dead-lock-when-number-of-jobs-exceeds-thread-pool-td17262.html
  

At the time .WithExecutor() was not implemented in the C# client so there was 
little option but to expand the size of the public thread pool sufficiently to 
prevent the deadlocking.

We have been revisiting this issue and see that .WithExecutor() is not 
supported in the AI 2.7.5 client.

Can this be supported in the C# client, or is there a workaround in the .Net 
environment? that does not require this capability?

Thanks,
Raymond.

RE: Error compiling query: entire LINQ expression should bespecifiedwithin lambda passed to Compile method. Part of the query can't beoutsidethe Compile method call.

[Alexandr Shapkin]

Hi!

As I understand correctly, right now CompiledQuery won’t work if there is a 
non-primitive value inside the WHERE clause.

This seems like a bug for me, because it’s pretty common to treat String class 
as a primitive one.
Right now you can rewrite your query to “x.Contains(y)” or “x.StartsWith(y)”, 
but it’s not the same as Equals.
 

From: siva
Sent: Friday, July 12, 2019 9:36 PM
To: user@ignite.apache.org
Subject: RE: Error compiling query: entire LINQ expression should 
bespecifiedwithin lambda passed to Compile method. Part of the query can't 
beoutsidethe Compile method call.

Hi ,


Thanks for reply 


i have attached sample code  git hub link.please verify it .






Thanks 
siva https://github.com/cvakarna/ApacheIgnite/blob/master/Program.cs
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Error compiling query: entire LINQ expression should be specifiedwithin lambda passed to Compile method. Part of the query can't be outsidethe Compile method call.

[Alexandr Shapkin]

Hi, Siva!

Can you share a simple example of that?


From: siva
Sent: Friday, July 12, 2019 3:43 PM
To: user@ignite.apache.org
Subject: Error compiling query: entire LINQ expression should be 
specifiedwithin lambda passed to Compile method. Part of the query can't be 
outsidethe Compile method call.

Hi,

I am trying to use Linq compiled queries on top of caches.

Getting  excepted  result  when i  pass int values to the complie query
arguments.

when when i pass string and  and using in where cluse getting following
exceotion 
*
Error compiling query: entire LINQ expression should be specified within
lambda passed to Compile method. Part of the query can't be outside the
Compile method call.*





Can you tell where i am missing



Thanks
siva










--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: How to increase Heap Space for Ignite Server in .Net Core?

[Alexandr Shapkin]

Hi, 

You can pass JVM arguments only to official standalone executable that actually 
takes care of the CLI arguments you provided. 
More information here 
https://apacheignite-net.readme.io/docs/standalone-nodes#section-configuring-standalone-node-via-command-line
  
For your IgniteApp.exe you have several options: 

1) App.config 

  http://ignite.apache.org/schema/dotnet/IgniteConfigurationSection;>
    
      -Xms8g
      -Xmx30g
      ... 
    

2) Code 

var config = new IgniteConfiguration 
{ 
    JvmOptions = new List {"-XX:+UseG1GC", "-XX:+DisableExplicitGC", 
"-Xms8g", 
    ... 
}


From: siva
Sent: Thursday, July 4, 2019 8:18 AM
To: user@ignite.apache.org
Subject: How to increase Heap Space for Ignite Server in .Net Core?

Hi,
I am using Apache Ignite 2.7.5 And .net core v2.2.103.

While loading data through .net core client heap space getting increase.so
for that i am trying to set more heap space.

I have tried to set Heap space from command line with running exe by using
below command.

C:\igniteapp\IgniteApp\bin\Release\netcoreapp2.2\win10-x64\publish>IgniteApp.exe
-server -Xms10g -Xmx15g -XX:+AlwaysPreTouch -XX:+UseG1GC
-XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -J-Xms8g -J-Xmx30g

but it is not applying to application.

[02:45:33] Topology snapshot [ver=1, locNode=115679d9, servers=1, clients=0,
state=INACTIVE, CPUs=16, offheap=11.0GB, heap=12.0GB]

Please Anyone help me.









--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Sql Query Based On DateTime in .Net

[Alexandr Shapkin]

Hi, 

I would suggest you to move your business logic out of queryable fields.
Put [QuerySqlField] to plain _JoinDate and rewrite your sql to "where
_JoinDate = ..."

This should work as your *cache.Get() result* example.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite HashSet clearing

[Alexandr Shapkin]

Hi!

Ignite iterates over the whole keys collection, then performs a remove(keys)
method. So if there are a lot of items and a distributed cluster, then this
would take a long time.

You can take a look into into regular IgniteCache, which has a destroy()
method. 
And use put/get for insert/check operations.






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Trouble with continuous queries

[Alexandr Shapkin]

Wait a sec, I have already posted an example, but for some reasons it was not
send properly.

Here are we go:


class Program
{
private static IContinuousQueryHandle _continuousQuery;
const string CacheName = "sample";
const string TableName = "entity";

static void Main(string[] args)
{
var ignite = Ignition.Start();

var cache = ignite.GetOrCreateCache(new
CacheConfiguration(CacheName) { SqlSchema = "PUBLIC" });

var createTalbeSql =
$"create table if not exists {TableName} (id int, field varchar,
primary key (Id)) " +
$"with \"key_type=int, value_type={typeof(MyEnity).FullName}\"";

cache.Query(new SqlFieldsQuery(createTalbeSql)).GetAll();

ICache entityCache = ignite.GetCache($"SQL_PUBLIC_{TableName.ToUpper()}");

StartContinuousQuery(entityCache);


var entity0 = new MyEnity { Id = 2, Field = "NEW" };
entityCache.Put(entity0.Id, entity0);

var entity = entityCache.Get(entity0.Id);
Console.WriteLine(entity);

var sql = $"update {TableName} set field = 'updated' where Id =
{entity.Id}";
var res = entityCache.Query(new SqlFieldsQuery(sql)).GetAll();
Console.WriteLine(res.First()[0]);

sql = $"insert into {TableName} (_key, field) VALUES(7, 'new
value')";
res = entityCache.Query(new SqlFieldsQuery(sql)).GetAll();
Console.WriteLine(res.First()[0]);

Console.ReadLine();

_continuousQuery.Dispose();
Ignition.Stop(null, true);
}

public static void StartContinuousQuery(ICache cache)
{
var query = new ContinuousQuery(new
ClrCacheSyncEventListener());
_continuousQuery = cache.QueryContinuous(query);
}

public class ClrCacheSyncEventListener : ICacheEntryEventListener
{
public void OnEvent(IEnumerable>
evts)
{
foreach (ICacheEntryEvent cacheEntryEvent in evts)
{
Console.WriteLine($"Action happened
{cacheEntryEvent.EventType}");
}
}
}


public class MyEnity
{
public MyEnity()
{
}

public int Id { get; set; }
[QuerySqlField]
public string Field { get; set; }

public override string ToString()
{
return $"Id = {this.Id}; TextField = {this.Field};";
}
}
}



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Trouble with continuous queries

[Alexandr Shapkin]

Hi, yes, it should work that way.

I was able to caught all events event with the raw SQL using this DBeaver
https://apacheignite-sql.readme.io/docs/sql-tooling





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Trouble with continuous queries

[Alexandr Shapkin]

Hi,

You can just save an instance of continuous query somewhere and dispose it
when required


  var listener = new SomeListener();
  var someFilter= new
CacheEntryEventFilter(_ignite.GetCluster().GetLocalNode(), Const.CacheName);
  var query = new ContinuousQuery(listener, someFilter);

   //save the reference in a private field
  _continuousQuery = cache.QueryContinuous(query);


Regardless of a cache value. 
As I understand correctly you have a mixed platform solution (Java + .NET)
This may lead to additional marshalling configuration.

I will try the posted solution a bit later and reply
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Replace or Put after PutAsync causes Ignite to hang

[Alexandr Shapkin]

Hi,

I took a look at the reproducer and it works just fine with different Ignite
and .net versions.

Is there just a single Ignite server with the default config?









--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/