unsubscribe
unsubscribe This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite Thin Client Continuous Query
Hi Val, It's a very simple, and I would say a very common use case. We want to send a filter to the grid, receive a snapshot, and then receive a continuous stream of updates that match that filter. Consider a trading window, showing a grid of stocks. I want to subscribe to the continuously changing market prices for a specific list of stocks, and maybe I want to subscribe to trade events for another specific set. The universe of stocks is huge and not all users will want to see the same stocks, so server side filtering is key. We are under time pressure, so for now we will probably end up building our own lightweight framework based on sockets and json for serialization between the java server side and the .net user app side. On the server side we will have to build a proxy service that will be responsible for managing subscriptions from the user apps, and then making those corresponding subscriptions onto the grid, using CQ. When it receives the results of the CQ it will need to serialize and push events back to the user apps. The high level use case is that we have a java ignite cluster which implements our trading platform deployed into a data centre. And we have a C# .NET desktop app which currently hosts an ignite thick client. This user app needs to connect to clusters which might be in the metro area, or even in other countries. We have found for metro, it's okay, but not great (despite a huge amount of time tuning comms parameters), and for remote cities, its unusable. If we were .net to .net we could (for example) do this easily with WCF. The user app just needs to - send RPC style commands (which we currently do using ignite service grid) (eg. start / stop trading strategy) - get cache snapshots based on some filter (eg. show me all the orders on Microsoft for yesterday) - subscribe to cache updates based on some filter ( eg. show me all the orders on Microsoft for today so far, placed by trading strategy X, and stream new orders or changes to existing orders as they occur) Thanks, Gordon. -Original Message- From: vkulichenko Sent: Tuesday, September 11, 2018 10:47 AM To: user@ignite.apache.org Subject: RE: Ignite Thin Client Continuous Query Gordon, Yes, generally we do recommend using thin client for such applications. However, it doesn't mean that you can't us client node in case it's really needed, although it might require some additional tuning. Would you mind telling if you have any other technology in mind? I highly doubt that you will find anything that can provide functionality similar to CQ in Ignite, especially with the same delivery guarantees, while being based on a lightweight client. I believe you either will not succeed in finding such an alternative, or your use case does not require continuous queries in the first place. Can you give some more details on what you're trying to achieve? I might be able to suggest some other approach then. -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/ This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite Thin Client Continuous Query
In my humble opinion there is a huge value here. We have these fantastic APIs between our cluster nodes, why should we have to go and use different APIs, and different serialization techniques in our end user apps? It’s totally acceptable that the reliability, and guaranteed delivery aspects are relaxed for the user app. The key is to use the same (or useful subset of) the cache interfaces. From: Valentin Kulichenko Sent: Wednesday, September 12, 2018 6:33 AM To: user@ignite.apache.org Cc: isap...@apache.org Subject: Re: Ignite Thin Client Continuous Query Igor, I just think that we're dealing with a trade off here, and that if we implement CQ for thin client, we will either end with a client that is not "thin" anymore, or semantics and guarantees of CQ would change so drastically that it would be a completely different feature. Either way, it's a big question weather there is a value in doing this. I'm open to discussion though. If you have any particular suggestions, let's discuss them on dev list. -Val On Tue, Sep 11, 2018 at 5:26 AM Igor Sapego mailto:isap...@apache.org>> wrote: Guys, Personally, I do not see any problems, why we can not implement Continuous Queries for thin clients. This will require a decent amount of work, and will not give such strong guaranties as thick clients give (for example, on server crash thin client will get an exception and will need to re-register listener once again), but to me it seems totally implementable. Val, Why do you think that such features are unlikely to appear in thin clients? Best Regards, Igor On Tue, Sep 11, 2018 at 3:07 PM Alexey Kuznetsov mailto:akuznet...@apache.org>> wrote: Gordon, How about to start several client nodes "near" to cluster and use them as "proxies" for your desktop GUI apps? You may write some code that will push data from client node to you GUI app. This will require some coding of course. -- Alexey Kuznetsov This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite Thin Client Continuous Query
Thanks Alexy, yes we have considered this approach. But I would normally consider this type of architecture an optimization, not a base requirement. It’s rather heavy and to me only makes sense when we have a large number of end users and we want to minimize bandwidth to the remote locations. From: Alexey Kuznetsov Sent: Tuesday, September 11, 2018 10:07 PM To: user@ignite.apache.org Subject: Re: Ignite Thin Client Continuous Query Gordon, How about to start several client nodes "near" to cluster and use them as "proxies" for your desktop GUI apps? You may write some code that will push data from client node to you GUI app. This will require some coding of course. -- Alexey Kuznetsov This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite Thin Client Continuous Query
Thanks Val. We are currently using a client node in our desktop gui, but it performs very poorly when the latency to our server nodes is high. We also have other problems, such as when new client nodes join, the whole cluster will pause, which is unacceptable for an end user application. I raised a question about this a few weeks ago, and we were advised that client node is not intended for use in end user applications. Any sort of financial desktop application needs streaming data and event based updates. So it seems like ignite is of no use for this type of application. And now I wonder if there is much value for us to keep using ignite on the server side of our application, since we need totally different technology to enable streaming data and event based notifications on our client. -Original Message- From: vkulichenko Sent: Tuesday, September 11, 2018 7:43 AM To: user@ignite.apache.org Subject: Re: Ignite Thin Client Continuous Query Gordon, Ignite thin client uses request-response model, which is not really suitable for functionality like this. I would never say never, but I think it's very unlikely that thin client would get any feature that imply pushing updates from server to client (this includes near caches, any type of listeners including continuous queries, etc.). If you have such a requirement, you should use client node instead. -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/ This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Ignite Thin Client Continuous Query
Hi, Is there any plan to support continuous query in the Ignite Thin client. This would be very useful for us. Without it, we probably have to use some other technology for serialization and streaming updates to the desktop. Thanks, Gordon This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Cache updates to nodes in client mode
Great, thanks Denis. I had no idea the thin client existed. This should be exactly what we need. From: Denis Mekhanikov Sent: Friday, August 3, 2018 7:31 PM To: user@ignite.apache.org Subject: Re: Cache updates to nodes in client mode Gordon, Client nodes are not developed to be used on user side of the application. Thin client should be used instead: https://apacheignite-net.readme.io/docs/thin-client Also ODBC client may be an option here: https://apacheignite-sql.readme.io/docs/odbc-driver Denis пт, 3 авг. 2018 г. в 10:13, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Thanks Denis. Yes, I know about this setting. Actually, the problem really is that when a client is trying to 1. Reconnect after a problem 2. Or even just connect for the first time on start up Then the updates to other nodes will pause while this new client is joining the cluster. Is there any way to prevent that? Imagine if we have 50 users, and each time one of them opens their GUI, the updates to all GUIs will pause until the connection is complete. Thanks, Gordon. From: Denis Mekhanikov mailto:dmekhani...@gmail.com>> Sent: Wednesday, August 1, 2018 10:28 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Cache updates to nodes in client mode Gordon, There is already a mechanism in discovery protocol, that makes client nodes to be disconnected from cluster in case, when they don't respond for a long time. It can be configured using IgniteConfiguration#clientFailureDetectionTimeout<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientFailureDetectionTimeout-long->. More information on it: https://apacheignite.readme.io/docs/tcpip-discovery#section-failure-detection-timeout But currently this functionality is broken: IGNITE-5103<https://issues.apache.org/jira/browse/IGNITE-5103> It is going to be fixed in Ignite 2.7 Denis ср, 1 авг. 2018 г. в 2:20, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Hi Denis, Each client runs on a separate user desktop. The client is a C#.NET application which loads ignite in client mode and joins our single java server node running on a remote server. So each client has it’s own JVM running inside ignite .NET. We only have one server node, but many clients. The lock up could be many reasons. 1. User machine has some problem in another application which is starving the client node from resources 2. Client node is running in a debugger and the developer as paused it on a break point 3. There is some bug in our user code which has caused the GUI to block and stop processing events 4. There is a network problem to a single client node Basically we are looking for some type of configuration that is very forgiving when there are problems delivering messages to any of the nodes running in client mode within the desktop GUIs. We are on ignite 2.2.0 Thanks, Gordon. From: Denis Mekhanikov mailto:dmekhani...@gmail.com>> Sent: Tuesday, July 31, 2018 10:39 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Cache updates to nodes in client mode Gordon, Does every GUI have its own client Ignite node, or they are shared between applications? Are they running in the same VM, or in separate ones? What do you mean by locking up? Is it related to some Ignite problem? Denis вт, 31 июл. 2018 г. в 9:23, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Hi, We have a single java server instance, and we run multiple .NET nodes (GUIs) in client mode. It seems that when one GUI is locked up (for whatever reason) the other GUIs (client nodes) will stop receiving updates. I.e. it seems that cache sync to client nodes is done on a round robin basis and updates must be acknowledged at each client before the next client is notified. This is very bad for us, as we don’t want an issue on one GUI to affect all other GUIs. Is there someway we can relax the synchronization of updates? Note, GUIs typically have many continuous subscriptions on the various cache entities. This is what I mean by “synchronization”. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252 This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine
RE: Cache updates to nodes in client mode
Thanks Denis. Yes, I know about this setting. Actually, the problem really is that when a client is trying to 1. Reconnect after a problem 2. Or even just connect for the first time on start up Then the updates to other nodes will pause while this new client is joining the cluster. Is there any way to prevent that? Imagine if we have 50 users, and each time one of them opens their GUI, the updates to all GUIs will pause until the connection is complete. Thanks, Gordon. From: Denis Mekhanikov Sent: Wednesday, August 1, 2018 10:28 PM To: user@ignite.apache.org Subject: Re: Cache updates to nodes in client mode Gordon, There is already a mechanism in discovery protocol, that makes client nodes to be disconnected from cluster in case, when they don't respond for a long time. It can be configured using IgniteConfiguration#clientFailureDetectionTimeout<https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientFailureDetectionTimeout-long->. More information on it: https://apacheignite.readme.io/docs/tcpip-discovery#section-failure-detection-timeout But currently this functionality is broken: IGNITE-5103<https://issues.apache.org/jira/browse/IGNITE-5103> It is going to be fixed in Ignite 2.7 Denis ср, 1 авг. 2018 г. в 2:20, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Hi Denis, Each client runs on a separate user desktop. The client is a C#.NET application which loads ignite in client mode and joins our single java server node running on a remote server. So each client has it’s own JVM running inside ignite .NET. We only have one server node, but many clients. The lock up could be many reasons. 1. User machine has some problem in another application which is starving the client node from resources 2. Client node is running in a debugger and the developer as paused it on a break point 3. There is some bug in our user code which has caused the GUI to block and stop processing events 4. There is a network problem to a single client node Basically we are looking for some type of configuration that is very forgiving when there are problems delivering messages to any of the nodes running in client mode within the desktop GUIs. We are on ignite 2.2.0 Thanks, Gordon. From: Denis Mekhanikov mailto:dmekhani...@gmail.com>> Sent: Tuesday, July 31, 2018 10:39 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Cache updates to nodes in client mode Gordon, Does every GUI have its own client Ignite node, or they are shared between applications? Are they running in the same VM, or in separate ones? What do you mean by locking up? Is it related to some Ignite problem? Denis вт, 31 июл. 2018 г. в 9:23, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Hi, We have a single java server instance, and we run multiple .NET nodes (GUIs) in client mode. It seems that when one GUI is locked up (for whatever reason) the other GUIs (client nodes) will stop receiving updates. I.e. it seems that cache sync to client nodes is done on a round robin basis and updates must be acknowledged at each client before the next client is notified. This is very bad for us, as we don’t want an issue on one GUI to affect all other GUIs. Is there someway we can relax the synchronization of updates? Note, GUIs typically have many continuous subscriptions on the various cache entities. This is what I mean by “synchronization”. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252 This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252 This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Cache updates to nodes in client mode
Hi Denis, Each client runs on a separate user desktop. The client is a C#.NET application which loads ignite in client mode and joins our single java server node running on a remote server. So each client has it’s own JVM running inside ignite .NET. We only have one server node, but many clients. The lock up could be many reasons. 1. User machine has some problem in another application which is starving the client node from resources 2. Client node is running in a debugger and the developer as paused it on a break point 3. There is some bug in our user code which has caused the GUI to block and stop processing events 4. There is a network problem to a single client node Basically we are looking for some type of configuration that is very forgiving when there are problems delivering messages to any of the nodes running in client mode within the desktop GUIs. We are on ignite 2.2.0 Thanks, Gordon. From: Denis Mekhanikov Sent: Tuesday, July 31, 2018 10:39 PM To: user@ignite.apache.org Subject: Re: Cache updates to nodes in client mode Gordon, Does every GUI have its own client Ignite node, or they are shared between applications? Are they running in the same VM, or in separate ones? What do you mean by locking up? Is it related to some Ignite problem? Denis вт, 31 июл. 2018 г. в 9:23, Gordon Reid (Nine Mile) mailto:gordon.r...@ninemilefinancial.com>>: Hi, We have a single java server instance, and we run multiple .NET nodes (GUIs) in client mode. It seems that when one GUI is locked up (for whatever reason) the other GUIs (client nodes) will stop receiving updates. I.e. it seems that cache sync to client nodes is done on a round robin basis and updates must be acknowledged at each client before the next client is notified. This is very bad for us, as we don’t want an issue on one GUI to affect all other GUIs. Is there someway we can relax the synchronization of updates? Note, GUIs typically have many continuous subscriptions on the various cache entities. This is what I mean by “synchronization”. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252 This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Cache updates to nodes in client mode
Hi, We have a single java server instance, and we run multiple .NET nodes (GUIs) in client mode. It seems that when one GUI is locked up (for whatever reason) the other GUIs (client nodes) will stop receiving updates. I.e. it seems that cache sync to client nodes is done on a round robin basis and updates must be acknowledged at each client before the next client is notified. This is very bad for us, as we don't want an issue on one GUI to affect all other GUIs. Is there someway we can relax the synchronization of updates? Note, GUIs typically have many continuous subscriptions on the various cache entities. This is what I mean by "synchronization". Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Transport compression (not store compression)
Hi Igniters, I see there is a lot of discussion in certain threads about compression. This seems to have diverged into conversations about object versus field compression, and even throwing encryption into the mix. For my use case, I am not interested in compressing the cache stored in memory, I have plenty of memory for my application. What I don't have is a good network. I have a high latency, low bandwidth network between my C# ignite client and my Java ignite server. I only want to compress data when it is sent over the network to remote nodes. It should be stored in the local memory uncompressed. How can we achive this? Can the TcpCommunicationSpi support compression? Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Java to .NET interop
Hi Pavel, Thanks for the quick reply. I have not explicitly tried to only deploy the service classes without the entity classes into the java classpath on the .NET side. Our entity and service classes are packaged into the same jar. I won’t investigate this further, as its just as much of an inconvenience regardless of what is in the jar. But it’s good to know that the .NET client only needs a new JAR when a service class changes. Do you think this is something that might be fixed when ignite supports java 9? It would be great to have no binary dependency on our java server build inside our .net client. Thanks, Gordon. From: Pavel Tupitsyn [mailto:ptupit...@apache.org] Sent: Thursday, October 26, 2017 7:43 PM To: user@ignite.apache.org Subject: Re: Java to .NET interop Hi Gordon, Entity Java classes are not needed in .NET client classpath. Can you give an example when that does not work for you? As for services, unfortunately, there is a limitation on Java side, Service class should be available on all nodes [1] [2]. Thanks, Pavel [1] https://apacheignite.readme.io/docs/service-grid [2] https://issues.apache.org/jira/browse/IGNITE-975 On Thu, Oct 26, 2017 at 9:51 AM, Gordon Reid (Nine Mile) <gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>> wrote: Hi Igniters, We are running Ignite 2.2.0, we have a .NET client (runs in ignite client mode) and Java servers. We call Ignite Services running on the Java side, from the .NET client. We also access our memory grid entities from the .NET client. Currently it seems we need to package our java server jar, publish it to nuget, and then import this on the .NET side. If we don’t have the entity and service classes available on the .NET side’s Java classpath then we cannot communicate from the .NET side to the Java side. We are using the binary marshaller and simple name mapper. Is there any way around this requirement? It is quite a hassle creating this tight binary coupling between our .NET client and Java server. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252 This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Java to .NET interop
Hi Igniters, We are running Ignite 2.2.0, we have a .NET client (runs in ignite client mode) and Java servers. We call Ignite Services running on the Java side, from the .NET client. We also access our memory grid entities from the .NET client. Currently it seems we need to package our java server jar, publish it to nuget, and then import this on the .NET side. If we don't have the entity and service classes available on the .NET side's Java classpath then we cannot communicate from the .NET side to the Java side. We are using the binary marshaller and simple name mapper. Is there any way around this requirement? It is quite a hassle creating this tight binary coupling between our .NET client and Java server. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: IGNITE-4548 does not seem to be completely fixed
Hi Mikhail,
Thanks for looking at this. I will create a reproducer for you.
Thanks,
Gordon.
From: Mikhail Cherkasov [mailto:mcherka...@gridgain.com]
Sent: Tuesday, 4 July 2017 2:14 AM
To: user@ignite.apache.org
Subject: Re: IGNITE-4548 does not seem to be completely fixed
Hi Gordon,
Could you please send you pom project to debug it?
Looks like it was changes on purpose by this ticket:
https://issues.apache.org/jira/browse/IGNITE-4949
but anyway, I want to check it with debugger, I will appreciate if you send
your test case.
Thanks,
Mikhail.
On Mon, Jul 3, 2017 at 10:37 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi Ignite Experts,
I have tested the fix for IGNITE-4548
Say I have enum like this
public enum SecurityType {
All,
Stock,
Future,
Option,
Commodity,
ETF,
Bond,
Unknown,
Equity,
REIT,
StapledSecurity,
CurrencyPair
}
Using this JDBC config
vals.add(new JdbcTypeField(Types.VARCHAR, "securityType", SecurityType.class,
"securityType"));
And some model class with field
@QuerySqlField(index=true)
private SecurityType securityType;
I can read a varchar from the database of “Bond” from the database into the
cache, and the securityType field on the cache record will be
SecurityType.Bond. GREAT.
BUT
When I save the record back to the database the enum ordinal “6” will be saved
into the securityType column. This is not what we want.
1. It makes reading the database difficult. 6 means nothing to someone who
can’t see the code.
2. More importantly, the enum may be refactored over time, and “6” may
represent something else in future
Reading string value from database into the enum field is working fine. But is
there a way I can force ignite to save the string value of the Enum into the
database also?
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
--
Thanks,
Mikhail.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite very very chatty when java server nodes and .net client when using Service Grid
Hi Pavel, Yes that’s correct, in this case we were calling Ignite.GetServices().GetServiceProxy() every second. When the .NET client is remote from the java server, this caused a large amount of traffic. As service descriptors for services that were not even deployed (so basically all the services types that we define), was being sent back every second. The reason we were calling like this, is because we didn’t want to have to manage a stale service proxy in the case of topology change. Thanks, Gordon. From: Pavel Tupitsyn [mailto:ptupit...@apache.org] Sent: Thursday, 6 July 2017 6:42 PM To: user@ignite.apache.org Subject: Re: Ignite very very chatty when java server nodes and .net client when using Service Grid Hi Gordon, I see that you solved the problem, but I'm still curious: what did you have and what did you change to solve this? Did you call Ignite.GetServices().GetServiceProxy() every second, or something else? Thanks, Pavel On Thu, Jul 6, 2017 at 3:07 AM, Gordon Reid (Nine Mile) <gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>> wrote: It looks like the problem here is that the Ignite.GetServices is a relatively expensive call from the client. We have a pinger which tells the server a client is “healthy” every 1 second. Caching the service proxy seems to do the trick. And we can manage topology change and refreshing a stale proxy another way. Thanks. From: Gordon Reid (Nine Mile) [mailto:gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>] Sent: Tuesday, 4 July 2017 3:50 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: RE: Ignite very very chatty when java server nodes and .net client when using Service Grid Some additional log which might be useful. Why is it scanning for services when it’s running in client=true? 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.resource.GridResourceProcessor - Injecting resources: org.apache.ignite.internal.cluster.ClusterGroupAdapter$AttributeFilter@3398bf2d<mailto:org.apache.ignite.internal.cluster.ClusterGroupAdapter$AttributeFilter@3398bf2d> 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter - Executing query [query=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=ServiceDeploymentPredicate [], transform=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed<mailto:prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed>, keepBinary=false, subjId=null, taskHash=0], nodes=[TcpDiscoveryNode [id=386f9b12-4dc3-48b4-a174-95a28387d624, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.21.182], sockAddrs=[/0:0:0:0:0:0:0:1%lo:48900, /127.0.0.1:48900<http://127.0.0.1:48900>, /172.31.21.182:48900<http://172.31.21.182:48900>], discPort=48900, order=1, intOrder=1, lastExchangeTime=1499146293372, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false]]] 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager - Executing distributed query: GridCacheQueryBean [qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=ServiceDeploymentPredicate [], transform=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed<mailto:prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed>, keepBinary=false, subjId=d30c7fa6-7a8e-4fe0-9bae-efb67d014a1e, taskHash=0], rdc=null, trans=null] 2017-07-04 15:33:27,305 [11] DEBUG org.apache.ignite.internal.processors.cache.GridCacheIoManager - Sending cache message [msg=GridCacheQueryRequest [id=4766, cacheName=ignite-sys-cache, type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=ServiceDeploymentPredicate [], rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false, incMeta=false, all=false, keepPortable=false, subjId=d30c7fa6-7a8e-4fe0-9bae-efb67d014a1e, taskHash=0, part=-1, topVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], super=GridCacheMessage [msgId=4767, depInfo=null, err=null, skipPrepare=false, cacheId=-2100569601, cacheId=-2100569601]], nodes=Grid nodes [cnt=1, ClusterNode [id=386f9b12-4dc3-48b4-a174-95a28387d624, order=1, addr=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.21.182], daemon=false]]] 2017-07-04 15:33:27,305 [11] DEBUG org.apache.ignite.internal.processors.cache.GridCacheIoManager - Sent cache message [msg=Grid
RE: Ignite very very chatty when java server nodes and .net client when using Service Grid
It looks like the problem here is that the Ignite.GetServices is a relatively expensive call from the client. We have a pinger which tells the server a client is "healthy" every 1 second. Caching the service proxy seems to do the trick. And we can manage topology change and refreshing a stale proxy another way. Thanks. From: Gordon Reid (Nine Mile) [mailto:gordon.r...@ninemilefinancial.com] Sent: Tuesday, 4 July 2017 3:50 PM To: user@ignite.apache.org Subject: RE: Ignite very very chatty when java server nodes and .net client when using Service Grid Some additional log which might be useful. Why is it scanning for services when it's running in client=true? 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.resource.GridResourceProcessor - Injecting resources: org.apache.ignite.internal.cluster.ClusterGroupAdapter$AttributeFilter@3398bf2d<mailto:org.apache.ignite.internal.cluster.ClusterGroupAdapter$AttributeFilter@3398bf2d> 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter - Executing query [query=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=ServiceDeploymentPredicate [], transform=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed<mailto:prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed>, keepBinary=false, subjId=null, taskHash=0], nodes=[TcpDiscoveryNode [id=386f9b12-4dc3-48b4-a174-95a28387d624, addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.21.182], sockAddrs=[/0:0:0:0:0:0:0:1%lo:48900, /127.0.0.1:48900, /172.31.21.182:48900], discPort=48900, order=1, intOrder=1, lastExchangeTime=1499146293372, loc=false, ver=2.0.0#20170430-sha1:d4eef3c6, isClient=false]]] 2017-07-04 15:33:27,304 [11] DEBUG org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager - Executing distributed query: GridCacheQueryBean [qry=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=ServiceDeploymentPredicate [], transform=null, part=null, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, keepAll=false, incBackups=false, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed<mailto:prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@628833ed>, keepBinary=false, subjId=d30c7fa6-7a8e-4fe0-9bae-efb67d014a1e, taskHash=0], rdc=null, trans=null] 2017-07-04 15:33:27,305 [11] DEBUG org.apache.ignite.internal.processors.cache.GridCacheIoManager - Sending cache message [msg=GridCacheQueryRequest [id=4766, cacheName=ignite-sys-cache, type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=ServiceDeploymentPredicate [], rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false, incMeta=false, all=false, keepPortable=false, subjId=d30c7fa6-7a8e-4fe0-9bae-efb67d014a1e, taskHash=0, part=-1, topVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], super=GridCacheMessage [msgId=4767, depInfo=null, err=null, skipPrepare=false, cacheId=-2100569601, cacheId=-2100569601]], nodes=Grid nodes [cnt=1, ClusterNode [id=386f9b12-4dc3-48b4-a174-95a28387d624, order=1, addr=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.21.182], daemon=false]]] 2017-07-04 15:33:27,305 [11] DEBUG org.apache.ignite.internal.processors.cache.GridCacheIoManager - Sent cache message [msg=GridCacheQueryRequest [id=4766, cacheName=ignite-sys-cache, type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=ServiceDeploymentPredicate [], rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false, incMeta=false, all=false, keepPortable=false, subjId=d30c7fa6-7a8e-4fe0-9bae-efb67d014a1e, taskHash=0, part=-1, topVer=AffinityTopologyVersion [topVer=18, minorTopVer=0], super=GridCacheMessage [msgId=4767, depInfo=null, err=null, skipPrepare=false, cacheId=-2100569601, cacheId=-2100569601]], nodes=Grid nodes [cnt=1, ClusterNode [id=386f9b12-4dc3-48b4-a174-95a28387d624, order=1, addr=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 172.31.21.182], daemon=false]]] 2017-07-04 15:33:27,565 [42] DEBUG org.apache.ignite.internal.processors.cache.GridCacheIoManager - Received cache ordered message [nodeId=386f9b12-4dc3-48b4-a174-95a28387d624, msg=GridCacheQueryResponse [finished=true, reqId=4766, err=null, fields=false, metadata=null]] 2017-07-04 15:33:27,565 [42] DEBUG org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager - Received query response: GridCacheQueryResponse [finished=true, reqId=4766, err=null, fields=false, metadata=null] 2017-0
IGNITE-4548 does not seem to be completely fixed
Hi Ignite Experts,
I have tested the fix for IGNITE-4548
Say I have enum like this
public enum SecurityType {
All,
Stock,
Future,
Option,
Commodity,
ETF,
Bond,
Unknown,
Equity,
REIT,
StapledSecurity,
CurrencyPair
}
Using this JDBC config
vals.add(new JdbcTypeField(Types.VARCHAR, "securityType", SecurityType.class,
"securityType"));
And some model class with field
@QuerySqlField(index=true)
private SecurityType securityType;
I can read a varchar from the database of "Bond" from the database into the
cache, and the securityType field on the cache record will be
SecurityType.Bond. GREAT.
BUT
When I save the record back to the database the enum ordinal "6" will be saved
into the securityType column. This is not what we want.
1. It makes reading the database difficult. 6 means nothing to someone who
can't see the code.
2. More importantly, the enum may be refactored over time, and "6" may
represent something else in future
Reading string value from database into the enum field is working fine. But is
there a way I can force ignite to save the string value of the Enum into the
database also?
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
IGNITE-4548 does not seem to be completely fixed
Hi Ignite Experts,
I have tested the fix for IGNITE-4548
Say I have enum like this
public enum SecurityType {
All,
Stock,
Future,
Option,
Commodity,
ETF,
Bond,
Unknown,
Equity,
REIT,
StapledSecurity,
CurrencyPair
}
Using this JDBC config
vals.add(new JdbcTypeField(Types.VARCHAR, "securityType", SecurityType.class,
"securityType"));
And some model class with field
@QuerySqlField(index=true)
private SecurityType securityType;
I can read a varchar from the database of "Bond" from the database into the
cache, and the securityType field on the cache record will be
SecurityType.Bond. GREAT.
BUT
When I save the record back to the database the enum ordinal "6" will be saved
into the securityType column. This is not what we want.
1. It makes reading the database difficult. 6 means nothing to someone who
can't see the code.
2. More importantly, the enum may be refactored over time, and "6" may
represent something else in future
Reading string value from database into the enum field is working fine. But is
there a way I can force ignite to save the string value of the Enum into the
database also?
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Modelling Question
Hi, I have a general question about data model design. Currently we have java server nodes, and .net client nodes. We use services, messaging and the data grid. The current design is one single ignite cache, which contains all entity types. The key for entity types is an object, but it has only one field, long id Because ignite .net has some limitations when mixed with Java, such as no continuous queries with filter, we do a lot of client side filtering based on the Key class type. In fact we also use cache events, and again we have to filter by key class and then cast before continuing. Furthermore, we experience some issues with java/.net Key equality, such as this https://issues.apache.org/jira/browse/IGNITE-5455 At the moment my feeling is that we should change our model * Have one ignite cache per type (we have about 30 types in our model) * Use long primitives for cache keys instead of objects wrapping the id Our two main drivers are interop between java servers and .net clients, and query performance. Also get by id performance is very important. Surely this is faster with primitive keys than objects as keys? We don't do a lot of joining, although I know we can join caches in ignite queries anyway. Any comments are much appreciated. Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Re: Get by key only working on .net for certain long literal values
Thanks very much for figuring this out Pavel. I will work around it for now.
Get Outlook for Android<https://aka.ms/ghei36>
From: Pavel Tupitsyn <ptupit...@apache.org>
Sent: Friday, June 9, 2017 1:47:19 AM
To: user@ignite.apache.org
Subject: Re: Get by key only working on .net for certain long literal values
Found the bug: https://issues.apache.org/jira/browse/IGNITE-5455
Hash code calculation is inconsistent in Java and .NET.
Which means that cache keys of user-defined types can't be used when both .NET
and Java are involved.
The only workaround is to use basic type as a key (all primitives, string,
Guid, decimal).
This will be fixed shortly in master and released in 2.1, which should come out
this month.
On Thu, Jun 8, 2017 at 5:15 PM, Pavel Tupitsyn
<ptupit...@apache.org<mailto:ptupit...@apache.org>> wrote:
Gordon, I have reproduced the issue.
I will investigate it and get back to you later.
Pavel
On Thu, Jun 8, 2017 at 2:46 PM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Thanks Pavel.
The objects are loaded on the Java side, which is the server node. Reading from
Java client node works fine always. But reading from .net client gives the
problem mentioned.
Yes, BinaryIdMapper and BinaryNameMapper are configured as per the wiki.
Java side
BinaryConfiguration bcfg = new BinaryConfiguration();
bcfg.setCompactFooter(true);
bcfg.setIdMapper(new BinaryBasicIdMapper(true));
bcfg.setNameMapper(new BinaryBasicNameMapper(true));
icfg.setBinaryConfiguration(bcfg);
.NET side
var binaryTypes = new []
{
typeof(SecurityKey),
typeof(Security),
}
cfg.BinaryConfiguration = new BinaryConfiguration(binaryTypes)
{
NameMapper = new BinaryBasicNameMapper { IsSimpleName = true }
};
From: Pavel Tupitsyn [mailto:ptupit...@apache.org<mailto:ptupit...@apache.org>]
Sent: Thursday, 8 June 2017 6:55 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Get by key only working on .net for certain long literal values
I've tried to use your class as a cache key and it seems to work, see attached
program.
* How do you add these objects to cache? Is it from Java or .NET?
* Have you configured BinaryIdMapper on .NET and Java sides properly so that
classes map to each other?
Pavel
On Thu, Jun 8, 2017 at 12:07 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi. Yes, I am on 2.0.
Get Outlook for Android<https://aka.ms/ghei36>
From: Pavel Tupitsyn <ptupit...@apache.org<mailto:ptupit...@apache.org>>
Sent: Thursday, June 8, 2017 12:26:41 AM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Get by key only working on .net for certain long literal values
Hi Gordon,
Seems like all your threads are related:
http://apache-ignite-users.70518.x6.nabble.com/BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13444.html
http://apache-ignite-users.70518.x6.nabble.com/UPDATED-BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13445.html
Let's discuss them here.
First of all, can you confirm that you are on Ignite 2.0?
Pavel
On Wed, Jun 7, 2017 at 11:03 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi There,
This one is very strange.
I have a .NET client, Java client and Java server. All running locally on
windows desktop.
On the server side I have some cache entries, SecurityKey,Security
I have two Security objects, one with id=4 and one with id=953
On the Java client, this works no problem
SystemContext.getCache().get(new SecurityKey(4));
SystemContext.getCache().get(new SecurityKey(953));
On the .NET client
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 4 });
Will succeed
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 953 });
Will FAIL, with KeyNotFoundException
This is the case 100% of the time, even after full restart of the environment.
The only thing that seems to cause the problem is the literal value of the id
Also note that if I iterate over the cache on the .NET side, I can absolutely
see the SecurityKey with id=953, but I will see KeyNotFound when I try to Get
foreach (var e in SystemContext.Cache)
{
if (e.Key is SecurityKey)
{
Log.Info(((SecurityKey)e.Key).Id); // this will print both
4 and 953
}
}
My only guess here is that something funny is going on internally with the
hashing of the
RE: Get by key only working on .net for certain long literal values
Thanks Pavel.
The objects are loaded on the Java side, which is the server node. Reading from
Java client node works fine always. But reading from .net client gives the
problem mentioned.
Yes, BinaryIdMapper and BinaryNameMapper are configured as per the wiki.
Java side
BinaryConfiguration bcfg = new BinaryConfiguration();
bcfg.setCompactFooter(true);
bcfg.setIdMapper(new BinaryBasicIdMapper(true));
bcfg.setNameMapper(new BinaryBasicNameMapper(true));
icfg.setBinaryConfiguration(bcfg);
.NET side
var binaryTypes = new []
{
typeof(SecurityKey),
typeof(Security),
}
cfg.BinaryConfiguration = new BinaryConfiguration(binaryTypes)
{
NameMapper = new BinaryBasicNameMapper { IsSimpleName = true }
};
From: Pavel Tupitsyn [mailto:ptupit...@apache.org]
Sent: Thursday, 8 June 2017 6:55 PM
To: user@ignite.apache.org
Subject: Re: Get by key only working on .net for certain long literal values
I've tried to use your class as a cache key and it seems to work, see attached
program.
* How do you add these objects to cache? Is it from Java or .NET?
* Have you configured BinaryIdMapper on .NET and Java sides properly so that
classes map to each other?
Pavel
On Thu, Jun 8, 2017 at 12:07 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi. Yes, I am on 2.0.
Get Outlook for Android<https://aka.ms/ghei36>
From: Pavel Tupitsyn <ptupit...@apache.org<mailto:ptupit...@apache.org>>
Sent: Thursday, June 8, 2017 12:26:41 AM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Get by key only working on .net for certain long literal values
Hi Gordon,
Seems like all your threads are related:
http://apache-ignite-users.70518.x6.nabble.com/BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13444.html
http://apache-ignite-users.70518.x6.nabble.com/UPDATED-BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13445.html
Let's discuss them here.
First of all, can you confirm that you are on Ignite 2.0?
Pavel
On Wed, Jun 7, 2017 at 11:03 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi There,
This one is very strange.
I have a .NET client, Java client and Java server. All running locally on
windows desktop.
On the server side I have some cache entries, SecurityKey,Security
I have two Security objects, one with id=4 and one with id=953
On the Java client, this works no problem
SystemContext.getCache().get(new SecurityKey(4));
SystemContext.getCache().get(new SecurityKey(953));
On the .NET client
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 4 });
Will succeed
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 953 });
Will FAIL, with KeyNotFoundException
This is the case 100% of the time, even after full restart of the environment.
The only thing that seems to cause the problem is the literal value of the id
Also note that if I iterate over the cache on the .NET side, I can absolutely
see the SecurityKey with id=953, but I will see KeyNotFound when I try to Get
foreach (var e in SystemContext.Cache)
{
if (e.Key is SecurityKey)
{
Log.Info(((SecurityKey)e.Key).Id); // this will print both
4 and 953
}
}
My only guess here is that something funny is going on internally with the
hashing of the binary object fields. So the equality test is failing when the
get is done from the .NET client??
I have attached the key class as it is on the .NET side and the Java side.
Any thoughts are much appreciated. Thanks!
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confiden
Re: Get by key only working on .net for certain long literal values
Hi. Yes, I am on 2.0.
Get Outlook for Android<https://aka.ms/ghei36>
From: Pavel Tupitsyn <ptupit...@apache.org>
Sent: Thursday, June 8, 2017 12:26:41 AM
To: user@ignite.apache.org
Subject: Re: Get by key only working on .net for certain long literal values
Hi Gordon,
Seems like all your threads are related:
http://apache-ignite-users.70518.x6.nabble.com/BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13444.html
http://apache-ignite-users.70518.x6.nabble.com/UPDATED-BinaryIdentityResolver-is-gone-in-2-0-but-default-seems-to-have-a-problem-td13445.html
Let's discuss them here.
First of all, can you confirm that you are on Ignite 2.0?
Pavel
On Wed, Jun 7, 2017 at 11:03 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Hi There,
This one is very strange.
I have a .NET client, Java client and Java server. All running locally on
windows desktop.
On the server side I have some cache entries, SecurityKey,Security
I have two Security objects, one with id=4 and one with id=953
On the Java client, this works no problem
SystemContext.getCache().get(new SecurityKey(4));
SystemContext.getCache().get(new SecurityKey(953));
On the .NET client
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 4 });
Will succeed
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 953 });
Will FAIL, with KeyNotFoundException
This is the case 100% of the time, even after full restart of the environment.
The only thing that seems to cause the problem is the literal value of the id
Also note that if I iterate over the cache on the .NET side, I can absolutely
see the SecurityKey with id=953, but I will see KeyNotFound when I try to Get
foreach (var e in SystemContext.Cache)
{
if (e.Key is SecurityKey)
{
Log.Info(((SecurityKey)e.Key).Id); // this will print both
4 and 953
}
}
My only guess here is that something funny is going on internally with the
hashing of the binary object fields. So the equality test is failing when the
get is done from the .NET client??
I have attached the key class as it is on the .NET side and the Java side.
Any thoughts are much appreciated. Thanks!
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Get by key only working on .net for certain long literal values
Hi There,
This one is very strange.
I have a .NET client, Java client and Java server. All running locally on
windows desktop.
On the server side I have some cache entries, SecurityKey,Security
I have two Security objects, one with id=4 and one with id=953
On the Java client, this works no problem
SystemContext.getCache().get(new SecurityKey(4));
SystemContext.getCache().get(new SecurityKey(953));
On the .NET client
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 4 });
Will succeed
(Security)SystemContext.Cache.Get(new SecurityKey() { Id = 953 });
Will FAIL, with KeyNotFoundException
This is the case 100% of the time, even after full restart of the environment.
The only thing that seems to cause the problem is the literal value of the id
Also note that if I iterate over the cache on the .NET side, I can absolutely
see the SecurityKey with id=953, but I will see KeyNotFound when I try to Get
foreach (var e in SystemContext.Cache)
{
if (e.Key is SecurityKey)
{
Log.Info(((SecurityKey)e.Key).Id); // this will print both
4 and 953
}
}
My only guess here is that something funny is going on internally with the
hashing of the binary object fields. So the equality test is failing when the
get is done from the .NET client??
I have attached the key class as it is on the .NET side and the Java side.
Any thoughts are much appreciated. Thanks!
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
namespace Dragon9GUI.SharedDotNetDataModel.Security
{
public class SecurityKey : IKey
{
public long Id { get; set; }
public override bool Equals(object obj)
{
if (ReferenceEquals(null, obj)) return false;
if (ReferenceEquals(this, obj)) return true;
if (obj.GetType() != this.GetType()) return false;
return Id == ((SecurityKey)obj).Id;
}
public override int GetHashCode()
{
return (int)(Id ^ (Id >> 32));
}
}
}
SecurityKey.java
Description: SecurityKey.java
BinaryIdentityResolver is gone in 2.0 but default seems to have a problem
Hi There,
I am able to load the same row into my cache from my persistent store twice,
and have it present in my cache twice. It seems the id hash is different for
the key.
The key class is very simple, just one int id.
And here is what I see debugging. How can the id hash be different here.??
[cid:image002.png@01D2DF90.E714F240]
[cid:image001.png@01D2DF8E.D56AEBF0]
Note that there are a few steps to get into this weird state.
Start java server. Key 953 is loaded. There is only one instance as expected.
Read through = true
Now start .NET client and call Cache.Get(new SecurityKey(){Id=953}). For some
reason it is not found in the java cache, and read through is invoked on the
java side.
Now we have two SecurityKey in the cache, both with id=953, but with different
id hash.
What could possibly be happening here?
Thanks,
Gordon.
The Key class just looks like this.
public class SecurityKey implements CacheKey {
private int id;
public SecurityKey() {
// No-op.
}
public SecurityKey(int id) {
this.id = id;
}
public int getId() {
return id;
}
@Override
public void setId(int id) {
this.id = id;
}
/** {@inheritDoc} */
@Override public String toString() {
return "SecurityKey [id=" + id +
"]";
}
}
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
CacheWriteSynchronizationMode
Hi Guys, A new write sync mode CacheWriteSynchronizationMode.LOCAL_SYNC Would be very handy for us. Has this been brought up before? Any plan to implement it? We are running in REPLICATED and we want the local node to be updated sync, but other nodes async. Using PRIMARY_SYNC doesn't quite work because the primary node for the key may not always be the local node. Or is there some other way to achieve this by ensuring that the local node is the primary? Thanks, This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Get all services in cluster by name
Hi There,
I know I can do this
ignite.services().services("MyService");
To get all locally deployed instances of my service. But how can I get a
collection of all MyService across the entire cluster? I want to know each
instance, not just any single remote instance.
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync
Thanks Humphrey, yes I have read that. But it is still not clear to me how the primary node for a key is assigned in a REPLICTED cache, where we have no specific key affinity specified. From: Humphrey Lopez [mailto:hmmlo...@gmail.com] Sent: Tuesday, 16 May 2017 2:52 PM To: user@ignite.apache.org Subject: Re: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync https://apacheignite.readme.io/v2.0/docs/primary-and-backup-copies#synchronous-and-asynchronous-backups Humphrey On 16 May 2017, at 03:14, Gordon Reid (Nine Mile) <gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>> wrote: Thanks Humphrey. And in the case of PRIMARY_SYNC, how is the primary node determined for the entity? Will it be the node where the entity was created? (we are in REPLICATED) Thanks,. -Original Message- From: Humphrey [mailto:hmmlo...@gmail.com] Sent: Monday, 15 May 2017 5:19 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync Hi Gordon, FULL_ASYNC: Do not wait for any server node to acknowledge So it will fire and forget, and will not wait for all nodes in cluster to be updated. That will happen in the background. Humphrey -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-0-CacheWriteSynchronizationMode-FULL-ASYNC-when-using-cache-putAsync-tp12837p12839.html Sent from the Apache Ignite Users mailing list archive at Nabble.com<http://Nabble.com>. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync
Thanks Humphrey. And in the case of PRIMARY_SYNC, how is the primary node determined for the entity? Will it be the node where the entity was created? (we are in REPLICATED) Thanks,. -Original Message- From: Humphrey [mailto:hmmlo...@gmail.com] Sent: Monday, 15 May 2017 5:19 PM To: user@ignite.apache.org Subject: Re: Ignite 2.0: CacheWriteSynchronizationMode = FULL_ASYNC when using cache.putAsync Hi Gordon, FULL_ASYNC: Do not wait for any server node to acknowledge So it will fire and forget, and will not wait for all nodes in cluster to be updated. That will happen in the background. Humphrey -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-2-0-CacheWriteSynchronizationMode-FULL-ASYNC-when-using-cache-putAsync-tp12837p12839.html Sent from the Apache Ignite Users mailing list archive at Nabble.com. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
Allow ALL nodes to keep working when a new node joins the cluster
Hi There, For our application (High Frequency Trading) it is critical that no node pauses when another server or client mode joins the cluster. We run GUIs in client mode. It is not acceptable that the server node pauses processing (even for 1 second) just because a trader opened the GUI. Is it possible to configure ignite such that processing continues while nodes join and leave the cluster? Thanks, Gordon. This email and any attachments are proprietary & confidential and are intended solely for the use of the individuals to whom it is addressed. Any views or opinions expressed are solely for those of the author and do not necessarily reflect those of Nine Mile Financial Pty. Limited. If you have received this email in error, please let us know immediately by reply email and delete from your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
RE: CacheAbstractJdbcStore batch deleteAll bug.
Sorry, I don’t have time to produce a sample right now, but I can see that this
bug still exists in ignite 2.0
Just look in CacheAbstractJdbcStore.deleteAll
You can see that delStmt never gets updated after it is first initialized
Thanks,
Gordon.
From: Alexey Kuznetsov [mailto:akuznet...@apache.org]
Sent: Thursday, 27 April 2017 6:50 PM
To: user@ignite.apache.org
Subject: Re: CacheAbstractJdbcStore batch deleteAll bug.
Hi, Gordon.
Do you have a reproducer we could debug?
Thanks.
On Wed, Apr 26, 2017 at 9:50 AM, Gordon Reid (Nine Mile)
<gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>>
wrote:
Actually the fix will be a little more complicated, because the variable “em”
has already been updated to the new type before the last batch has been
executed.
From: Gordon Reid (Nine Mile)
[mailto:gordon.r...@ninemilefinancial.com<mailto:gordon.r...@ninemilefinancial.com>]
Sent: Wednesday, 26 April 2017 12:05 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: CacheAbstractJdbcStore batch deleteAll bug.
Hi Guys,
I think there is a problem in CacheAbstractJdbcStore.deleteAll
I find that sometimes, I see messages like this
[DEBUG] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Delete entries from db [cache=D9Cache, keyType=com.nmf.model.trading.TradeKey,
cnt=2]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade,
entry=RiskRuleKey [id=1], expected=1, actual=0]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade,
entry=RiskRuleKey [id=2], expected=1, actual=0]
Note the entity types are different!
Also look at the attached image. You can see the statement is still targeting
Trade, but our entity is now Position.
You can see in the code, that the delStmt never gets refreshed, when changing
over to a different entity type.
for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);
em = entryMapping(cacheName, keyTypeId);
if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);
currKeyTypeId = keyTypeId;
}
if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");
executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);
fromIdx += prepared;
prepared = 0;
currKeyTypeId = keyTypeId;
}
I believe should be something like this
for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);
em = entryMapping(cacheName, keyTypeId);
if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);
currKeyTypeId = keyTypeId;
}
if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");
executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);
fromIdx += prepared;
prepared = 0;
currKeyTypeId = keyTypeId;
delStmt = conn.prepareStatement(em.remQry);
}
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
--
Alexey Kuznetsov
This
RE: CacheAbstractJdbcStore batch deleteAll bug.
Actually the fix will be a little more complicated, because the variable "em"
has already been updated to the new type before the last batch has been
executed.
From: Gordon Reid (Nine Mile) [mailto:gordon.r...@ninemilefinancial.com]
Sent: Wednesday, 26 April 2017 12:05 PM
To: user@ignite.apache.org
Subject: CacheAbstractJdbcStore batch deleteAll bug.
Hi Guys,
I think there is a problem in CacheAbstractJdbcStore.deleteAll
I find that sometimes, I see messages like this
[DEBUG] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Delete entries from db [cache=D9Cache, keyType=com.nmf.model.trading.TradeKey,
cnt=2]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade,
entry=RiskRuleKey [id=1], expected=1, actual=0]
[WARN ] 2017-04-26 11:08:54.097 [flusher-0-#31%null%] CacheJdbcPojoStore -
Batch deleteAll returned unexpected updated row count [table=public.nmfctrade,
entry=RiskRuleKey [id=2], expected=1, actual=0]
Note the entity types are different!
Also look at the attached image. You can see the statement is still targeting
Trade, but our entity is now Position.
You can see in the code, that the delStmt never gets refreshed, when changing
over to a different entity type.
for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);
em = entryMapping(cacheName, keyTypeId);
if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);
currKeyTypeId = keyTypeId;
}
if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");
executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);
fromIdx += prepared;
prepared = 0;
currKeyTypeId = keyTypeId;
}
I believe should be something like this
for (Object key : keys) {
Object keyTypeId = typeIdForObject(key);
em = entryMapping(cacheName, keyTypeId);
if (delStmt == null) {
delStmt = conn.prepareStatement(em.remQry);
currKeyTypeId = keyTypeId;
}
if (!currKeyTypeId.equals(keyTypeId)) {
if (log.isDebugEnabled())
log.debug("Delete entries from db [cache=" + U.maskName(cacheName) +
", keyType=" + em.keyType() + ", cnt=" + prepared + "]");
executeBatch(em, delStmt, "deleteAll", fromIdx, prepared, lazyKeys);
fromIdx += prepared;
prepared = 0;
currKeyTypeId = keyTypeId;
delStmt = conn.prepareStatement(em.remQry);
}
Thanks,
Gordon.
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
This email and any attachments are proprietary & confidential and are intended
solely for the use of the individuals to whom it is addressed. Any views or
opinions expressed are solely for those of the author and do not necessarily
reflect those of Nine Mile Financial Pty. Limited. If you have received this
email in error, please let us know immediately by reply email and delete from
your system. Nine Mile Financial Pty. Limited. ABN: 346 1349 0252
