Usign Apache for non-SQL related grid computing

2017-02-02 Thread Raymond Wilson
Hi, I have a potential use case where I have a collection of blocks of binary data that I would like to perform parameterised compute operations across. The data is not stored in an RDBMS, but more in a flat file oriented fashion. The blocks of data are spatial in origin (in that they are

RE: Effective size limit for cache items in Ignite

2017-02-13 Thread Raymond Wilson
by using Binary Mode: https://apacheignite-net.readme.io/docs/binary-mode Pavel On Mon, Feb 13, 2017 at 6:38 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: I found this code snippet in the Ignite v1.8 source: This appears to be the core place where the value of CopyOnRead has an

Continuous reduce in Ignite

2017-02-10 Thread Raymond Wilson
I’m looking at the very nice MapReduce implementation in Ignite and have a question about the reduce stage. It appears that the reduce stage only occurs once all mapped results have been accumulated. I have a use case where the size of that data set is very large and problematic to hold in

Re: Continuous reduce in Ignite

2017-02-10 Thread Raymond Wilson
Hi Val, This would be from .Net. Is there an option for extending the .Net client to support this? Thanks, Raymond. Sent from my iPhone > On 11/02/2017, at 11:19 AM, vkulichenko wrote: > > Hi Raymond, > > Are you implementing tasks in Java or .NET? For Java

RE: Continuous reduce in Ignite

2017-02-12 Thread Raymond Wilson
It turns out this annotation is also present in .Net and works the same way as the Java annotation. Eg: [Serializable] [ComputeTaskNoResultCache] class MyComputeTask : IComputeTask { } Sweet! :) -Original Message- From: vkulichenko

Effective size limit for cache items in Ignite

2017-02-12 Thread Raymond Wilson
Hi, What is the practical size limit for items in an Ignite cache? I suspect the answer is something “As large as the memory you have to hold it”, but my question is more aimed at the practicality of large items in a cache due to the overhead of pulling copies of the items out of the cache

RE: Effective size limit for cache items in Ignite

2017-02-12 Thread Raymond Wilson
he cache items for (int i = 0; i < 3; i++) { MyCacheClass first = cache.Get("First"); } } *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Monday, February 13, 2017 11:35 AM *To:* user@ignite.apa

RE: Effective size limit for cache items in Ignite

2017-02-12 Thread Raymond Wilson
p; !isBinaryEnabled(ccfg)); ctx.resource().injectGeneric(res.defaultAffMapper()); return res; } Thanks, Raymond. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Monday, February 13, 2017 2:43 PM *To:* user@ignite.apache.org *Subject:* RE: Effective si

Exception being thrown when setting Active to true in Ignite 2.1 with Persitence enabled

2017-08-01 Thread Raymond Wilson
I’m testing out a first piece of code to use the Ignite persistence layer in the C# client. I have added a persistence layer aspect to the configuration for the grid, like this: cfg.PersistentStoreConfiguration = new PersistentStoreConfiguration() {

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-02 Thread Raymond Wilson
activate and deactivate the cluster? On Wed, Aug 2, 2017 at 3:18 PM, Michael Cherkasov < michael.cherka...@gmail.com> wrote: when all nodes are up, so in latest topology snapshot you can see servers count == servers count you run, then cluster can be activated. 2017-08-02 0:51 GMT+03:00

RE: Accessing array elements within items cached in Ignite without deserialising the entire item

2017-08-02 Thread Raymond Wilson
. I've filed a ticket: https://issues.apache.org/jira/browse/IGNITE-5904 Thank you for a good suggestion! And, by the way, everyone is welcome to contribute, this ticket can be a perfect start! Pavel On Wed, Aug 2, 2017 at 12:46 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi

RE: Accessing array elements within items cached in Ignite without deserialising the entire item

2017-08-01 Thread Raymond Wilson
use WriteObject<>/ReadObject<> to avoid dealing with nullables 3. Random array access is not possible with current API. Thanks, Pavel On Tue, Aug 1, 2017 at 2:46 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi, I’ve been looking at IBinarizable and IBinarySeria

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-01 Thread Raymond Wilson
nodes can still accept request for data update and other nodes won't see it, so understand which node has the latest data we need to start all nodes first and only then activate cluster. Thanks, Mikhail. On Tue, Aug 1, 2017 at 5:05 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi

RE: Accessing array elements within items cached in Ignite without deserialising the entire item

2017-08-03 Thread Raymond Wilson
any assistance. Pavel On Thu, Aug 3, 2017 at 5:28 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Pavel, Thanks for putting it on the plan. I’ve been reading through the ‘how to contribute’ documentation to see what’s required and have pulled a static download o

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-03 Thread Raymond Wilson
ivated. 2017-08-02 0:51 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: Hi Mikhail, Thanks for the clarifications. Yes, I knew setting active was only required when using the persistence layer, which is the topic of the question J I was interested if there were b

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-03 Thread Raymond Wilson
/persistentstore does not address these aspects. Thanks, Raymond. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Friday, August 4, 2017 1:56 PM *To:* 'user@ignite.apache.org' *Subject:* RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1 Hi

Cache item factory

2017-07-26 Thread Raymond Wilson
Hi, I would like to control the way objects are created in response to ICache.Get() methods. For instance given a cache defined like this: ICache MyCache; I would like to provide a factory to create the MemoryStream instance when Ignite needs to create an

RE: Cache item factory

2017-07-26 Thread Raymond Wilson
costs, use object pools, etc. Let me know if you need help on this with your particular use case. Thanks, Pavel On Wed, Jul 26, 2017 at 12:53 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi, I would like to control the way objects are created in response to ICac

RE: Possible race condition when starting up Ignite nodes taking part in partition affinity map

2017-07-03 Thread Raymond Wilson
to my client test app with the same data set. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday, July 4, 2017 10:00 AM *To:* user@ignite.apache.org *Subject:* Possible race condition when starting up Ignite nodes taking part in partition affinity map Hi, I have

Possible race condition when starting up Ignite nodes taking part in partition affinity map

2017-07-03 Thread Raymond Wilson
Hi, I have been working on a POC using the Ignite v1.9 C# client and have been simulating clusters of Ignite nodes by running collections of Ignite processes on the same physical machine. As a part of this I am running 4 Ignite nodes (where each node is a windows forms application for

Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-07-31 Thread Raymond Wilson
Hi, I am experimenting with a POC looking into using the Ignite persistence layer. One aspect of this is setting the grid to be ‘Active’ after all cache grid nodes have instantiated. In practical terms, what is the best practice for ensuring the cluster is running and in a good state to

Accessing array elements within items cached in Ignite without deserialising the entire item

2017-07-31 Thread Raymond Wilson
Hi, I’ve been looking at IBinarizable and IBinarySerializer with regards to controlling object serialization (using the Ignite.Net client). A couple of questions: 1. Some of the APIs in IBinarizable allow for a factory methods to control construction of collection and dictionary

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-07 Thread Raymond Wilson
cluster. Thanks, Mikhail. On Mon, Aug 7, 2017 at 4:52 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Michael, I wrote a simple application to act as an activator for a grid. As this is just a POC, it’s simply a form with a button on it that does this:

RE: Best practise for setting Ignite.Active to true when using persistence layer in Ignite 2.1

2017-08-06 Thread Raymond Wilson
g/apache/ignite/examples/persistentstore does not address these aspects. *Unfortunately, there's no event for this now, so you should spin around Ignite.active()* Thanks, Mikhail. On Fri, Aug 4, 2017 at 5:01 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Sorry, adding a

RE: Write behind using Grid Gain

2017-06-13 Thread Raymond Wilson
your use case fits well into traditional Ignite model of write-through cache store with backing database. Why do you want to avoid a DB? Do you plan to store data on disk directly as a set of files? Pavel On Mon, Jun 12, 2017 at 2:14 AM, Raymond Wilson <raymond_wil...@trimble.com> wrot

RE: Write behind using Grid Gain

2017-06-13 Thread Raymond Wilson
The store should be released by the community in the nearest. — Denis On Jun 13, 2017, at 2:02 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Pavel, It’s a little complicated. The system is essentially a DB in its own right; actually it’s an IMDG a bit like Ignite, but dev

RE: Write behind using Grid Gain

2017-06-11 Thread Raymond Wilson
Hi Pavel, Thanks for the blog – it explains it quite well. I have a slightly different use case where groups of records within a much larger data set are clustered together for efficiency (ie: each of the cached items in the Ignite grid cache has significant internal structure). You can

RE: Possible dead lock when number of jobs exceeds thread pool

2017-10-04 Thread Raymond Wilson
Hi Alex, There's a fair bit of source involved, I think I'd need to build a reproducer app. I'd like to talk it through a bit before putting in that effort. There are actually three parts: Client --> Service A --> Service B A call from Client to Service A causes Service A to request a series

Possible dead lock when number of jobs exceeds thread pool

2017-10-03 Thread Raymond Wilson
I am testing an aspect of a POC I have written in C# to determine how well it performs when processing multiple tasks at one time. I’m using Ignite 2.2. Briefly, a client node sends a request to a server node for a list of items. Each of the items requested is send back to a listener in the

Error with ScanQuery

2017-10-15 Thread Raymond Wilson
I have an existing Ignite (v2.2) cache using the Persistency layer that contains a number of cache entries. While using a test application on top of the cache I experienced an issue where the local computer ran out of virtual memory and essentially crashed (long story). I have since resolved

Re: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
I can do that tomorrow. In the meantime, can you speak to what that error means? Thanks, Raymond. Sent from my iPhone > On 16/10/2017, at 9:18 PM, dkarachentsev wrote: > > Hi Raymond, > > Could you please attach full log and config for failed node? > > Thanks!

RE: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
Hi Dmitry, I don’t seem to get any Java exceptions reported in the log. Below is the inner exception detail from the IDE error dialog: class org.apache.ignite.binary.BinaryObjectException: Requesting mapping from grid failed for [platformId=1, typeId=349095370] at

RE: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
) in C:\Dev\VSS.Raptor.IgnitePOC\WindowsFormsApplication1\Form1.cs:line 254 --- OK --- *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday, October 17, 2017 12:17 AM *To:* 'user@ignite.apache.org' <user@ignite.apache.org> *Su

RE: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
byte[] instead (call MemoryStream.ToArray()). Thanks, Pavel On Mon, Oct 16, 2017 at 11:46 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: I can do that tomorrow. In the meantime, can you speak to what that error means? Thanks, Raymond. Sent from my iPhone > On 16/10/2017, a

Re: Possible dead lock when number of jobs exceeds thread pool

2017-10-05 Thread Raymond Wilson
Hi Alex, Thanks for clarifying. I have been testing increasing the public pool size and it does help. Is there a general work programme to bring the C# client into parity with the Java client? Thanks, Raymond. Sent from my iPhone > On 6/10/2017, at 2:26 AM, afedotov

RE: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
I just read through the Jira ticket and wonder if this is the underlying cause for what I am seeing as I have KeepBinaryInStore=false in my configuration. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday, October 17, 2017 9:49 AM *To:* 'user@ignite.apache.org' <u

RE: Error with ScanQuery

2017-10-16 Thread Raymond Wilson
I am using the Ignite native persistence introduced in the 2.* releases. *From:* Alexey Kukushkin [mailto:kukushkinale...@gmail.com] *Sent:* Tuesday, October 17, 2017 12:48 AM *To:* user@ignite.apache.org *Subject:* Re: Error with ScanQuery Raymond, Do you have native or 3-rd party

RE: Massive commit sizes for processes with local Ignite Grid servers

2017-09-07 Thread Raymond Wilson
ry MemoryPolicies = new[] { new MemoryPolicyConfiguration { Name = "defaultPolicy", MaxSize = 4L * 1024 * 1024 * 1025 // 4 GB } } чт, 7 сент. 2017 г. в 12:44, Raymond Wilson <raymond_wil...@trimble.com>: I tried an exper

Massive commit sizes for processes with local Ignite Grid servers

2017-09-07 Thread Raymond Wilson
I’m running a set of four server applications on a local system to simulate a cluster. Each of the servers has the following memory configurations set: public override void ConfigureRaptorGrid(IgniteConfiguration cfg) { cfg.JvmInitialMemoryMb = 512; // Set to

RE: Massive commit sizes for processes with local Ignite Grid servers

2017-09-07 Thread Raymond Wilson
I tried an experiment where I ran only two instances of the server locally, this is the result in the Task Manager: *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Thursday, September 7, 2017 9:21 PM *To:* user@ignite.apache.org; 'd...@ignite.apache.org' &l

RE: Oddity running multiple server nodes locally

2017-09-07 Thread Raymond Wilson
It appears the Commit Size issue I raised in another thread may have been the cause of this. When I defined a default memory policy in the server nodes to restrict memory use to 1Gb then the servers activate as expected. -Original Message- From: Raymond Wilson [mailto:raymond_wil

RE: Accessing array elements within items cached in Ignite without deserialising the entire item

2017-08-30 Thread Raymond Wilson
s important, but correctness is always a priority. On Fri, Aug 4, 2017 at 5:25 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: I had not seen that page yet – very useful. There’s a few moving parts to getting it working, so not sure I will get time to really dig into, but will have a lo

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
Sep 4, 2017 at 6:07 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Dmitriy, I set up an XML file based on the default one and added the two elements you noted. However, this has brought up an issue in that the XML file and an IgniteConfiguration instance can’t both

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
*To:* user <user@ignite.apache.org> *Cc:* Raymond Wilson <raymond_wil...@trimble.com> *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi, It’s possible this could cause ch

RE: Specifying location of persistent storage location

2017-09-05 Thread Raymond Wilson
7-09-05 4:07 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: > Dmitriy, > > > > I set up an XML file based on the default one and added the two elements > you noted. > > > > However, this has brought up an issue in that the XML file and an >

RE: Specifying location of persistent storage location

2017-09-05 Thread Raymond Wilson
olders under work directory with their sizes and dates. --Yakov 2017-09-05 4:07 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: Dmitriy, I set up an XML file based on the default one and added the two elements you noted. However, this has brought up an issue in that

RE: Oddity running multiple server nodes locally

2017-09-06 Thread Raymond Wilson
Message- From: Raymond Wilson [mailto:raymond_wil...@trimble.com] Sent: Thursday, September 7, 2017 11:29 AM To: 'user@ignite.apache.org' <user@ignite.apache.org> Subject: RE: Oddity running multiple server nodes locally Hi Val, I'm not getting any obvious logging. I'm using Log4Net to con

RE: Oddity running multiple server nodes locally

2017-09-06 Thread Raymond Wilson
Correction to the information below. The activator app does not stall on starting a local Ignite instance, it stalls when calling Ignite.Start() Apologies for the confusion, not sure how that happened! -Original Message- From: Raymond Wilson [mailto:raymond_wil...@trimble.com] Sent

RE: Oddity running multiple server nodes locally

2017-09-06 Thread Raymond Wilson
And with some help from Google I found JStack. :) -Original Message- From: Raymond Wilson [mailto:raymond_wil...@trimble.com] Sent: Thursday, September 7, 2017 1:25 PM To: 'user@ignite.apache.org' <user@ignite.apache.org> Subject: RE: Oddity running multiple server nodes l

RE: Massive commit sizes for processes with local Ignite Grid servers

2017-09-07 Thread Raymond Wilson
app takes. Is it possible your app really loads additional 3GB of referenced libraries and data besides Ignite? Did you try temporarily changing the code to NOT start Ignite and see how much memory such an app takes? On Thu, Sep 7, 2017 at 1:49 PM, Raymond Wilson <raymond_wil...@trimble.com> w

Oddity running multiple server nodes locally

2017-09-06 Thread Raymond Wilson
I’m using Ignite 2.1 with persistence and have a POC running successfully using a single server node. I now want to simulate running a cluster of server nodes locally. To do this I’m running multiple (4) instances of my server node via a batch file in a command shell. I can see from the naming

RE: Oddity running multiple server nodes locally

2017-09-06 Thread Raymond Wilson
Hi Val, I'm not getting any obvious logging. I'm using Log4Net to configure logging, and I am getting Ignite logging during some operations, but perhaps I need to modify the logging configuration to pick up additional information. I have both the Log4Net and Apache.Ignite.Log4Net nugget packages

Re: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
vives > node restarts. > > Sincerely, > Dmitriy Pavlov > > > сб, 2 сент. 2017 г. в 23:40, Raymond Wilson <raymond_wil...@trimble.com>: >> Hi, >> >> >> >> I’m running a POC looking at the Ignite Persistent Store fea

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday, September 5, 2017 6:04 AM *To:* user@ignite.apache.org *Subject:* Re: Specifying location of persistent storage location Thank you Dmitry! Sent from my iPhone On 5/09/2017, at 1:12 AM, Dmitry Pavlov <dpavlov@gmail.

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
… also, the documentation for ClusterNode here ( https://www.gridgain.com/sdk/pe/latest/javadoc/org/apache/ignite/cluster/ClusterNode.html) only describes a getter for the consistent ID, I need to be able to set it. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
, Dmitriy Pavlov вт, 5 сент. 2017 г. в 0:12, Raymond Wilson <raymond_wil...@trimble.com>: … also, the documentation for ClusterNode here ( https://www.gridgain.com/sdk/pe/latest/javadoc/org/apache/ignite/cluster/ClusterNode.html) only describes a getter for the consistent ID, I need to be able

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
rent folders created under the work folder, you probably had more than one node running at the same time. Can you please make sure that it was not the case? D. On Mon, Sep 4, 2017 at 2:55 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Dmitry, I looked at IgniteConfiguratio

RE: Specifying location of persistent storage location

2017-09-04 Thread Raymond Wilson
:* Dmitriy Setrakyan [mailto:dsetrak...@apache.org] *Sent:* Tuesday, September 5, 2017 11:17 AM *To:* user <user@ignite.apache.org> *Cc:* Raymond Wilson <raymond_wil...@trimble.com> *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 3:37

Specifying location of persistent storage location

2017-09-02 Thread Raymond Wilson
Hi, I’m running a POC looking at the Ignite Persistent Store feature. I have added a section to the configuration for the Ignite grid as follows: cfg.PersistentStoreConfiguration = new PersistentStoreConfiguration() { PersistentStorePath =

RE: Possible dead lock when number of jobs exceeds thread pool

2017-10-04 Thread Raymond Wilson
It seems that the C# API does not support custom thread pools. There is no WithExecutor() method on ICompute, and looking at the client source I don't see any relevant executor support. Is this due to the Async method variants supported on the C# API? -Original Message- From: Raymond

RE: Possible dead lock when number of jobs exceeds thread pool

2017-10-04 Thread Raymond Wilson
Alex, Thanks for the pointer to the custom thread pools. The ExecutorConfiguration aspect of the grid configuration does not appear to be present in the C# API. Is this only supported in the XML configuration file for C# clients? Raymond. -Original Message- From: afedotov

Running out of memory using ScanQuery. Memory leak?

2017-10-17 Thread Raymond Wilson
I have run into something odd using ScanQuery (using Ignite 2.2, C# client and native persistence). I have a fairly simple piece of scanning code below, it simple looks through all the keys in a cache and emits them to a file. It creates a ScanQuery, sets its page size to 1, and explicitly

RE: Error with ScanQuery

2017-10-17 Thread Raymond Wilson
I made the changes to store cache items as byte[] instead of MemoryStream which has reduced the overall storage requirement by around 30%, which is a nice result  Thanks! Raymond. *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Tuesday, October 17, 2017 12:22 AM

RE: Running out of memory using ScanQuery. Memory leak?

2017-10-17 Thread Raymond Wilson
IQueryCursor<ICacheEntry<SubGridSpatialAffinityKey, byte[]>> queryCursor = cache.Query(scanQuery); foreach (ICacheEntry<SubGridSpatialAffinityKey, byte[]> cacheEntry in queryCursor) { writer.WriteLine(cacheEntry.Key.ToString()); }

RE: Error with ScanQuery

2017-10-19 Thread Raymond Wilson
essage -- From: *Raymond Wilson* <raymond_wil...@trimble.com> Date: Mon, Oct 16, 2017 at 8:35 PM Subject: RE: Error with ScanQuery To: Dmitriy Setrakyan <dsetrak...@apache.org> Cc: ptupit...@apache.org Dmitry, Pavel, Attached are the logs from running the system from s

Error when deploying a service into an Ignite cluster

2017-11-12 Thread Raymond Wilson
I have made a simple C# service in Ignite 2.2. When I deploy it, like this: services.DeployNodeSingleton(“AddSurveyedSurface”, new AddSurveyedSurfaceService()); I get the following error: ERROR 2017-11-13 15:29:35,984 319109ms GridServiceProcessor ?

Does the Ignite C# client support distributed queues?

2017-11-30 Thread Raymond Wilson
I’ve been reading up on the distributed queue support in Ignite which may be a good fit for a use case we have. Looking at the Ignite C# client this does not seem to be supported. Is this supported in the C# client? If not, when is it planned to be included? Thanks, Raymond.

RE: Does the Ignite C# client support distributed queues?

2017-11-30 Thread Raymond Wilson
Hi Val, This seems to be related to queries, not queues as described here: https://apacheignite.readme.io/docs/queue-and-set Thanks, Raymond. -Original Message- From: vkulichenko [mailto:valentin.kuliche...@gmail.com] Sent: Friday, December 1, 2017 11:49 AM To: user@ignite.apache.org

Did the default page size change between Ignite 2.2 and 2.3

2017-11-30 Thread Raymond Wilson
I am upgrading from Ignite 2.2 to 2.3. I use persistent storage and have an existing data store. When activating the cluster I get the following error in the C# client: class org.apache.ignite.IgniteCheckedException: Failed to verify store file (invalid page size) [expectedPageSize=4096,

RE: Does the Ignite C# client support distributed queues?

2017-11-30 Thread Raymond Wilson
Looking at it I see it's blocked by 2701 (which has additional dependencies, all of which say they are blocked by 2701). I understand there is an intention to bring the C# client up to par with the Java client. Is there a ticket/schedule yet for this? Raymond. -Original Message- From:

RE: Did the default page size change between Ignite 2.2 and 2.3

2017-11-30 Thread Raymond Wilson
this exception. — Denis On Nov 30, 2017, at 6:16 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: I am upgrading from Ignite 2.2 to 2.3. I use persistent storage and have an existing data store. When activating the cluster I get the following error in the C# client:

RE: Does the Ignite C# client support distributed queues?

2017-12-01 Thread Raymond Wilson
2701 It was created almost 1 year ago. Please vote for this feature. As far as I know there is no plan/schedule for it. Thank you, Alexey *From: *Raymond Wilson <raymond_wil...@trimble.com> *Sent: *Friday, December 1, 2017 5:58 AM *To: *user@ignite.apache.org *Cc: *d...@ignite.apache.org *Su

Obtaining metadata about items in the cache

2017-12-03 Thread Raymond Wilson
Hi, I’d like to be able to scan all the items in a cache where all I am interested in is the cache key and other metadata about the cached item (such as its size). I can do this now by running a cache query that simple reads out all the cache items, but this is a lot of IO when I don’t care

RE: Does the Ignite C# client support distributed queues?

2017-12-03 Thread Raymond Wilson
distributed queues? Hi Raymond, There is no exact schedule, but we always strive to keep .NET API up to speed with Java. Usually new features are added right in the same release (persistence in 2.1, for example). Pavel On Sat, Dec 2, 2017 at 9:51 AM, Raymond Wilson <raymond_

RE: Does the Ignite C# client support distributed queues?

2017-12-04 Thread Raymond Wilson
/dotnet/Apache.Ignite.Core.Tests/ApiParity/IgniteParityTest.cs#L39 Pavel On Sun, Dec 3, 2017 at 11:47 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Pavel, Is there a document that captures all the current gaps? Thanks, Raymond. *From:* Pavel Tupitsyn [mailto:

RE: Does the Ignite C# client support distributed queues?

2017-12-04 Thread Raymond Wilson
. As far as I know there is no plan/schedule for it. Thank you, Alexey *From: *Raymond Wilson <raymond_wil...@trimble.com> *Sent: *Friday, December 1, 2017 5:58 AM *To: *user@ignite.apache.org *Cc: *d...@ignite.apache.org *Subject: *RE: Does the Ignite C# client support distributed

Tips for implementing unit tests using Ignite.Net

2017-12-13 Thread Raymond Wilson
I have a Visual Studio solution with some implementation that uses Ignite.Net and also a collection of unit tests. When I try to run a unit test that uses an Ignite capability I run into issues where the unit test fails with an unexpected error when executing a line like this:

Ignite caches with separate storage

2017-12-17 Thread Raymond Wilson
I have a use case I’d like to explore with Ignite: We have a data ingest pipeline that transforms in bound data into a large collections of mutable records (stored in an Ignite cache). Changes to this mutable data is relatively frequent and imposes significant IO updating changes. From those

RE: Error serialising arrays using Ignite 2.2 C# client

2017-11-13 Thread Raymond Wilson
Thanks Alexey. -Original Message- From: Alexey Popov [mailto:tank2.a...@gmail.com] Sent: Tuesday, November 14, 2017 5:09 AM To: user@ignite.apache.org Subject: Re: Error serialising arrays using Ignite 2.2 C# client Hi Raymond, You are right. True multidimensional arrays are not

Error serialising arrays using Ignite 2.2 C# client

2017-11-10 Thread Raymond Wilson
I’m using the Ignite 2.2 C# client to make ComputeTask calls. I have a type in an argument to one of those calls that is a two dimensional array of floats, like this: [Serializable] Public float[,] = new float[32, 32]; When this field is present in a structure being serialized (either in

RE: Obtaining metadata about items in the cache

2017-12-04 Thread Raymond Wilson
che("CacheName").*withKeepBinary*(); IgniteQuery<...> qry = new ScanQuery<>( (k, v) -> { ... }; qry.*setLocal*(true); QueryCursor<Cache.Entry<BO, BO> cur = cache.query( ); ... }); } On Mon, Dec 4, 2017 at 1:33 AM, R

RE: Did the default page size change between Ignite 2.2 and 2.3

2017-12-02 Thread Raymond Wilson
BTW, is there a way of determining what the page size of a persistent store is before opening it? *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com] *Sent:* Friday, December 1, 2017 4:36 PM *To:* 'user@ignite.apache.org' <user@ignite.apache.org> *Subject:* RE: Did the default pag

Re: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-27 Thread Raymond Wilson
If I use local continuous queries can't I omit the remote filter and just use the local delivery handler for the continuous query? Sent from my iPhone > On 28/04/2018, at 9:43 AM, vkulichenko wrote: > > Raymond, > > It sounds like you want to run certain

Re: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-27 Thread Raymond Wilson
Val, Are the interceptors invoked in the affinity co-located context of the item? The help is a little unclear on that. Thanks, Raymond. Sent from my iPhone > On 28/04/2018, at 12:12 PM, vkulichenko wrote: > > Raymond, > > If you go with approach I described

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-28 Thread Raymond Wilson
Further investigation shows CacheInterceptor is not a part of the Ignite.NET API. Is there a plan/ticket for this to be done? -Original Message- From: Raymond Wilson [mailto:raymond_wil...@trimble.com] Sent: Saturday, April 28, 2018 1:08 PM To: user@ignite.apache.org Subject: Re: Using

RE: Binary type has different fields error

2018-05-16 Thread Raymond Wilson
, I'm a bit out of the loop on this. Thanks, Pavel On Wed, May 16, 2018 at 7:06 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: I just changed a field in a class from a long to a Guid. The class in question is marked [Serializable] and is passed to Ignite compute functions as

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-05-15 Thread Raymond Wilson
:* Wednesday, May 16, 2018 3:45 AM *To:* user@ignite.apache.org *Cc:* d...@ignite.apache.org *Subject:* Re: Using a cache as an affinity co-located processing buffer in Ignite.Net Hi Raymond, Was this question answered? Sincerely, Dmitriy Pavlov вт, 1 мая 2018 г. в 0:20, Raymond Wilson

Re: Running Ignite/GridGain in AWS

2018-05-15 Thread Raymond Wilson
We are going to go a similar route in AWS (.Net client with .Net standard artifacts inning on Kubernetes). Currently we are looking at whether EBS or EFS is the best option, but the comments here are concerning re Ignite IO bandwidth on those services. What page size are you using? If you

RE: Binary type has different fields error

2018-05-21 Thread Raymond Wilson
8 12:51 AM *To:* user@ignite.apache.org *Subject:* Re: Binary type has different fields error Hello! Yes, they are stored under work/marshaller. Should empty this dir before restarting node. Regards, -- Ilya Kasnacheev 2018-05-16 10:22 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: T

RE: Binary type has different fields error

2018-05-23 Thread Raymond Wilson
objects from serializing. Regards, -- Ilya Kasnacheev 2018-05-23 14:31 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: Hi Ilya, I got to the bottom of it today. It is an error on my part. The Ignite exception is correct, but wasn’t initially obvious why it was complaining

RE: Binary type has different fields error

2018-05-23 Thread Raymond Wilson
Kasnacheev 2018-05-22 4:50 GMT+03:00 Raymond Wilson <raymond_wil...@trimble.com>: Hi Ilya, I found the folder and removed it. The issue still persist in a different context, with the exception below being thrown. This suggests PeerClassLoading is enabled, though I have not conf

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-05-17 Thread Raymond Wilson
Cache interceptors are not available in Ignite.NET and there are no plans to add them. On Tue, May 15, 2018 at 10:37 PM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Hi Dmitriy, My question regarding cache interceptor availability in .Net is still outstanding. I have workin

Binary type has different fields error

2018-05-15 Thread Raymond Wilson
I just changed a field in a class from a long to a Guid. The class in question is marked [Serializable] and is passed to Ignite compute functions as a part of an argument to the compute function and is not saved to the persistent store. When I run the modified code against an Ignite grid

RE: Continuous query - Exactly once based event across multiple nodes..

2018-06-07 Thread Raymond Wilson
I another possibility to create a continuous query per node in your node affinity set for the cache and have the continuous query return local values, like this: using (IContinuousQueryHandle> queryHandle = queueCache.QueryContinuous (qry: new ContinuousQuery< Key,

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-30 Thread Raymond Wilson
Cross posting to dev list for comment on cache interceptor availability on Ignite .Net client. -Original Message- From: Raymond Wilson [mailto:raymond_wil...@trimble.com] Sent: Saturday, April 28, 2018 10:35 PM To: 'user@ignite.apache.org' <user@ignite.apache.org> Subject: RE:

Re: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-26 Thread Raymond Wilson
Val, What I really want is a .Net implementation of the Ignite distributed queue, assuming co-located server processing can pull members off the queue that are co-located with that node (yeah, that's probably mangling queue semantics a bit, but not that badly :) ) I have previously made a POC

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-26 Thread Raymond Wilson
Val, Thanks for the feedback. Essentially what I really want in Ignite.Net version of the Ignite distributed Queue with local consumers on each server reading and processing just the elements that server is responsible for (which may be a slight mangling of queue semantics...) I have written a

RE: Using a cache as an affinity co-located processing buffer in Ignite.Net

2018-04-25 Thread Raymond Wilson
nal data) On Wed, Apr 25, 2018 at 8:40 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: Let’s say I do this, where I define a single class FileFilter that implements both remote filter interfaces (BTW, it seems weird that there are two different interfaces for essentially the sam

Creating multiple Ignite grids on same machine

2018-01-09 Thread Raymond Wilson
I’m trying out a proposal for an Ignite based topology that will have two grids. I am using Ignite.Net 2.3. One grid is responsible for processing inbound events, the second is responsible for servicing read requests against an immutable query-efficient projection of the data held in the first

.Net standard target for Ignite.Net client

2018-01-21 Thread Raymond Wilson
All, Are there any plans for porting the current Ignite.Net client to .Net Standard? Has anyone investigated how much effort there would be involved? We would like to use dockerised deployments and as we use a .Net development stack .Net Standard is our target platform for those deployments.

  1   2   >