RE: Re: Ignite.Net configuration for storing large cache data

Wed, 13 Nov 2019 02:34:45 -0800

Thin client does not store data and is not a part of a cluster, instead, you can think of it as an interface of accessing and manipulating your data.

Please, note, that you can not configure these properties for IgniteClientConfiguration.

The data region configuration makes sense only for server nodes.

 

From: Sudhir Patil
Sent: Wednesday, November 13, 2019 1:17 PM
To: user@ignite.apache.org
Subject: Re: Ignite.Net configuration for storing large cache data

 

Outside this question, such large data will not work with thin client???

 

Regards,

Sudhir

On Wednesday, November 13, 2019, Pavel Tupitsyn <ptupit...@apache.org> wrote:

Yes, region sizes are in bytes (thanks for pointing this out, I'll update the XMLDoc)

Yes, MaxSize is the right property. Docs say it should not be less than 10 mb.

Here is an example of C# config with multiple regions:

var cfg = new IgniteConfiguration
{
    DataStorageConfiguration = new DataStorageConfiguration
    {
        DefaultDataRegionConfiguration = new DataRegionConfiguration
        {
            MaxSize = 16L * 1024 * 1024 * 1024,
            Name = "default_region"
        },
        DataRegionConfigurations = new[]
        {
            new DataRegionConfiguration
            {
                MaxSize = 8L * 1024 * 1024 * 1024,
                Name = "custom_region"
            }
        }
    },
    CacheConfiguration = new []
    {
        new CacheConfiguration
        {
            Name = "cache_in_default_region"
        },
        new CacheConfiguration
        {
            Name = "cache_in_custom_region",
            DataRegionName = "custom_region"
        },
    }
};

var ignite = Ignition.Start(cfg);
 
As a result, in the Ignite log we can see:
[12:27:23] Data Regions Configured:
[12:27:23]   ^-- default_region [initSize=256.0 MiB, maxSize=16.0 GiB, persistence=false]
[12:27:23]   ^-- custom_region [initSize=256.0 MiB, maxSize=8.0 GiB, persistence=false]

 

On Wed, Nov 13, 2019 at 11:25 AM Sudhir Patil <patilsudhi...@gmail.com> wrote:

Thanks Pavel. 

1) i will look into this.

 

2) ok.

 

3) DefaultDataRegionConfiguration - 

What is default unit used  for different properties to set size values ? Is it byte??

 

For - By default, only one region is configured with max size set to 20% of available RAM.

 

What is the way to set max size ? MaxSize property? But documentation says it can be set maximum to 10 mb ??

 

How to set multiple regions?

 

Regards

Sudhir

On Wednesday, November 13, 2019, Pavel Tupitsyn <ptupit...@apache.org> wrote:

Hi Sudhir,

 

There are 3 things to consider:

 

1) Capacity planning - how much memory do you need for your data?

Depending on type, 30-50 properties can take very different amount of memory to store.

- n+4+1 for strings, where n is size in utf8 encoding

- 4+1 bytes for int

- 8+1 bytes for long

and so on

(every field takes extra byte for type code)

 

 

2) JVM settings

Ignite does not store your data in Java Heap, but some memory is still required for processing

 

 

3) Ignite DataRegionConfiguration

Ignite stores cache data in so-called Data Regions, in unmanaged heap (aka "offheap").

By default, only one region is configured with max size set to 20% of available RAM.

You may want to increase this by changing IgniteConfiguration.DataStorageConfiguration.DefaultDataRegionConfiguration

 

 

On Wed, Nov 13, 2019 at 4:13 AM Sudhir Patil <patilsudhi...@gmail.com> wrote:

Hi All,

 

What should be configuration changes required for storing large data e.g. 10 million records of custom class objects having around 30- 50 properties?

 

I am using basic default configuration for Ignite.net.

JvmOptions used are like

-Djava.net.preferIPv4Stack=true

-Xms512m

-XX:+UseG1GC

-XX:+DisabkeExplicitGC

 

Regards,

Sudhir

 



--
Thanks & Regards,
Sudhir Patil,
+91 9881095647.



--
Thanks & Regards,
Sudhir Patil,
+91 9881095647.



--
Thanks & Regards,
Sudhir Patil,
+91 9881095647.

 

Reply via email to