Re: storing blob from 1 mb to 1 gb in apache ignite as persistent cache

[mcherkasov]

Hi,

I don't see any limitations from Ignite point of view, I think it will fit
your requirements.
However if were you, I would build some sample app to check this.
So if you will do this, my advice is to deliver code to data and do
processing locally: 
https://apacheignite.readme.io/docs/jcache#entryprocessor
https://apacheignite.readme.io/docs/affinity-collocation#ignitecompute-vs-entryprocessor
to avoid 1gb value moving by a network.

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

storing blob from 1 mb to 1 gb in apache ignite as persistent cache

[rhshriva]

We need a durable cache for storing blobs which are as follows 1. The blob
could be from 1 MB to 1 GB. 2. We do not have to index the blog document. 3.
The cache entry should be durable in case of node failure. So we need
replication and partitioning. 4. There should be a write-behind hook so that
we can use external data storage to write data. 5. Very High transaction. We
would like to run like 100+ nodes .

Can someone comment on requirement number 1. I am concerned if Apache Ignite
is right choice for the same.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Ignite Persistent Cache DotNet

[Hemasundara Rao]

Hi Team,
I am testing Ignite cache persistent configuration on version 2.30.
I am experiencing strange behavior and it is not consistent behavior.
Could you please suggest me how to configure and test persistent  behavior?


Thanks and Regards,
Hemasundar.

Re: Persistent Cache

[Kamal C]

Thanks Denis!

On Wed, May 4, 2016 at 5:55 PM, Denis Magda  wrote:

> Yes, it does. But it’s available as a part of paid enterprise version
> delivered by GridGain. It’s not a part of Apache Ignite.
>
> —
> Denis
>
> On May 4, 2016, at 3:12 PM, Kamal C  wrote:
>
> Denis,
>
> Is Local Recoverable CacheStore applicable for all the cache modes?
> (Local, Replicated and Partitioned)
>
> --Kamal
>
> On Wed, May 4, 2016 at 5:21 PM, Denis Magda  wrote:
>
>> Hi Kamal,
>>
>> Ignite doesn’t have an implementation of the CacheStore that stores cache
>> data in files on disk.
>> You should implement it on your own or use the one that is provided by
>> other vendors (i.e. GridGain delivers Local Store [1] as a part of it
>> enterprise product built on top of Ignite).
>>
>> [1] https://gridgain.readme.io/docs/local-recoverable-store
>>
>> —
>> Denis
>>
>> On May 4, 2016, at 2:47 PM, Kamal C  wrote:
>>
>> Hi,
>>
>>I have a requirement in which state of the caches should be preserved
>> during application restart.
>>
>> I've gone through the [1] link. But, I don't want to store my contents
>> in the database. How to store the contents of cache in the disk ?
>>
>> [1] https://apacheignite.readme.io/docs/persistent-store
>>
>> --Kamal
>>
>>
>>
>
>

Re: Persistent Cache

[Denis Magda]

Yes, it does. But it’s available as a part of paid enterprise version delivered 
by GridGain. It’s not a part of Apache Ignite.

—
Denis

> On May 4, 2016, at 3:12 PM, Kamal C  wrote:
> 
> Denis,
> 
> Is Local Recoverable CacheStore applicable for all the cache modes? (Local, 
> Replicated and Partitioned)
> 
> --Kamal
> 
> On Wed, May 4, 2016 at 5:21 PM, Denis Magda  > wrote:
> Hi Kamal,
> 
> Ignite doesn’t have an implementation of the CacheStore that stores cache 
> data in files on disk. 
> You should implement it on your own or use the one that is provided by other 
> vendors (i.e. GridGain delivers Local Store [1] as a part of it enterprise 
> product built on top of Ignite).
> 
> [1] https://gridgain.readme.io/docs/local-recoverable-store 
> 
> 
> —
> Denis
> 
>> On May 4, 2016, at 2:47 PM, Kamal C > > wrote:
>> 
>> Hi,
>> 
>>I have a requirement in which state of the caches should be preserved
>> during application restart.
>> 
>> I've gone through the [1] link. But, I don't want to store my contents
>> in the database. How to store the contents of cache in the disk ?
>> 
>> [1] https://apacheignite.readme.io/docs/persistent-store 
>> 
>> 
>> --Kamal
> 
> 

Re: Persistent Cache

[Kamal C]

Denis,

Is Local Recoverable CacheStore applicable for all the cache modes? (Local,
Replicated and Partitioned)

--Kamal

On Wed, May 4, 2016 at 5:21 PM, Denis Magda  wrote:

> Hi Kamal,
>
> Ignite doesn’t have an implementation of the CacheStore that stores cache
> data in files on disk.
> You should implement it on your own or use the one that is provided by
> other vendors (i.e. GridGain delivers Local Store [1] as a part of it
> enterprise product built on top of Ignite).
>
> [1] https://gridgain.readme.io/docs/local-recoverable-store
>
> —
> Denis
>
> On May 4, 2016, at 2:47 PM, Kamal C  wrote:
>
> Hi,
>
>I have a requirement in which state of the caches should be preserved
> during application restart.
>
> I've gone through the [1] link. But, I don't want to store my contents
> in the database. How to store the contents of cache in the disk ?
>
> [1] https://apacheignite.readme.io/docs/persistent-store
>
> --Kamal
>
>
>

Re: Persistent Cache

[Denis Magda]

Hi Kamal,

Ignite doesn’t have an implementation of the CacheStore that stores cache data 
in files on disk. 
You should implement it on your own or use the one that is provided by other 
vendors (i.e. GridGain delivers Local Store [1] as a part of it enterprise 
product built on top of Ignite).

[1] https://gridgain.readme.io/docs/local-recoverable-store 


—
Denis

> On May 4, 2016, at 2:47 PM, Kamal C  wrote:
> 
> Hi,
> 
>I have a requirement in which state of the caches should be preserved
> during application restart.
> 
> I've gone through the [1] link. But, I don't want to store my contents
> in the database. How to store the contents of cache in the disk ?
> 
> [1] https://apacheignite.readme.io/docs/persistent-store 
> 
> 
> --Kamal

Persistent Cache

[Kamal C]

Hi,

   I have a requirement in which state of the caches should be preserved
during application restart.

I've gone through the [1] link. But, I don't want to store my contents
in the database. How to store the contents of cache in the disk ?

[1] https://apacheignite.readme.io/docs/persistent-store

--Kamal

Distributed persistent cache store

[Erik Vanherck]

Hi All,

I gather that the current CacheStore is designed to be implemented by using 
some kind of centralised storage backend. However I need to store data for 
longer periods of time and a database is poorly suited for it (mainly small 
blobs) because pure write throughput matters, Hadoop HDFS massive overkill and 
too much latency (already tried that), and shared filesystems … well they 
present their own set of challenges and weird behaviour, safety and visibility 
guarantees (even something like glusterfs). 

So I thought about doing it easy, just store it in large sequential files on 
local disks for each node. This would work fine apart from having a need to 
ensure some kind of reliability by replicating the data since nodes may be lost.

I see that there are events broadcast for a PUT and REMOVE on a cache, so 
potentially I could hook into that and do some custom logic by using some hash 
and then sharding, but if there are better solutions or reasons why this would 
not work, I would love any suggestions.

Cheers,
Erik

Re: Distributed persistent cache store

[vkulichenko]

Hi Erik,

Generally, cache store is designed to be shared across all nodes. Dumping
data to local FS can lead to different data consistency issues and
definitely not an easy task.

If you want to persist your data, but avoid large latencies of DB, you can
use write-behind caching [1].

BTW, GridGain has functionality similar to what you describe in their
enterprise edition [1], but it's actually used mostly for recovery after
full cluster restart, but not for persistence.

[1]
https://apacheignite.readme.io/docs/persistent-store#write-behind-caching
[2] https://gridgain.readme.io/docs/local-recoverable-store

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Distributed-persistent-cache-store-tp2275p2285.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.