Re: cache.removeAll cache operation is slow

2019-01-22 Thread Prasad Bhalerao
The data I am removing from cache is colocated on the same node. So I think
exchange will happen only for updating the backup copies. Please correct me
if I am wrong.

I believe that remove all operation first acquires the lock on all input
keys and then removes the data. If I go for single remove, will it not
acquire lock per key per remove operation. Will it not slow down?

Will try your suggestions.



On Tue 22 Jan, 2019, 7:22 PM Denis Mekhanikov  Prasad,
>
> When you run a transaction, that involves many entries, the whole key set
> is sent between nodes multiple times.
> It also generates a lot of garbage, which cannot be released until the
> transaction is committed.
> It's better to remove values in small batches.
> So, try changing the batch size. It's possible, that singular removes will
> work faster.
>
> Denis
>
> вт, 22 янв. 2019 г. в 16:37, Prasad Bhalerao  >:
>
>> Hi,
>>
>> I am removing around 2 million entries from cache using cache.removeAll
>> api.
>> I removing entries in a batch of 100k.  To remove 2 million
>> entries ignite takes around 218 seconds.
>> This cache is transactiona and write synchronization mode is full sync.
>>
>> Is there any way to improve the performance of removeAll operation?
>>
>> Thanks,
>> Prasad
>>
>>
>>


Re: cache.removeAll cache operation is slow

2019-01-22 Thread Denis Mekhanikov
Prasad,

When you run a transaction, that involves many entries, the whole key set
is sent between nodes multiple times.
It also generates a lot of garbage, which cannot be released until the
transaction is committed.
It's better to remove values in small batches.
So, try changing the batch size. It's possible, that singular removes will
work faster.

Denis

вт, 22 янв. 2019 г. в 16:37, Prasad Bhalerao :

> Hi,
>
> I am removing around 2 million entries from cache using cache.removeAll
> api.
> I removing entries in a batch of 100k.  To remove 2 million entries ignite
> takes around 218 seconds.
> This cache is transactiona and write synchronization mode is full sync.
>
> Is there any way to improve the performance of removeAll operation?
>
> Thanks,
> Prasad
>
>
>


cache.removeAll cache operation is slow

2019-01-22 Thread Prasad Bhalerao
Hi,

I am removing around 2 million entries from cache using cache.removeAll api.
I removing entries in a batch of 100k.  To remove 2 million entries ignite
takes around 218 seconds.
This cache is transactiona and write synchronization mode is full sync.

Is there any way to improve the performance of removeAll operation?

Thanks,
Prasad