Thanks Alexey.


This would certainly reduce the IO, but does still require all the data to
be read.



My use case is not really a production one: I want to iterate all items in
the cache to determine if the page size for persistency was suitable.
Reading all the data is not too painful, but a meta data scan would be much
faster, especially if spread across the cluster in your example below.



Raymond.



*From:* Alexey Kukushkin [mailto:[email protected]]
*Sent:* Monday, December 4, 2017 11:10 PM
*To:* [email protected]
*Subject:* Re: Obtaining metadata about items in the cache



Hi Raymond,



I do not think Ignite supports iterating other metadata but you could
minimise IO by:

   - collocated processing (analyse entries locally without sending them
   over the network)
   - working with binary object representation directly (without
   serialisation/deserialisation)

You could send you analysis job to each partition and then execute a local
scan query that would work with binary objects. In the below code I
highlighted the affinityCall, withKeepBinary and setLocal methods you need
to use to achieve the above optimizations:



IgniteCompute compute = ignite.compute(ignite.cluster().forServers());

for (int i = 0; i < ignite.affinity("CacheName").partitions(); ++i) {



    compute.*affinityRun*(Collections.singletonList("CacheName"), i, () -> {



        IgniteCache<BinaryObject, BinaryObject> cache =
ignite.cache("CacheName").*withKeepBinary*();



        IgniteQuery<...> qry = new ScanQuery<>( (k, v) -> { ... };

        qry.*setLocal*(true);



        QueryCursor<Cache.Entry<BO, BO> cur = cache.query( );

       ...



    });



}











On Mon, Dec 4, 2017 at 1:33 AM, Raymond Wilson <[email protected]>
wrote:

Hi,



I’d like to be able to scan all the items in a cache where all I am
interested in is the cache key and other metadata about the cached item
(such as its size).



I can do this now by running a cache query that simple reads out all the
cache items, but this is a lot of IO when I don’t care about the content of
the items themselves.



Does anyone here do this?



Thanks,

Raymond.







-- 

Best regards,

Alexey

Reply via email to