Re: Transformer not called on every ScanQuery
Hi Guilherme, Thanks for the repro. It is a bug. I've create a ticket [1] [1] https://issues.apache.org/jira/browse/IGNITE-5804 On Fri, Jul 21, 2017 at 2:04 PM, Guilherme Melowrote: > Hello Andrei, upon further investigation it seems I have found a bug, the > scenario is this: when running a query from a client node, if the result is > bigger than the default, or set page size the objects returned are not > transformed, but rather org.apache.ignite.internal.processors.cache.query. > GridCacheQueryResponseEntry. > I found this while running 1.9 > below is a simple test I wrote to prove it, its not the most elegant way > to show it, but I did not have a lot of time. > > public class SimpleAlwaysTruePredicate implements > IgniteBiPredicate { > @Override > public boolean apply(String s, Integer integer) { > return true; > } > } > > > public class SimpleOnlyValueTransformer implements > IgniteClosure , Integer> { > @Override > public Integer apply(Cache.Entry stringIntegerEntry) { > return stringIntegerEntry.getValue(); > } > } > > > public static void main(String[] args) { > > Ignition.setClientMode(true); > Ignite ignite = Ignition.start(); > > IgniteCache cache = ignite.createCache("test"); > > System.out.println("Running a query with a transformer less than the > default page size"); > //less than the default page size, no problem > for (int i = 0; i < 1024; i++) { > cache.put(String.valueOf(i), i); > } > > List returned = cache.query(new ScanQuery (new > SimpleAlwaysTruePredicate()), > new SimpleOnlyValueTransformer()).getAll(); > > returned.forEach(i -> doNothing(i)); > cache.clear(); > cache.destroy(); > > cache = ignite.createCache("test"); > > System.out.println("Running a query with a transformer more than the > default page size"); > //one more than the default page size, throws exception > for (int i = 0; i < 1025; i++) { > cache.put(String.valueOf(i), i); > } > > returned = cache.query(new ScanQuery (new > SimpleAlwaysTruePredicate()), > new SimpleOnlyValueTransformer()).getAll(); > > try { > returned.forEach(i -> doNothing(i)); > > } catch (Exception e) { > System.out.println(e); > } > cache.clear(); > cache.destroy(); > //just to make sure: > System.out.println("Running a query with a default page size and one more > entry then it"); > cache = ignite.createCache("test"); > cache.put("one", 1); > cache.put("two", 2); > Query > query = new ScanQuery<>(new > SimpleAlwaysTruePredicate()); > query.setPageSize(1); > > returned = cache.query(query, > new SimpleOnlyValueTransformer()).getAll(); > > > try { > returned.forEach(i -> doNothing(i)); > > } catch (Exception e) { > System.out.println(e); > } > cache.clear(); > cache.destroy(); > Ignition.stop(true); > } > > public static void doNothing(Integer i) { > > } > > > Cheers, > > Guilherme Melo > www.gmelo.org > > On 20 July 2017 at 23:13, Andrey Mashenkov > wrote: > >> Hi Guilherme, >> >> What version of ignite do you use? >> Can you share a reproducer? >> >> On Thu, Jul 20, 2017 at 1:32 PM, Guilherme Melo >> wrote: >> >>> Hello, >>> Has anyone had issues with Transformers not being called on every >>> instance of the scanQuery? >>> only about 10% of the objects get transformed, the rest are added to the >>> list returned from cache.query() are IgniteBiTuple, not what was passed to >>> the Transformer. >>> this does not happen when running the query without the transformer. >>> >>> Thank you >>> >>> >> >> >> -- >> Best regards, >> Andrey V. Mashenkov >> > > -- Best regards, Andrey V. Mashenkov
Re: Transformer not called on every ScanQuery
Hello Andrei, upon further investigation it seems I have found a bug, the scenario is this: when running a query from a client node, if the result is bigger than the default, or set page size the objects returned are not transformed, but rather org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry. I found this while running 1.9 below is a simple test I wrote to prove it, its not the most elegant way to show it, but I did not have a lot of time. public class SimpleAlwaysTruePredicate implements IgniteBiPredicate{ @Override public boolean apply(String s, Integer integer) { return true; } } public class SimpleOnlyValueTransformer implements IgniteClosure , Integer> { @Override public Integer apply(Cache.Entry stringIntegerEntry) { return stringIntegerEntry.getValue(); } } public static void main(String[] args) { Ignition.setClientMode(true); Ignite ignite = Ignition.start(); IgniteCache cache = ignite.createCache("test"); System.out.println("Running a query with a transformer less than the default page size"); //less than the default page size, no problem for (int i = 0; i < 1024; i++) { cache.put(String.valueOf(i), i); } List returned = cache.query(new ScanQuery (new SimpleAlwaysTruePredicate()), new SimpleOnlyValueTransformer()).getAll(); returned.forEach(i -> doNothing(i)); cache.clear(); cache.destroy(); cache = ignite.createCache("test"); System.out.println("Running a query with a transformer more than the default page size"); //one more than the default page size, throws exception for (int i = 0; i < 1025; i++) { cache.put(String.valueOf(i), i); } returned = cache.query(new ScanQuery (new SimpleAlwaysTruePredicate()), new SimpleOnlyValueTransformer()).getAll(); try { returned.forEach(i -> doNothing(i)); } catch (Exception e) { System.out.println(e); } cache.clear(); cache.destroy(); //just to make sure: System.out.println("Running a query with a default page size and one more entry then it"); cache = ignite.createCache("test"); cache.put("one", 1); cache.put("two", 2); Query > query = new ScanQuery<>(new SimpleAlwaysTruePredicate()); query.setPageSize(1); returned = cache.query(query, new SimpleOnlyValueTransformer()).getAll(); try { returned.forEach(i -> doNothing(i)); } catch (Exception e) { System.out.println(e); } cache.clear(); cache.destroy(); Ignition.stop(true); } public static void doNothing(Integer i) { } Cheers, Guilherme Melo www.gmelo.org On 20 July 2017 at 23:13, Andrey Mashenkov wrote: > Hi Guilherme, > > What version of ignite do you use? > Can you share a reproducer? > > On Thu, Jul 20, 2017 at 1:32 PM, Guilherme Melo > wrote: > >> Hello, >> Has anyone had issues with Transformers not being called on every >> instance of the scanQuery? >> only about 10% of the objects get transformed, the rest are added to the >> list returned from cache.query() are IgniteBiTuple, not what was passed to >> the Transformer. >> this does not happen when running the query without the transformer. >> >> Thank you >> >> > > > -- > Best regards, > Andrey V. Mashenkov >
Re: Transformer not called on every ScanQuery
Hi Guilherme, What version of ignite do you use? Can you share a reproducer? On Thu, Jul 20, 2017 at 1:32 PM, Guilherme Melowrote: > Hello, > Has anyone had issues with Transformers not being called on every instance > of the scanQuery? > only about 10% of the objects get transformed, the rest are added to the > list returned from cache.query() are IgniteBiTuple, not what was passed to > the Transformer. > this does not happen when running the query without the transformer. > > Thank you > > -- Best regards, Andrey V. Mashenkov
Transformer not called on every ScanQuery
Hello, Has anyone had issues with Transformers not being called on every instance of the scanQuery? only about 10% of the objects get transformed, the rest are added to the list returned from cache.query() are IgniteBiTuple, not what was passed to the Transformer. this does not happen when running the query without the transformer. Thank you