Speaking of Pagination... https://jira.spring.io/browse/SGF-524
NOTE: Sorting in SDG Repos is already supported. This is also planned for SD *Lovelace*. On Thu, Nov 30, 2017 at 1:37 PM, Wayne Lund <[email protected]> wrote: > John, > > Oddly enough one of my use cases was for a client using Grails before we > spun that out and the Grails engineering team said that GORM support for > Gemfire had gone into mothballs for just this lack of functionality. The > generated controllers relied on streaming and paginated data, which is a > common requirement against data stores. I’ll go back and resurrect that > project and upgrade the gemfire-plugin to the Grails 3.0 when it comes. > I’ll just need to understand how we’re implementing it. > > Thanks, > > Wayne > > *Wayne Lund* > Advisory Platform Architect > 916.296.1893 <(916)%20296-1893> > [email protected] > > > <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> > > <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> > > On Nov 30, 2017, at 1:16 PM, John Blum <[email protected]> wrote: > > Wayne- > > The following 2 things are upcoming and coming for the next SD Release > Train (Lovelace), for both Spring Data for Apache Geode, and Spring Data > GemFire... > > https://jira.spring.io/browse/SGF-452 - Function Executions from a SD > Repository > > https://jira.spring.io/browse/SGF-377 - which is essentially Reactive > Repository support with CQ-based query methods, like this... > > Flux<Customer> customers = customerRepository.findByLastName("doe"); > > ... > > Yes, much more to come! > > > On Thu, Nov 30, 2017 at 1:05 PM, Wayne Lund <[email protected]> wrote: > >> I have wanted this from the early days of my using gemfire. Keep >> pushing. >> >> *Wayne Lund* >> Advisory Platform Architect >> 916.296.1893 <(916)%20296-1893> >> [email protected] >> >> <PastedGraphic-1.tiff> >> <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> >> >> <https://springoneplatform.io/?utm_source=employee-email-footer&utm_medium=email-footer-link&utm_campaign=employee-gmail> >> >> On Nov 30, 2017, at 12:51 PM, Amit Pandey <[email protected]> >> wrote: >> >> Thanks John..this is helpful and yes a fully reactive result collector >> from Geode will be awesome...I do think for large data sets this a must. >> >> On Fri, Dec 1, 2017 at 2:05 AM, John Blum <[email protected]> wrote: >> >>> 1 last thing... >>> >>> If you run the PeerCacheFunctionExecutionResultStreamingIntegrationTests >>> test >>> class, you will see the effects of Function Stream Result processing in you >>> IDE (or on the command-line). >>> >>> Enjoy! >>> -j >>> >>> >>> On Thu, Nov 30, 2017 at 12:31 PM, John Blum <[email protected]> wrote: >>> >>>> Hi Amit, all- >>>> >>>> 2 things... >>>> >>>> >>>> 1. Have a look at... DATAGEODE-36 - "*Add Async, Reactive and >>>> Streaming capabilities to the Function Annotation support.*" [1] >>>> >>>> I already started experimenting with Streaming Function results here >>>> [2]. Essentially the only thing preventing Apache Geode from Streaming >>>> results now is the default o.a.g.cache.execute.ResultCollector >>>> implementation (i.e. o.a.g.internal.cache.execute.D >>>> efaultResultCollector [3]) provided by Apache Geode OOTB. >>>> >>>> >>>> 2. *Dan is correct* about the batching functionality provided in SDG! >>>> >>>> You can see how it work by looking at the >>>> o.s.d.g.function.BatchingResultSender class [4], which is used by the >>>> o.s.d.g.function.PojoFunctionWrapper class [5] (which is how the >>>> entire Function "implementation" annotated POJO methods work, BTW! Well, >>>> that class and some *Spring (Data Geode)* infrastructure to enable it, >>>> ;-). >>>> >>>> >>>> Hope this helps! >>>> >>>> -John >>>> >>>> [1] https://jira.spring.io/browse/DATAGEODE-36 >>>> [2] https://github.com/jxblum/spring-gemfire-tests/blob/mast >>>> er/src/test/java/org/spring/data/gemfire/cache/PeerCacheFunc >>>> tionExecutionResultStreamingIntegrationTests.java >>>> [3] https://github.com/apache/geode/blob/rel/v1.3.0/geode-core/s >>>> rc/main/java/org/apache/geode/internal/cache/execute/Default >>>> ResultCollector.java >>>> [4] https://github.com/spring-projects/spring-data-gemfire/b >>>> lob/2.0.2.RELEASE/src/main/java/org/springframework/data/gem >>>> fire/function/BatchingResultSender.java >>>> [5] https://github.com/spring-projects/spring-data-gemfire/b >>>> lob/2.0.2.RELEASE/src/main/java/org/springframework/data/gem >>>> fire/function/PojoFunctionWrapper.java#L143-L148 >>>> >>>> >>>> On Thu, Nov 30, 2017 at 7:59 AM, Anthony Baker <[email protected]> >>>> wrote: >>>> >>>>> FWIW, I really like this suggestion. There is some interaction with >>>>> the `isHA()` behavior but I would love to have a reactive Function that >>>>> returns a mono / flux [1]. >>>>> >>>>> Anthony >>>>> >>>>> [1] https://spring.io/blog/2016/04/19/understanding-reactive-types >>>>> >>>>> >>>>> On Nov 28, 2017, at 10:23 PM, Amit Pandey <[email protected]> >>>>> wrote: >>>>> >>>>> Hey Guys, >>>>> >>>>> I filed this JIRA ticket :- https://issues.apache.org/j >>>>> ira/browse/GEODE-4027 >>>>> >>>>> Regards >>>>> >>>>> On Wed, Nov 29, 2017 at 2:01 AM, Amit Pandey < >>>>> [email protected]> wrote: >>>>> >>>>>> Sure. >>>>>> >>>>>> On Wed, Nov 29, 2017 at 1:43 AM, Swapnil Bawaskar < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Amit, >>>>>>> Can you please file a JIRA? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> On Tue, Nov 28, 2017 at 12:02 PM Amit Pandey < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hey Udo, >>>>>>>> >>>>>>>> if I was confident I wouldn't have griped about the non >>>>>>>> availability. I think though that it will be a good addition. Because >>>>>>>> for >>>>>>>> large data collections it's not enough to batch it from the server >>>>>>>> side. >>>>>>>> >>>>>>>> Without a streaming result collector its futile >>>>>>>> >>>>>>>> On Wed, Nov 29, 2017 at 1:05 AM, Udo Kohlmeyer < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi there Amit. >>>>>>>>> >>>>>>>>> Geode does also work for PRs.... We actually look forward to the >>>>>>>>> community contributing to the product... :) >>>>>>>>> >>>>>>>>> But on a more serious note... a streaming results collector does >>>>>>>>> require more thought and potentially significant (positive) changes >>>>>>>>> to the >>>>>>>>> system behavior. >>>>>>>>> >>>>>>>>> --Udo >>>>>>>>> >>>>>>>>> On Tue, Nov 28, 2017 at 11:21 AM, Amit Pandey < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Thanks Dan. >>>>>>>>>> >>>>>>>>>> A Streaming Result Collector in Geode is needed now . This will >>>>>>>>>> be great addition. I know we can make it, but its pretty error prone >>>>>>>>>> to do >>>>>>>>>> it >>>>>>>>>> >>>>>>>>>> On Wed, Nov 29, 2017 at 12:05 AM, Dan Smith <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Amit, >>>>>>>>>>> >>>>>>>>>>> Looking at the code, it looks like if your annotated method >>>>>>>>>>> returns an array, or an Iteratable (like a List), and you have >>>>>>>>>>> batchSize >>>>>>>>>>> set, then spring will send your data in batches back to the caller >>>>>>>>>>> of the >>>>>>>>>>> function. Underneath the covers is it using geode's ResultSender and >>>>>>>>>>> passing the batches to the ResultSender sendResult method. >>>>>>>>>>> >>>>>>>>>>> So it looks like this would reduce the amount of data that gets >>>>>>>>>>> sent to the client in a single message. >>>>>>>>>>> >>>>>>>>>>> -Dan >>>>>>>>>>> >>>>>>>>>>> On Sun, Nov 26, 2017 at 9:11 PM, Amit Pandey < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi John and everyone, >>>>>>>>>>>> >>>>>>>>>>>> How does Spring Data Geode Function annotation batch size work? >>>>>>>>>>>> >>>>>>>>>>>> So if we set it up like >>>>>>>>>>>> @GemfireFunction(id="lotMapService",HA=true, >>>>>>>>>>>> batchSize =1000) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> what are the implications ? What happens if there is a million >>>>>>>>>>>> data to be streamed on the server as well as on the client side. >>>>>>>>>>>> >>>>>>>>>>>> Regards >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Kindest Regards >>>>>>>>> ----------------------------- >>>>>>>>> *Udo Kohlmeyer* | *Pivotal* >>>>>>>>> [email protected] >>>>>>>>> <http://www.gopivotal.com/> >>>>>>>>> www.pivotal.io >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> -John >>>> john.blum10101 (skype) >>>> >>> >>> >>> >>> -- >>> -John >>> john.blum10101 (skype) >>> >> >> >> > > > -- > -John > john.blum10101 (skype) > > > -- -John john.blum10101 (skype)
