Hi Amit, all- 2 things...
1. Have a look at... DATAGEODE-36 - "*Add Async, Reactive and Streaming capabilities to the Function Annotation support.*" [1] I already started experimenting with Streaming Function results here [2]. Essentially the only thing preventing Apache Geode from Streaming results now is the default o.a.g.cache.execute.ResultCollector implementation (i.e. o.a.g.internal.cache.execute.DefaultResultCollector [3]) provided by Apache Geode OOTB. 2. *Dan is correct* about the batching functionality provided in SDG! You can see how it work by looking at the o.s.d.g.function.BatchingResultSender class [4], which is used by the o.s.d.g.function.PojoFunctionWrapper class [5] (which is how the entire Function "implementation" annotated POJO methods work, BTW! Well, that class and some *Spring (Data Geode)* infrastructure to enable it, ;-). Hope this helps! -John [1] https://jira.spring.io/browse/DATAGEODE-36 [2] https://github.com/jxblum/spring-gemfire-tests/blob/master/src/test/java/org/spring/data/gemfire/cache/PeerCacheFunctionExecutionResultStreamingIntegrationTests.java [3] https://github.com/apache/geode/blob/rel/v1.3.0/geode-core/src/main/java/org/apache/geode/internal/cache/execute/DefaultResultCollector.java [4] https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/BatchingResultSender.java [5] https://github.com/spring-projects/spring-data-gemfire/blob/2.0.2.RELEASE/src/main/java/org/springframework/data/gemfire/function/PojoFunctionWrapper.java#L143-L148 On Thu, Nov 30, 2017 at 7:59 AM, Anthony Baker <[email protected]> wrote: > FWIW, I really like this suggestion. There is some interaction with the > `isHA()` behavior but I would love to have a reactive Function that returns > a mono / flux [1]. > > Anthony > > [1] https://spring.io/blog/2016/04/19/understanding-reactive-types > > > On Nov 28, 2017, at 10:23 PM, Amit Pandey <[email protected]> > wrote: > > Hey Guys, > > I filed this JIRA ticket :- https://issues.apache.org/ > jira/browse/GEODE-4027 > > Regards > > On Wed, Nov 29, 2017 at 2:01 AM, Amit Pandey <[email protected]> > wrote: > >> Sure. >> >> On Wed, Nov 29, 2017 at 1:43 AM, Swapnil Bawaskar <[email protected]> >> wrote: >> >>> Amit, >>> Can you please file a JIRA? >>> >>> Thanks! >>> >>> On Tue, Nov 28, 2017 at 12:02 PM Amit Pandey <[email protected]> >>> wrote: >>> >>>> Hey Udo, >>>> >>>> if I was confident I wouldn't have griped about the non availability. I >>>> think though that it will be a good addition. Because for large data >>>> collections it's not enough to batch it from the server side. >>>> >>>> Without a streaming result collector its futile >>>> >>>> On Wed, Nov 29, 2017 at 1:05 AM, Udo Kohlmeyer <[email protected]> >>>> wrote: >>>> >>>>> Hi there Amit. >>>>> >>>>> Geode does also work for PRs.... We actually look forward to the >>>>> community contributing to the product... :) >>>>> >>>>> But on a more serious note... a streaming results collector does >>>>> require more thought and potentially significant (positive) changes to the >>>>> system behavior. >>>>> >>>>> --Udo >>>>> >>>>> On Tue, Nov 28, 2017 at 11:21 AM, Amit Pandey < >>>>> [email protected]> wrote: >>>>> >>>>>> Thanks Dan. >>>>>> >>>>>> A Streaming Result Collector in Geode is needed now . This will be >>>>>> great addition. I know we can make it, but its pretty error prone to do >>>>>> it >>>>>> >>>>>> On Wed, Nov 29, 2017 at 12:05 AM, Dan Smith <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Amit, >>>>>>> >>>>>>> Looking at the code, it looks like if your annotated method returns >>>>>>> an array, or an Iteratable (like a List), and you have batchSize set, >>>>>>> then >>>>>>> spring will send your data in batches back to the caller of the >>>>>>> function. >>>>>>> Underneath the covers is it using geode's ResultSender and passing the >>>>>>> batches to the ResultSender sendResult method. >>>>>>> >>>>>>> So it looks like this would reduce the amount of data that gets sent >>>>>>> to the client in a single message. >>>>>>> >>>>>>> -Dan >>>>>>> >>>>>>> On Sun, Nov 26, 2017 at 9:11 PM, Amit Pandey < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi John and everyone, >>>>>>>> >>>>>>>> How does Spring Data Geode Function annotation batch size work? >>>>>>>> >>>>>>>> So if we set it up like @GemfireFunction(id="lotMapService",HA=true, >>>>>>>> batchSize =1000) >>>>>>>> >>>>>>>> >>>>>>>> what are the implications ? What happens if there is a million data >>>>>>>> to be streamed on the server as well as on the client side. >>>>>>>> >>>>>>>> Regards >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Kindest Regards >>>>> ----------------------------- >>>>> *Udo Kohlmeyer* | *Pivotal* >>>>> [email protected] >>>>> <http://www.gopivotal.com/> >>>>> www.pivotal.io >>>>> >>>> >>>> >> > > -- -John john.blum10101 (skype)
