John, This one : https://github.com/Pivotal-Open-Source-Hub/geode-demo-application
Its pivotal and as per my understanding thats why its not maintained, but porting this is to geode will be awesome Regards On Mon, Dec 12, 2016 at 4:26 AM, John Blum <[email protected]> wrote: > Hi Amit- > > To which Geode demo application are you referring? Something off of > geode.apache.org? > > Thanks, > John > > > On Sun, Dec 11, 2016 at 12:47 AM, Amit Pandey <[email protected]> > wrote: > >> Hey John many thanks. >> >> This is really helpful. Yes I was confused why there were no Write Behind >> examples and the one which talked of doing write behind seemed to be using >> Spring Async Executor mechanism. >> >> I am really looking forward to your new refactored demoes. >> >> If you get times kindly let some one rework the Geode demo application >> with the latest code or atleats provide me a guide to update to the latest >> Spring Data and Geode dist , because that one seemed really useful to me to >> get started on something. >> >> On Sun, Dec 11, 2016 at 6:32 AM, John Blum <[email protected]> wrote: >> >>> Amit- >>> >>> The example >>> <https://github.com/spring-projects/spring-gemfire-examples/tree/master/basic/write-through> >>> [1] >>> to which you are referring is a "*Write-Through*" example only. This >>> evident from the configuration >>> <https://github.com/spring-projects/spring-gemfire-examples/blob/master/basic/write-through/src/main/resources/cache-config.xml#L16-L20> >>> [2] >>> of the "*Product*" Region used to store the application Product >>> <https://github.com/spring-projects/spring-gemfire-examples/blob/master/spring-gemfire-examples-common/src/main/java/org/springframework/data/gemfire/examples/domain/Product.java> >>> [3] >>> data. >>> >>> Whenever CacheLoaders >>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/CacheLoader.html> >>> [4] >>> (to pull from an external DataSource) and CacheWriters >>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/CacheWriter.html> >>> [5] >>> (to write to an external DataSource) are in play, then you know this is >>> a *synchronous* operation between the cache and the external data >>> store, as explained here >>> <http://geode.apache.org/docs/guide/developing/outside_data_sources/chapter_overview.html> >>> [6]. >>> >>> In order to achieve asynchronous "*Write-Behind*" behavior to an >>> external data store, you need to use an AsyncEventQueue >>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueue.html> >>> [7] >>> (set on the Region) along with an AsyncEventListener >>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventListener.html> >>> [8] >>> registered on the AEQ. The Listener serves the same purpose as the >>> CacheWriter to write to the external DataSource, however does so in an >>> asynchronous, "write-behind" manner given the nature of an AEQ, as the AEQ >>> Listener Javadoc alludes to... >>> >>> "*A callback for events passing through the AsyncEventQueue to which >>> this listener is attached. Implementers of >>> interface AsyncEventListener process batches of AsyncEvent delivered by the >>> corresponding AsyncEventQueue.*" >>> >>> Of course, the AEQ Listener can do whatever it wants with the (async) >>> events, but like the CacheWriter, you can also use it to write-behind >>> to an external data store. >>> >>> The AEQ controls >>> <http://geode.apache.org/docs/guide/reference/topics/cache_xml.html#async-event-queue> >>> [9] >>> many things: the order of events (order-policy), and how many events >>> are in a batch (batch-size), the time interval (batch-time-interval) at >>> which time the Listener is called with the batch of events regardless of >>> whether the batch-size is reached even, whether the AEQ is durable ( >>> disk-store-ref/disk-synchronous), and so on and so forth. >>> >>> When Region data access operations occur (e.g. *puts*), then it writes >>> an AsyncEvent >>> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventListener.html> >>> [10] >>> to the AEQ. Eventually (again, either determined by batch-size or >>> batch-time-interval), the AEQ will trigger the Listener. All of this >>> is explained in more detail here >>> <http://geode.apache.org/docs/guide/developing/events/implementing_write_behind_event_handler.html> >>> [11]. >>> >>> Now, that we have laid down the foundation for understanding >>> "Write-Behind" and how this works in Apache Geode (as well as Pivotal >>> GemFire)... *how * >>> *to do this with Spring Data Geode (/GemFire)? *Simple! >>> >>> Assuming you are configuring your application and Geode components using >>> *Spring >>> (Data Geode*) XML, first you need to define your AEQ Listener... >>> >>> <bean id="ExampleQueueListener" class="example.ExternalDataSto >>> reAsyncEventListener"> >>> ... >>> </bean> >>> >>> Obviously, you could inject a JDBC DataSource and wrap it with >>> *Spring's* JdbcTemplate, or use a SD JPA Repository, or whatever you >>> want, to access your RDBMS of choice. >>> >>> You would code the AEQ Listener to perform a data access operation for >>> each AsyncEvent passed to the Listener to write to the underlying data >>> store. Or, if you weren't using the SD JPA Repository abstraction, rather >>> the *Spring* JdbcTemplate, you could perform a "batch" JDBC data access >>> operation to the underlying data store as well since the AEQ and associated >>> Listener deal in "batches" of events. Hint, hint, ;-) >>> >>> Next, you need to define your AEQ and register your custom Listener(s). >>> >>> <gfe:async-event-queue id="ExampleQueue" >>> batch-conflation-enabled="true" >>> batch-size="10" >>> batch-time-interval="3" >>> dispatcher-threads="4" >>> disk-store-ref="ExampleQueueDiskStore" >>> disk-synchronous="true" >>> maximum-queue-memory="50" >>> parallel="false" >>> persistent="true" >>> order-policy="KEY"> >>> <gfe:async-event-listener ref="ExampleQueueListener"/> >>> </gfe:async-event-queue> >>> >>> NOTE: I copied this configuration from one of the tests in the SDG test >>> suite. This test is just ensuring that "all" the configuration settings >>> (i.e. XML <gfe:async-event-queue> element attributes) are properly >>> parsed and configured on the resulting Geode/GemFire AEQ component created >>> by SDG. Therefore, this is an exhaustive list of attributes available for >>> you to use and control the behavior of the AEQ. It is NOT representative >>> of settings that are necessarily appropriate for your application. >>> >>> Also know I can register more than 1 AEQ Listener. In fact, I can >>> register as many AEQ Listeners as I like, all performing a different tasks >>> as required by my application. >>> >>> You may have noticed that the "disk-store-ref" and "disk-synchronous" >>> attributes were specified. This makes the AEQ "durable". Additionally, >>> you should keep in mind that the "disk-synchronous" attribute as >>> nothing to do with the asynchronous, write-behind nature of an AEQ/Listener >>> associated with the Region implementing asynchronous, "write-behind" >>> semantics. The disk-synchronous setting specifically controls how >>> events on the "queue" are stored to disk when the AEQ is durable. >>> >>> Finally, you associate the AEQ with your Region, like so... >>> >>> <gfe:replicated-region id="ExampleRegionWithAEQ" > >>> <gfe:async-event-queue-ref bean="ExampleQueue"/*>* >>> * ...* >>> </gfe:replicated-region> >>> >>> >>> Note, this entire set of configuration could have easily been achieved >>> using JavaConfig was well. >>> >>> Also note, XML configuration is very flexible in that both the Listener >>> and Queue could have been defined as "anonymous" *Spring* beans on the >>> Region. >>> >>> You are probably wondering why there is not an example of "Write-Behind" >>> in the Spring GemFire Examples >>> <https://github.com/spring-projects/spring-gemfire-examples> [12], and >>> you would be correct, there should be. However, I am in the process of >>> completely re-working these examples, all of them, to be *Spring Boot* >>> based along with JavaConfig based using my new Annotation configuration >>> model that will greatly simplify configuring both Pivotal GemFire and >>> Apache Geode. >>> >>> Anyway, I hope this helps. If you have additional questions, let me >>> know. >>> >>> Cheers! >>> -John >>> >>> [1] https://github.com/spring-projects/spring-gemfire-exampl >>> es/tree/master/basic/write-through >>> [2] https://github.com/spring-projects/spring-gemfire-exampl >>> es/blob/master/basic/write-through/src/main/resources/cache- >>> config.xml#L16-L20 >>> [3] https://github.com/spring-projects/spring-gemfire-exampl >>> es/blob/master/spring-gemfire-examples-common/src/main/java/ >>> org/springframework/data/gemfire/examples/domain/Product.java >>> [4] http://geode.apache.org/releases/latest/javadoc/org/apac >>> he/geode/cache/CacheLoader.html >>> [5] http://geode.apache.org/releases/latest/javadoc/org/apac >>> he/geode/cache/CacheWriter.html >>> [6] http://geode.apache.org/docs/guide/developing/outside_da >>> ta_sources/chapter_overview.html >>> [7] http://geode.apache.org/releases/latest/javadoc/org/apac >>> he/geode/cache/asyncqueue/AsyncEventQueue.html >>> [8] http://geode.apache.org/releases/latest/javadoc/org/apac >>> he/geode/cache/asyncqueue/AsyncEventListener.html >>> [9] http://geode.apache.org/docs/guide/reference/topics/cach >>> e_xml.html#async-event-queue >>> [10] http://geode.apache.org/releases/latest/javadoc/org/apa >>> che/geode/cache/asyncqueue/AsyncEventListener.html >>> [11] http://geode.apache.org/docs/guide/developing/events/im >>> plementing_write_behind_event_handler.html >>> >>> >>> On Sat, Dec 10, 2016 at 8:15 AM, Amit Pandey <[email protected]> >>> wrote: >>> >>>> Sorry to be pedantic. Thats the example which confused me most. That >>>> says a async profile is setup . >>>> >>>> " Spring Data JPA Repository is used to access the database. The >>>> 'async' command line argument enables the 'async' Spring profile which >>>> enables the asynchronous behavior in ProductDBWriter." >>>> >>>> What I am confused is , we need to set up event queues etc for backup, >>>> we can make them persistable ensuring no data loss, where are those done? >>>> >>>> Regards >>>> >>>> On Sat, Dec 10, 2016 at 9:04 PM, Nilkanth Patel < >>>> [email protected]> wrote: >>>> >>>>> Have a look at following if that helps! >>>>> >>>>> https://github.com/spring-projects/spring-gemfire-examples >>>>> >>>>> Nilkanth Patel. >>>>> >>>>> On Sat, Dec 10, 2016 at 7:24 PM, Amit Pandey < >>>>> [email protected]> wrote: >>>>> >>>>>> Thanks I understood this. >>>>>> >>>>>> I want to know how to do this with Spring Data geode? >>>>>> >>>>>> On Sat, Dec 10, 2016 at 6:24 PM, Avinash Dongre <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> This should help. >>>>>>> >>>>>>> http://geode.apache.org/docs/guide/developing/events/impleme >>>>>>> nting_write_behind_event_handler.html >>>>>>> >>>>>>> Thanks >>>>>>> Avinash >>>>>>> >>>>>>> >>>>>>> On Sat, Dec 10, 2016 at 4:22 PM, Amit Pandey < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hey Guys can anyone please provide a link? >>>>>>>> >>>>>>>> On Fri, Dec 9, 2016 at 8:24 PM, Amit Pandey < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi Guys, >>>>>>>>> >>>>>>>>> Is there any geode example of write behind with Spring Gemfire or >>>>>>>>> even without it using reliable event queue (which persists to disk ) ? >>>>>>>>> >>>>>>>>> Looking at this example what config changes will enable write >>>>>>>>> behind keeping write order and not write orders with a reliable event >>>>>>>>> queue. >>>>>>>>> >>>>>>>>> Regards >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> >>> -- >>> -John >>> 503-504-8657 >>> john.blum10101 (skype) >>> >> >> > > > -- > -John > 503-504-8657 > john.blum10101 (skype) >
