What a great answer. I think your test names are just too short though :) Wayne Lund Platform Architect 916.296.1893 [email protected] <mailto:[email protected]> www.pivotal.io <http://www.pivotal.io/> https://spring.io <https://spring.io/>
> On Dec 8, 2016, at 3:44 PM, John Blum <[email protected]> wrote: > > Hi Andrei- > > I don't have much time to give a complete explanation, but I do very similar > testing in Spring Data GemFire/Geode for both Pivotal GemFire and Apache > Geode. > > I also have done very similar things in Spring Session with the GemFire > support. > > Here are a few examples... > > 1. Most recently, I created a client/server test > <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java> > [1] where I test Geode's Security infrastructure/framework in my Contact > Application RI > <https://github.com/jxblum/contacts-application/tree/apache-geode> [2]. The > test "forks > <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L131-L141>" > [3] a GemFire Server for testing purposes (note: the test acts as the cache > client, of course). > > This example is unique since it showcases many of the new features in SDG > (like using and building on my new Annotation configuration model for SDG > GemFire/Geode applications, Spring Boot style, for both clients > <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L177-L182> > [12] and servers > <https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L196-L208> > [13]) that I am developing to greatly simplify the "getting started" > experience, even above and beyond the "Geode in 5 min > <https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutesGeodein5minutes>" > [4], which is well, cumbersome since it does not bode well for creating an > actual "automated", "client/server, integration" test. > > 2. The SDG test suite has many client/server integration tests, most notably > in the org.springframework.data.gemfire.client > <https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/client> > package [5]. > > 3. There are also some CQ related tests in > org.springframework.data.gemfire.listener > <https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/listener> > package [6]. > > 4. In fact, if you search the SDG test suite source for uses of ForkUtil > <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/ForkUtil.java> > [7] (old) and ProcessExecutor > <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/process/ProcessExecutor.java> > [8] (relatively new), you will see all the (integration) tests that fork > processes. > > Some tests like > GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests > <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java> > [9] even fork and coordinate > <https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java#L105-L108> > [10] (via Locator) multiple Geode Servers. > > 5. Additionally, you can see the integration test infrastructure support > <https://github.com/spring-projects/spring-session/tree/1.3.0.RC1/spring-session/src/integration-test/java/org/springframework/session/data/gemfire> > [11] in the Spring Session GemFire support. > > Unfortunately, I have learned a lot since then and need to really clean all > this up and standardize around a common test programming model for [Spring > Data] GemFire/Geode, which is already in the works and is based on several > things... first, my new Annotation configuration model (which will greatly > simplify boilerplate test setup/tearDown, ugh), Spring TestContext and > abstraction and JUnit 5. > > Anyway, even if you are not using Spring [Data GemFire/Geode], hopefully this > gives you some ideas of how to organize and create your own (integration) > tests. > > > Cheers! > John > > [1] > https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java > [2] https://github.com/jxblum/contacts-application/tree/apache-geode > [3] > https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L131-L141 > [4] > https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutesGeodein5minutes > [5] > https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/client > [6] > https://github.com/spring-projects/spring-data-gemfire/tree/master/src/test/java/org/springframework/data/gemfire/listener > [7] > https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/ForkUtil.java > [8] > https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/process/ProcessExecutor.java > [9] > https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java > [10] > https://github.com/spring-projects/spring-data-gemfire/blob/master/src/test/java/org/springframework/data/gemfire/GemfireTemplateQueriesOnGroupedPooledClientCacheRegionsIntegrationTests.java#L105-L108 > [11] > https://github.com/spring-projects/spring-session/tree/1.3.0.RC1/spring-session/src/integration-test/java/org/springframework/session/data/gemfire > [12] > https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L177-L182 > [13] > https://github.com/jxblum/contacts-application/blob/apache-geode/security-example/src/test/java/example/app/geode/security/GeodeSecurityIntegrationTests.java#L196-L208 > > > On Thu, Dec 8, 2016 at 3:38 PM, Udo Kohlmeyer <[email protected] > <mailto:[email protected]>> wrote: > Hi Andrei, > > #1 The reason why this restriction is there is because currently the Cache is > a singleton. I suppose the correct way of stating this is, legacy. There are > some JIRA's relating to changing all of this, but it will take a little more > time. But you are most welcome to join the cause to help us improve the Geode > product. :D > #2 Not really a follow up question here... But I imagine you'd want to know > how mimic this... A CacheListener is a "post-commit" event processor. Which > means, it will be invoke after the entry "commit". So in order to best mimic > similar behavior on a server you would have to implement a CacheListener (or > extend CacheListenerAdapter). Then you implement the afterCreate, > afterUpdate, afterInvalidate and afterDestroy methods. Most likely your code > in each of the methods would be a simple filter statement to determine if the > data entry matches your "query". > > Please note, you cannot use OQL or the Query engine here, so it is a little > more manual and labor intensive to write some code that could take an OQL > statement and convert it to a filter. > > Once your filter code returns with a true, you could invoke a callback > method for a registered processor object (look at the init() method on the > CacheListener > <https://cwiki.apache.org/confluence/display/GEODE/CacheWriter+and+CacheListener+Best+Practices>) > to trigger some other behavior. (there are many ways to skin a cat, so > please bear with me) > > #3 Correct. You'd have to imagine that a client is a "Server Lite". It does > not have redundancy or FT but you can still create regions on a client and > use them. So creating a region from a client is "harder". In order to do > something on the server (from a client), it would almost be best to use a > Function to invoke the behavior/action on the server(s). > > --Udo > > On 12/8/16 15:13, Andrei Sereda wrote: >> Hi Udo, >> >> Thanks for your reply. I looked at JUnit4BasicDUnitTest >> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/tests/JUnit4BasicDUnitTest.java> >> and it seems it is using RMI to create regions on the server cache (similar >> to Function concept you proposed). Some follow-ups: >> >> #1 : I would like to avoid writing infrastructure code around managing >> individual VMs. But it seems for our use-case (single client, one cache) >> there is no other way. Any reason why there is such restriction of distinct >> VMs for client/server ? >> >> #2 : Our code makes extensive use of Queries and CQs already, so I wan't be >> able to simulate this functionality with CacheListeners. >> >> #3 : We can't create region just on the client (via ClientRegionFactory >> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/client/ClientRegionFactory.html>) >> since put / query / CQ will not work together (have tried both PROXY / >> PROXY_CACHING shortcuts) "Create region" call must originate from client >> (before test runs) therefore one doesn't have access to Server Region >> Factory >> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/Cache.html#createRegionFactory-->. >> I will try the Function approach. >> >> Please let me know if above reasoning is correct. >> >> Regards, >> Andreo. >> >> >> On Thu, Dec 8, 2016 at 4:25 PM, Udo Kohlmeyer <[email protected] >> <mailto:[email protected]>> wrote: >> Hi there Andrei. >> >> #1: Clients and Servers cannot coexist within the same JVM. What has been >> done for testing within Geode, is that we use the concept of a VM >> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/VM.java>. >> Which allows us to start multiple servers/clients in the same JVM. You can >> review the JUnit4BasicDUnitTest >> <https://github.com/apache/geode/blob/develop/geode-core/src/test/java/org/apache/geode/test/dunit/tests/JUnit4BasicDUnitTest.java> >> to get an idea how we use it. >> >> #2: Correct. CQ's is a client-side concept. If you want similar behavior, >> you could try writing a custom CacheListener (or AsyncEventListener for a >> more batch oriented approach) if you need to trigger something on the >> server-side. >> >> #3: We do have a Java API that you could use to create Regions. Geode >> JavaDoc <http://geode.apache.org/releases/latest/javadoc/index.html>. You >> could also look at the Geode in 5min >> <https://cwiki.apache.org/confluence/display/GEODE/Index#Index-Geodein5minutesGeodein5minutes>, >> as the Java API creates a region on the Client. If you were to use that >> approach on the Server you would use Server Region Factory >> <http://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/Cache.html#createRegionFactory--> >> to create a region (in Java code) on the server. If you needed to trigger >> this from a client, you could always use a Function >> <http://geode.apache.org/docs/guide/developing/function_exec/chapter_overview.html> >> that could create a Region on the server. Once, the regions are create on >> the server(s) those regions will be usable for queries and cq's. >> >> --Udo >> >> >> On 12/8/16 10:28, Andrei Sereda wrote: >>> Hello, >>> >>> I'm trying to bootstrap my application with prefabricated geode (gemfire) >>> cache for simple tests. >>> >>> Basically what is necessary is to create a couple of regions, populate them >>> and run some queries (including CQ) on the top of existing data, perhaps >>> also validate transaction functionality. >>> >>> Ideally one should be able to re-create regions (recycle cache) multiple >>> times. This also, has to run independently (no external process >>> dependencies) as part of existing build >>> >>> I can't find an easy way to do so for the following reasons (please >>> correct me if I'm wrong) : >>> >>> 1) Client and Server caches can't coexists on the same JVM. Does it mean >>> one has to start a separate process for server ? >>> >>> 2) It seems that CQs can't be executed on server cache. >>> >>> 3) No easy way to create (non-local) regions programmatically on a remote >>> server cache (see (1)). They should be visible for queries / CQs. >>> Preferably avoiding starting a locator (another process) ? >>> >>> Please point me to some examples if they exist. >>> >>> Current version of product is gemfire v7 but looking at the code of geode >>> concepts remained the same. >>> >>> Thanks, >>> Andrei. >>> >>> >> >> > > > > > -- > -John > 503-504-8657 > john.blum10101 (skype)
