Hi Dennis- As promised, I am following up with a concrete example (*function-example* [1]); yay!
Unfortunately, I have not documented this example yet (nor are any of the other examples in the *Contacts Application* SDG RI documented for that matter; 1 of my SDG goals in the *New Year*), but it demonstrates your UC and then some. Like DATAGEODE-68 [2], I found several other issues (argh!) prior to *SpringOne* Platform (S1P) 2017 when I was preparing my examples (i.e. [3] & [4]) for my talks. 1 of the more important issues include DATAGEODE-73 [5], which involves CQ and @EnableClusterConfiguration annotation. So, I am using a BUILD-SNAPSHOT for the *function-example* [1] to get this to work. This 2.0.3.BUILD-SNAPSHOT does not currently contain the fixes yet since I have not checked in, but will soon. However, the issues are fixed and know that DATAGEODE-68 [2], DATAGEODE-73 [5] along with several other issues are going to be included in the upcoming SDG 2.0.3.RELEASE (*Kay-SR3* service release), probably due in the new year now. I only mention this because I threw everything along with the kitchen sink into this *new* *function-example* [1], ;-). This example includes Entity-defined Regions [6], CQs [7] (with a CQ event handler [8]), a client-side Function execution [9] (enabled with [10]) along with a server-side Function implementation [11] (enabled by [12]), a GemFire Repository [13] (enabled with [14]) and enables cluster configuration push (using HTTP) from the client [15] to the *Spring Boot*-configured/bootstrapped Apache Geode cache server application [16]. Additionally, on the server, I have enabled both an embedded *Locator* [17] and *Manager* [18], which starts immediately. By enabling both a *Locator* and *Manager*, this allows me to connect to this *Spring Boot*-configured/bootstrapped Apache Geode cache server using *Gfsh*, simply by doing this... gfsh> connect Both the embedded *Locator* (10334) and *Manager* (1099) are running with their default ports, as is the *CacheServer* (40404). The *Spring Boot* cache client application will connect directly to the *CacheServer* on port 40404 (the default), but, of course, all of this is configurable. Because I am using HTTP for the client-side cluster configuration push, I need to have a full installation of Apache Geode when I run the *Spring Boot,* Apache Geode cache server application. Therefore, I set the GEODE_HOME environment variable to the installation directory of Apache Geode 1.2.1 (on my machine) in my IDE run profile. Of course, I do not need to absolutely do this either, since I also created a nested *Spring* *@Configuration* class in the server code to manually register the internal SDG "administrative" Functions [19] I explained previously. When you enable this *Spring* profile (by using the Java System property on the command-line: -Dspring.profiles.active=non-http-cluster-config), then you can remove the *useHttp* attribute in the @EnableClusterConfiguration annotation on the client [15], since then the client will rely on the Apache Geode server being a *Spring* (*Data Geode*) enabled server with these *Functions* registered, which is why the server also @EnableGemfireFunctions [12], that and to register the AddressGeocodingFunction implementation on the server [20], which is used by the application, and specifically the client to, well, geocode Addresses. To geocode addresses, I included a dependency [21] on the *caching-example* [22], which demonstrates using Apache Geode as a caching provider in *Spring's Cache Abstraction*. Clearly, geocoding is a good candidate (and example ;-) of caching. There were several application service components that I recycled for this example in my server *Function* implementation. Anyway, it all works as expected! Hooray! Of course, as I mentioned, I need to check in fixes for some of the issues I encountered before you will be able to run it. Otherwise, you will encounter some of the issues I uncovered for S1P. As I said, they are already resolved and I am currently on the commit path so that the fixes are included in SDG 2.0.3.RELEASE ( *Kay-SR3*). I will follow-up on this thread again when I have the fixes committed and when the SDG 2.0.3.BUILD-SNAPSHOT available repo.spring.io/libs-snapshot in should include the changes. Regarding... *> I also discovered a few minor typos in the documentation, where should I report those?* Please file a JIRA ticket in the DATAGEODE JIRA project [23]. If you'd like, you are also welcome to submit a PR [24] with the doc edits. Contributions are always welcomed and appreciated. I am going to follow up with 1 more thing in just a minute, :-) But, I hope this sample clears up the muddy waters a bit, along with showing you (and others) the power of these new annotations. Cheers! -John [1] https://github.com/jxblum/contacts-application/tree/master/function-example [2] https://jira.spring.io/browse/DATAGEODE-68 [3] https://github.com/jxblum/scaling-springbootapplications-realtime [4] https://github.com/jxblum/simplifying-apache-geode-with-spring-data [5] https://jira.spring.io/browse/DATAGEODE-73 [6] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L61-L63 [7] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L60 [8] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L94-L104 [9] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/functions/AddressGeocodingFunctionExecution.java#L35-L39 [10] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L64 [11] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/functions/AddressGeocodingFunction.java#L58-L84 [12] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L48 [13] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/repo/AddressRepository.java [14] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L65 [15] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L66 [16] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L44-L50 [17] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L47 [18] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L46 [19] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L76-L94 [20] https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L60-L63 [21] https://github.com/jxblum/contacts-application/blob/master/function-example/pom.xml#L63-L67 [22] https://github.com/jxblum/contacts-application/tree/master/caching-example [23] https://jira.spring.io/issues/?jql=project%20%3D%20DATAGEODE [24] https://github.com/spring-projects/spring-data-geode/pulls On Fri, Dec 15, 2017 at 1:07 PM, Dennis Dai <[email protected]> wrote: > Wow, thanks a lot for the detailed explanations and references! I was > slowly going through the documentation and trying out different things. > FYI, I did hit the bug you reported in DATAGEODE-68 along the way and > resolved it with your suggestions. I also discovered a few minor typos in > the documentation, where should I report those? > > Thanks, > Dennis > > > On Dec 15, 2017, at 12:07 PM, John Blum <[email protected]> wrote: > > Hi Dennis- > > I apologize for your troubles. I possibly need to refine the docs; I am > open to any suggestions here on how I might make it more clear. > > I sense there is possibly some confusion around how multiple annotations > work in tandem (e.g. @EnableEntityDefinedRegions with > @EnableClusterConfiguration along with using SDG's Function annotation > support, enabled with @EnableGemfireFunctions). Without seeing your > code/configuration, I am basing my understanding solely on your provided > description and the log messages. > > Let's start with @EnableClusterConfiguration. > > > 1) First, this annotation only sends *Region* and *Index* (both OQL and > *Lucene* Indexes are supported) schema object definitions from the client > to the server. Functions are not included in the cluster configuration > push from the client. > > *NOTE: Functions are not supported in the client to server cluster > configuration push using @EnableClusterConfiguration since this would > require the Function class definition(s) to be on the servers' classpath > when the Function(s) get created/registered, which would entail uploading a > JAR file. Additionally, Apache Geode does not recognize SDG POJO annotated > Function "implementations" [1] (i.e. with @GemfireFunction). See here [2] > for more details.* > > > 2) Second, the default behavior of @EnableClusterConfiguration when > "pushing" schema object definitions (i.e. only *Regions/Indexes*) from > the client to the server is to invoke "internal", "administrative" > Functions provided by SDG. E.g. to create *Regions* on the servers from > a client SDG uses the o.s.d.g.config.admin.functions.CreateRegionFunction [3]. > To create *Indexes* on the servers from a client SDG uses the > o.s.d.g.config.admin.functions.CreateIndexFunction [4]. Again, these are > "internal" SDG "administrative" Functions and the *default behavior* for @ > EnableClusterConfiguration when pushing schema object definitions from > the client to the server. > > Additionally, these internal SDG administrative Functions are not > automatically registered by SDG on the servers since the servers may not be > *Spring* enabled; e.g. they may have been started with *Gfsh* or via some > other approach (although, in your case, the servers are *Spring* enabled > since you bootstrapped the servers with *Spring Boot*; +1). > > Furthermore, these SDG provided Functions are not recommended for > production purposes either since their "actions" are not recorded by Apache > Geode's *Cluster Configuration Service* [5] (yet). Therefore a user must > register these Functions manually (although, I am already getting ideas > here). For instance, see my SDG integration test for > @EnableClusterConfiguration based configuration [6]. Specially, have a > look at the "forked" Apache Geode server (*Spring*) configuration here > [7] and here [8]. > > > 3) Third, the alternative to using these internal SDG administrative > Functions is to set the @EnableClusterConfiguration(*useHttp = true*), By > setting the useHttp annotation attribute to *true*, the SDG > @EnableClusterConfiguration push will use Apache Geode's *Management* > REST API to push the schema object definitions from the client to the > servers. > > This approach has several advantages. One, the internal SDG > administrative Functions are not used and therefore are not required to be > manually registered on the servers in the cluster. Two, by using HTTP to > push configuration from the client to the server, Apache Geode's *Cluster > Configuration Service* will record the "actions" sent from the client. > Thus, when you add additional members to the cluster, either with Spring > Boot, or event with *Gfsh*, they will have the same/consistent > configuration. If you shutdown the entire cluster and bring it back up, it > will retain the configuration from before; all compliments of Apache *Geode's > Cluster Configuration Service*., which SDG uses when useHttp is set to > *true*. > > However, using HTTP does require a full installation of Apache Geode to be > present on the system where the server is running. When using *Spring > Boot* to bootstrap your servers, you can point these *Spring Boot* > configured/bootstrapped servers to a full Apache Geode installation by > setting the GEODE_HOME environment variable to the installation directory. > > FYI, though the Management REST API is not publicly advertised (like the > "Developer REST API" [9]), it does exist because 1) I created it when I was > working the GemFire/Geode engineering team, and 2) it is the same > functionality used by *Gfsh* when you connect via HTTP, like so... > > gfsh> connect --use-http --url=http://localhost:7070/gemfire/v1 > > This "Management REST API" was created for the purposes of managing Apache > Geode clusters running in a Cloud environment (e.g. AWS, or GCP) so users > would not have to worry about firewalls when connecting *Gfsh* locally to > the Apache Geode Manager in the cluster running in the Cloud. SDG's > @EnableClusterConfiguration uses this vary Management REST API to carry > out its function when useHttp is set to *true*. > > > As for your application-defined Apache Geode Functions using SDG's > Function annotation support, these will be registered and executable (using > SDG's Function execution annotation support [10]) as you would expect. > > To make all of this more concrete, let me put a quick example together for > you showcasing your UC. > > I will follow up in a short bit once I have completed the example; stay > tuned. > > Cheers, > John > > > [1] https://docs.spring.io/spring-data/geode/docs/current/reference/html/# > function-implementation > [2] https://docs.spring.io/spring-data/geode/docs/ > current/reference/html/#_implementation_vs_execution > [3] https://docs.spring.io/spring-data/gemfire/docs/current/api/org/ > springframework/data/gemfire/config/admin/functions/ > CreateRegionFunction.html > [4] https://docs.spring.io/spring-data/gemfire/docs/current/api/org/ > springframework/data/gemfire/config/admin/functions/ > CreateIndexFunction.html > [5] http://geode.apache.org/docs/guide/12/configuring/ > cluster_config/gfsh_persist.html > [6] https://github.com/spring-projects/spring-data-geode/ > blob/2.0.2.RELEASE/src/test/java/org/springframework/data/ > gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java > [7] https://github.com/spring-projects/spring-data-geode/ > blob/2.0.2.RELEASE/src/test/java/org/springframework/data/ > gemfire/config/annotation/EnableClusterConfigurationInte > grationTests.java#L195-L197 > [8] https://github.com/spring-projects/spring-data-geode/ > blob/2.0.2.RELEASE/src/test/java/org/springframework/data/ > gemfire/config/annotation/EnableClusterConfigurationInte > grationTests.java#L207-L220 > [9] http://geode.apache.org/docs/guide/12/rest_apps/book_intro.html > [10] https://docs.spring.io/spring-data/geode/docs/ > current/reference/html/#function-execution > > > On Thu, Dec 14, 2017 at 5:35 PM, Dennis Dai <[email protected]> wrote: > >> Hello, >> >> I am running a Geode server instance from spring boot (with >> @CacheServerApplication annotation). Then on the client side (also spring >> boot applications with @ClientCacheApplication annotation), I am trying to >> define some regions (with @Region annotation). Together with >> @EnableEntityDefinedRegions and @EnableClusterConfiguration on the client >> side and @EnableGemfireFuctions on the server side, I was hoping the >> regions would be automatically created on the server. But when I ran the >> client app, I am getting an error saying: >> >> org.apache.geode.cache.client.ServerOperationException: remote server on >> 192.168.1.100(SpringBasedCacheClientApplication:50794:loner) >> :55570:4310ba57:SpringBasedCacheClientApplication: The function is not >> registered for function id CreateRegionFunction >> >> On the server side, I got a similar error: >> >> 18:11:40.636 [ServerConnection on port 40405 Thread 1] WARN >> org.apache.geode.internal.cache.tier.sockets.BaseCommand - Server >> connection from [identity(192.168.1.100(Spring >> BasedCacheClientApplication:50794:loner):55570:4310ba57:Sp >> ringBasedCacheClientApplication,connection=1; port=55570]: The function >> is not registered for function id CreateRegionFunction >> >> From @EnableGemfireFunctions API doc: >> >> Enables GemFire annotated Function implementations. Causes the container >> to discover any beans that are annotated with {code}@GemfireFunction{code}, >> wrap them in a PojoFunctionWrapper >> <https://docs.spring.io/spring-data/geode/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html>, >> and register them with the cache. >> >> I thought that is all I need to get the spring data geode’s functions >> (e.g., CreateRegionFunction) registered on the server side. What am I >> missing here? >> >> Thanks, >> Dennis >> > > > > -- > -John > john.blum10101 (skype) > > > -- -John john.blum10101 (skype)
