Hi Dennis, all- Alright, 1 more update...
I filed DATAGEODE-76 [1], which is an "improvement" to automatically register SDG's internal, "administrative" *Functions* (e.g. o.s.d.g.config.admin.functions.CreateRegionFunction [2]) when using SDG's Annotation configuration support to configure and bootstrap Apache Geode server's with *Spring* (*Data Geode* + *Spring Boot*) [3]. This is particularly useful since the @EnableClusterConfiguration annotation, *useHttp* attributes is disabled (i.e. *false*) by default, primarily because it requires a full installation of Apache Geode on the server. I may change this default in the future, but for now, I will just automatically register the SDG administrative Functions. This is already in progress, so this improvement will also make its way into SDG 2.0.3.RELEASE (*Kay-SR3*). Let me know if you have any questions/feedback. Thanks, John [1] https://jira.spring.io/browse/DATAGEODE-76 [2] https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html [3] https://docs.spring.io/spring-data/geode/docs/current/reference/html/#bootstrap-annotation-config-client-server-applications On Fri, Dec 15, 2017 at 7:13 PM, John Blum <[email protected]> wrote: > Hi Dennis- > > As promised, I am following up with a concrete example (*function-example* > [1]); yay! > > Unfortunately, I have not documented this example yet (nor are any of the > other examples in the *Contacts Application* SDG RI documented for that > matter; 1 of my SDG goals in the *New Year*), but it demonstrates your UC > and then some. > > Like DATAGEODE-68 [2], I found several other issues (argh!) prior to > *SpringOne* Platform (S1P) 2017 when I was preparing my examples (i.e. > [3] & [4]) for my talks. 1 of the more important issues include > DATAGEODE-73 [5], which involves CQ and @EnableClusterConfiguration > annotation. So, I am using a BUILD-SNAPSHOT for the *function-example* > [1] to get this to work. This 2.0.3.BUILD-SNAPSHOT does not currently > contain the fixes yet since I have not checked in, but will soon. However, > the issues are fixed and know that DATAGEODE-68 [2], DATAGEODE-73 [5] along > with several other issues are going to be included in the upcoming SDG > 2.0.3.RELEASE (*Kay-SR3* service release), probably due in the new year > now. > > I only mention this because I threw everything along with the kitchen sink > into this *new* *function-example* [1], ;-). > > This example includes Entity-defined Regions [6], CQs [7] (with a CQ event > handler [8]), a client-side Function execution [9] (enabled with [10]) > along with a server-side Function implementation [11] (enabled by [12]), a > GemFire Repository [13] (enabled with [14]) and enables cluster > configuration push (using HTTP) from the client [15] to the *Spring > Boot*-configured/bootstrapped > Apache Geode cache server application [16]. Additionally, on the server, I > have enabled both an embedded *Locator* [17] and *Manager* [18], which > starts immediately. By enabling both a *Locator* and *Manager*, this > allows me to connect to this *Spring Boot*-configured/bootstrapped Apache > Geode cache server using *Gfsh*, simply by doing this... > > gfsh> connect > > Both the embedded *Locator* (10334) and *Manager* (1099) are running with > their default ports, as is the *CacheServer* (40404). The *Spring Boot* > cache client application will connect directly to the *CacheServer* on > port 40404 (the default), but, of course, all of this is configurable. > > Because I am using HTTP for the client-side cluster configuration push, I > need to have a full installation of Apache Geode when I run the *Spring > Boot,* Apache Geode cache server application. Therefore, I set the > GEODE_HOME environment variable to the installation directory of Apache > Geode 1.2.1 (on my machine) in my IDE run profile. > > Of course, I do not need to absolutely do this either, since I also > created a nested *Spring* *@Configuration* class in the server code to > manually register the internal SDG "administrative" Functions [19] I > explained previously. When you enable this *Spring* profile (by using > the Java System property on the command-line: > -Dspring.profiles.active=non-http-cluster-config), then you can remove > the *useHttp* attribute in the @EnableClusterConfiguration annotation on > the client [15], since then the client will rely on the Apache Geode server > being a *Spring* (*Data Geode*) enabled server with these *Functions* > registered, which is why the server also @EnableGemfireFunctions [12], > that and to register the AddressGeocodingFunction implementation on the > server [20], which is used by the application, and specifically the client > to, well, geocode Addresses. > > To geocode addresses, I included a dependency [21] on the > *caching-example* [22], which demonstrates using Apache Geode as a > caching provider in *Spring's Cache Abstraction*. Clearly, geocoding is > a good candidate (and example ;-) of caching. There were several > application service components that I recycled for this example in my > server *Function* implementation. > > Anyway, it all works as expected! Hooray! Of course, as I mentioned, I > need to check in fixes for some of the issues I encountered before you will > be able to run it. Otherwise, you will encounter some of the issues I > uncovered for S1P. As I said, they are already resolved and I am currently > on the commit path so that the fixes are included in SDG 2.0.3.RELEASE ( > *Kay-SR3*). > > I will follow-up on this thread again when I have the fixes committed and > when the SDG 2.0.3.BUILD-SNAPSHOT available repo.spring.io/libs-snapshot > in should include the changes. > > Regarding... > > *> I also discovered a few minor typos in the documentation, where should > I report those?* > > Please file a JIRA ticket in the DATAGEODE JIRA project [23]. If you'd > like, you are also welcome to submit a PR [24] with the doc edits. > Contributions are always welcomed and appreciated. > > I am going to follow up with 1 more thing in just a minute, :-) > > But, I hope this sample clears up the muddy waters a bit, along with > showing you (and others) the power of these new annotations. > > Cheers! > -John > > > [1] https://github.com/jxblum/contacts-application/tree/ > master/function-example > [2] https://jira.spring.io/browse/DATAGEODE-68 > [3] https://github.com/jxblum/scaling-springbootapplications-realtime > [4] https://github.com/jxblum/simplifying-apache-geode-with-spring-data > [5] https://jira.spring.io/browse/DATAGEODE-73 > [6] https://github.com/jxblum/contacts-application/blob/ > master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L61-L63 > [7] https://github.com/jxblum/contacts-application/blob/ > master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L60 > [8] https://github.com/jxblum/contacts-application/blob/ > master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L94-L104 > [9] https://github.com/jxblum/contacts-application/blob/ > master/function-example/src/main/java/example/app/client/functions/ > AddressGeocodingFunctionExecution.java#L35-L39 > [10] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L64 > [11] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/functions/ > AddressGeocodingFunction.java#L58-L84 > [12] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L48 > [13] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/ > client/repo/AddressRepository.java > [14] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L65 > [15] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/client/ > FunctionExampleClientApplication.java#L66 > [16] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L44-L50 > [17] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L47 > [18] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L46 > [19] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L76-L94 > [20] https://github.com/jxblum/contacts-application/ > blob/master/function-example/src/main/java/example/app/server/ > FunctionExampleServerApplication.java#L60-L63 > [21] https://github.com/jxblum/contacts-application/ > blob/master/function-example/pom.xml#L63-L67 > [22] https://github.com/jxblum/contacts-application/ > tree/master/caching-example > [23] https://jira.spring.io/issues/?jql=project%20%3D%20DATAGEODE > [24] https://github.com/spring-projects/spring-data-geode/pulls > > > On Fri, Dec 15, 2017 at 1:07 PM, Dennis Dai <[email protected]> wrote: > >> Wow, thanks a lot for the detailed explanations and references! I was >> slowly going through the documentation and trying out different things. >> FYI, I did hit the bug you reported in DATAGEODE-68 along the way and >> resolved it with your suggestions. I also discovered a few minor typos in >> the documentation, where should I report those? >> >> Thanks, >> Dennis >> >> >> On Dec 15, 2017, at 12:07 PM, John Blum <[email protected]> wrote: >> >> Hi Dennis- >> >> I apologize for your troubles. I possibly need to refine the docs; I am >> open to any suggestions here on how I might make it more clear. >> >> I sense there is possibly some confusion around how multiple annotations >> work in tandem (e.g. @EnableEntityDefinedRegions with >> @EnableClusterConfiguration along with using SDG's Function annotation >> support, enabled with @EnableGemfireFunctions). Without seeing your >> code/configuration, I am basing my understanding solely on your provided >> description and the log messages. >> >> Let's start with @EnableClusterConfiguration. >> >> >> 1) First, this annotation only sends *Region* and *Index* (both OQL and >> *Lucene* Indexes are supported) schema object definitions from the >> client to the server. Functions are not included in the cluster >> configuration push from the client. >> >> *NOTE: Functions are not supported in the client to server cluster >> configuration push using @EnableClusterConfiguration since this would >> require the Function class definition(s) to be on the servers' classpath >> when the Function(s) get created/registered, which would entail uploading a >> JAR file. Additionally, Apache Geode does not recognize SDG POJO annotated >> Function "implementations" [1] (i.e. with @GemfireFunction). See here [2] >> for more details.* >> >> >> 2) Second, the default behavior of @EnableClusterConfiguration when >> "pushing" schema object definitions (i.e. only *Regions/Indexes*) from >> the client to the server is to invoke "internal", "administrative" >> Functions provided by SDG. E.g. to create *Regions* on the servers from >> a client SDG uses the o.s.d.g.config.admin.functions.CreateRegionFunction >> [3]. >> To create *Indexes* on the servers from a client SDG uses the >> o.s.d.g.config.admin.functions.CreateIndexFunction [4]. Again, these >> are "internal" SDG "administrative" Functions and the *default behavior* >> for @EnableClusterConfiguration when pushing schema object definitions >> from the client to the server. >> >> Additionally, these internal SDG administrative Functions are not >> automatically registered by SDG on the servers since the servers may not be >> *Spring* enabled; e.g. they may have been started with *Gfsh* or via >> some other approach (although, in your case, the servers are *Spring* >> enabled since you bootstrapped the servers with *Spring Boot*; +1). >> >> Furthermore, these SDG provided Functions are not recommended for >> production purposes either since their "actions" are not recorded by Apache >> Geode's *Cluster Configuration Service* [5] (yet). Therefore a user >> must register these Functions manually (although, I am already getting >> ideas here). For instance, see my SDG integration test for >> @EnableClusterConfiguration based configuration [6]. Specially, have a >> look at the "forked" Apache Geode server (*Spring*) configuration here >> [7] and here [8]. >> >> >> 3) Third, the alternative to using these internal SDG administrative >> Functions is to set the @EnableClusterConfiguration(*useHttp = true*), >> By setting the useHttp annotation attribute to *true*, the SDG >> @EnableClusterConfiguration push will use Apache Geode's *Management* >> REST API to push the schema object definitions from the client to the >> servers. >> >> This approach has several advantages. One, the internal SDG >> administrative Functions are not used and therefore are not required to be >> manually registered on the servers in the cluster. Two, by using HTTP to >> push configuration from the client to the server, Apache Geode's *Cluster >> Configuration Service* will record the "actions" sent from the client. >> Thus, when you add additional members to the cluster, either with Spring >> Boot, or event with *Gfsh*, they will have the same/consistent >> configuration. If you shutdown the entire cluster and bring it back up, it >> will retain the configuration from before; all compliments of Apache *Geode's >> Cluster Configuration Service*., which SDG uses when useHttp is set to >> *true*. >> >> However, using HTTP does require a full installation of Apache Geode to >> be present on the system where the server is running. When using *Spring >> Boot* to bootstrap your servers, you can point these *Spring Boot* >> configured/bootstrapped servers to a full Apache Geode installation by >> setting the GEODE_HOME environment variable to the installation directory. >> >> FYI, though the Management REST API is not publicly advertised (like the >> "Developer REST API" [9]), it does exist because 1) I created it when I was >> working the GemFire/Geode engineering team, and 2) it is the same >> functionality used by *Gfsh* when you connect via HTTP, like so... >> >> gfsh> connect --use-http --url=http://localhost:7070/gemfire/v1 >> >> This "Management REST API" was created for the purposes of managing >> Apache Geode clusters running in a Cloud environment (e.g. AWS, or GCP) so >> users would not have to worry about firewalls when connecting *Gfsh* >> locally to the Apache Geode Manager in the cluster running in the Cloud. >> SDG's @EnableClusterConfiguration uses this vary Management REST API to >> carry out its function when useHttp is set to *true*. >> >> >> As for your application-defined Apache Geode Functions using SDG's >> Function annotation support, these will be registered and executable (using >> SDG's Function execution annotation support [10]) as you would expect. >> >> To make all of this more concrete, let me put a quick example together >> for you showcasing your UC. >> >> I will follow up in a short bit once I have completed the example; stay >> tuned. >> >> Cheers, >> John >> >> >> [1] https://docs.spring.io/spring-data/geode/docs/current/ >> reference/html/#function-implementation >> [2] https://docs.spring.io/spring-data/geode/docs/current/ >> reference/html/#_implementation_vs_execution >> [3] https://docs.spring.io/spring-data/gemfire/docs/current/ >> api/org/springframework/data/gemfire/config/admin/ >> functions/CreateRegionFunction.html >> [4] https://docs.spring.io/spring-data/gemfire/docs/current/ >> api/org/springframework/data/gemfire/config/admin/ >> functions/CreateIndexFunction.html >> [5] http://geode.apache.org/docs/guide/12/configuring/cluste >> r_config/gfsh_persist.html >> [6] https://github.com/spring-projects/spring-data-geode/blo >> b/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfi >> re/config/annotation/EnableClusterConfigurationIntegrationTests.java >> [7] https://github.com/spring-projects/spring-data-geode/blo >> b/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfi >> re/config/annotation/EnableClusterConfigurationIntegrationTe >> sts.java#L195-L197 >> [8] https://github.com/spring-projects/spring-data-geode/blo >> b/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfi >> re/config/annotation/EnableClusterConfigurationIntegrationTe >> sts.java#L207-L220 >> [9] http://geode.apache.org/docs/guide/12/rest_apps/book_intro.html >> [10] https://docs.spring.io/spring-data/geode/docs/current/ >> reference/html/#function-execution >> >> >> On Thu, Dec 14, 2017 at 5:35 PM, Dennis Dai <[email protected]> wrote: >> >>> Hello, >>> >>> I am running a Geode server instance from spring boot (with >>> @CacheServerApplication annotation). Then on the client side (also spring >>> boot applications with @ClientCacheApplication annotation), I am trying to >>> define some regions (with @Region annotation). Together with >>> @EnableEntityDefinedRegions and @EnableClusterConfiguration on the client >>> side and @EnableGemfireFuctions on the server side, I was hoping the >>> regions would be automatically created on the server. But when I ran the >>> client app, I am getting an error saying: >>> >>> org.apache.geode.cache.client.ServerOperationException: remote server >>> on 192.168.1.100(SpringBasedCacheClientApplication:50794:loner) >>> :55570:4310ba57:SpringBasedCacheClientApplication: The function is not >>> registered for function id CreateRegionFunction >>> >>> On the server side, I got a similar error: >>> >>> 18:11:40.636 [ServerConnection on port 40405 Thread 1] WARN >>> org.apache.geode.internal.cache.tier.sockets.BaseCommand - Server >>> connection from [identity(192.168.1.100(Spring >>> BasedCacheClientApplication:50794:loner):55570:4310ba57:Spri >>> ngBasedCacheClientApplication,connection=1; port=55570]: The function >>> is not registered for function id CreateRegionFunction >>> >>> From @EnableGemfireFunctions API doc: >>> >>> Enables GemFire annotated Function implementations. Causes the container >>> to discover any beans that are annotated with {code}@GemfireFunction{code}, >>> wrap them in a PojoFunctionWrapper >>> <https://docs.spring.io/spring-data/geode/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html>, >>> and register them with the cache. >>> >>> I thought that is all I need to get the spring data geode’s functions >>> (e.g., CreateRegionFunction) registered on the server side. What am I >>> missing here? >>> >>> Thanks, >>> Dennis >>> >> >> >> >> -- >> -John >> john.blum10101 (skype) >> >> >> > > > -- > -John > john.blum10101 (skype) > -- -John john.blum10101 (skype)
