Thanks again John for your great efforts! These are more than enough for me to try out in the next couple of days (or even weeks). I will also see if I can submit a PR for the doc edits.
Dennis > On Dec 15, 2017, at 8:37 PM, John Blum <[email protected]> wrote: > > Hi Dennis, all- > > Alright, 1 more update... > > I filed DATAGEODE-76 [1], which is an "improvement" to automatically register > SDG's internal, "administrative" Functions (e.g. > o.s.d.g.config.admin.functions.CreateRegionFunction [2]) when using SDG's > Annotation configuration support to configure and bootstrap Apache Geode > server's with Spring (Data Geode + Spring Boot) [3]. > > This is particularly useful since the @EnableClusterConfiguration annotation, > useHttp attributes is disabled (i.e. false) by default, primarily because it > requires a full installation of Apache Geode on the server. I may change > this default in the future, but for now, I will just automatically register > the SDG administrative Functions. > > This is already in progress, so this improvement will also make its way into > SDG 2.0.3.RELEASE (Kay-SR3). > > Let me know if you have any questions/feedback. > > Thanks, > John > > > [1] https://jira.spring.io/browse/DATAGEODE-76 > <https://jira.spring.io/browse/DATAGEODE-76> > [2] > https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html > > <https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html> > [3] > https://docs.spring.io/spring-data/geode/docs/current/reference/html/#bootstrap-annotation-config-client-server-applications > > <https://docs.spring.io/spring-data/geode/docs/current/reference/html/#bootstrap-annotation-config-client-server-applications> > > > On Fri, Dec 15, 2017 at 7:13 PM, John Blum <[email protected] > <mailto:[email protected]>> wrote: > Hi Dennis- > > As promised, I am following up with a concrete example (function-example > [1]); yay! > > Unfortunately, I have not documented this example yet (nor are any of the > other examples in the Contacts Application SDG RI documented for that matter; > 1 of my SDG goals in the New Year), but it demonstrates your UC and then some. > > Like DATAGEODE-68 [2], I found several other issues (argh!) prior to > SpringOne Platform (S1P) 2017 when I was preparing my examples (i.e. [3] & > [4]) for my talks. 1 of the more important issues include DATAGEODE-73 [5], > which involves CQ and @EnableClusterConfiguration annotation. So, I am using > a BUILD-SNAPSHOT for the function-example [1] to get this to work. This > 2.0.3.BUILD-SNAPSHOT does not currently contain the fixes yet since I have > not checked in, but will soon. However, the issues are fixed and know that > DATAGEODE-68 [2], DATAGEODE-73 [5] along with several other issues are going > to be included in the upcoming SDG 2.0.3.RELEASE (Kay-SR3 service release), > probably due in the new year now. > > I only mention this because I threw everything along with the kitchen sink > into this new function-example [1], ;-). > > This example includes Entity-defined Regions [6], CQs [7] (with a CQ event > handler [8]), a client-side Function execution [9] (enabled with [10]) along > with a server-side Function implementation [11] (enabled by [12]), a GemFire > Repository [13] (enabled with [14]) and enables cluster configuration push > (using HTTP) from the client [15] to the Spring Boot-configured/bootstrapped > Apache Geode cache server application [16]. Additionally, on the server, I > have enabled both an embedded Locator [17] and Manager [18], which starts > immediately. By enabling both a Locator and Manager, this allows me to > connect to this Spring Boot-configured/bootstrapped Apache Geode cache server > using Gfsh, simply by doing this... > > gfsh> connect > > Both the embedded Locator (10334) and Manager (1099) are running with their > default ports, as is the CacheServer (40404). The Spring Boot cache client > application will connect directly to the CacheServer on port 40404 (the > default), but, of course, all of this is configurable. > > Because I am using HTTP for the client-side cluster configuration push, I > need to have a full installation of Apache Geode when I run the Spring Boot, > Apache Geode cache server application. Therefore, I set the GEODE_HOME > environment variable to the installation directory of Apache Geode 1.2.1 (on > my machine) in my IDE run profile. > > Of course, I do not need to absolutely do this either, since I also created a > nested Spring @Configuration class in the server code to manually register > the internal SDG "administrative" Functions [19] I explained previously. > When you enable this Spring profile (by using the Java System property on the > command-line: -Dspring.profiles.active=non-http-cluster-config), then you can > remove the useHttp attribute in the @EnableClusterConfiguration annotation on > the client [15], since then the client will rely on the Apache Geode server > being a Spring (Data Geode) enabled server with these Functions registered, > which is why the server also @EnableGemfireFunctions [12], that and to > register the AddressGeocodingFunction implementation on the server [20], > which is used by the application, and specifically the client to, well, > geocode Addresses. > > To geocode addresses, I included a dependency [21] on the caching-example > [22], which demonstrates using Apache Geode as a caching provider in Spring's > Cache Abstraction. Clearly, geocoding is a good candidate (and example ;-) > of caching. There were several application service components that I > recycled for this example in my server Function implementation. > > Anyway, it all works as expected! Hooray! Of course, as I mentioned, I need > to check in fixes for some of the issues I encountered before you will be > able to run it. Otherwise, you will encounter some of the issues I uncovered > for S1P. As I said, they are already resolved and I am currently on the > commit path so that the fixes are included in SDG 2.0.3.RELEASE (Kay-SR3). > > I will follow-up on this thread again when I have the fixes committed and > when the SDG 2.0.3.BUILD-SNAPSHOT available repo.spring.io/libs-snapshot > <https://repo.spring.io/libs-snapshot> in should include the changes. > > Regarding... > > > I also discovered a few minor typos in the documentation, where should I > > report those? > > Please file a JIRA ticket in the DATAGEODE JIRA project [23]. If you'd like, > you are also welcome to submit a PR [24] with the doc edits. Contributions > are always welcomed and appreciated. > > I am going to follow up with 1 more thing in just a minute, :-) > > But, I hope this sample clears up the muddy waters a bit, along with showing > you (and others) the power of these new annotations. > > Cheers! > -John > > > [1] > https://github.com/jxblum/contacts-application/tree/master/function-example > <https://github.com/jxblum/contacts-application/tree/master/function-example> > [2] https://jira.spring.io/browse/DATAGEODE-68 > <https://jira.spring.io/browse/DATAGEODE-68> > [3] https://github.com/jxblum/scaling-springbootapplications-realtime > <https://github.com/jxblum/scaling-springbootapplications-realtime> > [4] https://github.com/jxblum/simplifying-apache-geode-with-spring-data > <https://github.com/jxblum/simplifying-apache-geode-with-spring-data> > [5] https://jira.spring.io/browse/DATAGEODE-73 > <https://jira.spring.io/browse/DATAGEODE-73> > [6] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L61-L63 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L61-L63> > [7] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L60 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L60> > [8] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L94-L104 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L94-L104> > [9] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/functions/AddressGeocodingFunctionExecution.java#L35-L39 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/functions/AddressGeocodingFunctionExecution.java#L35-L39> > [10] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L64 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L64> > [11] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/functions/AddressGeocodingFunction.java#L58-L84 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/functions/AddressGeocodingFunction.java#L58-L84> > [12] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L48 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L48> > [13] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/repo/AddressRepository.java > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/repo/AddressRepository.java> > [14] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L65 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L65> > [15] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L66 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/client/FunctionExampleClientApplication.java#L66> > [16] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L44-L50 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L44-L50> > [17] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L47 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L47> > [18] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L46 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L46> > [19] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L76-L94 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L76-L94> > [20] > https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L60-L63 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/src/main/java/example/app/server/FunctionExampleServerApplication.java#L60-L63> > [21] > https://github.com/jxblum/contacts-application/blob/master/function-example/pom.xml#L63-L67 > > <https://github.com/jxblum/contacts-application/blob/master/function-example/pom.xml#L63-L67> > [22] > https://github.com/jxblum/contacts-application/tree/master/caching-example > <https://github.com/jxblum/contacts-application/tree/master/caching-example> > [23] https://jira.spring.io/issues/?jql=project%20%3D%20DATAGEODE > <https://jira.spring.io/issues/?jql=project%20%3D%20DATAGEODE> > [24] https://github.com/spring-projects/spring-data-geode/pulls > <https://github.com/spring-projects/spring-data-geode/pulls> > > > On Fri, Dec 15, 2017 at 1:07 PM, Dennis Dai <[email protected] > <mailto:[email protected]>> wrote: > Wow, thanks a lot for the detailed explanations and references! I was slowly > going through the documentation and trying out different things. FYI, I did > hit the bug you reported in DATAGEODE-68 along the way and resolved it with > your suggestions. I also discovered a few minor typos in the documentation, > where should I report those? > > Thanks, > Dennis > > >> On Dec 15, 2017, at 12:07 PM, John Blum <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Dennis- >> >> I apologize for your troubles. I possibly need to refine the docs; I am open >> to any suggestions here on how I might make it more clear. >> >> I sense there is possibly some confusion around how multiple annotations >> work in tandem (e.g. @EnableEntityDefinedRegions with >> @EnableClusterConfiguration along with using SDG's Function annotation >> support, enabled with @EnableGemfireFunctions). Without seeing your >> code/configuration, I am basing my understanding solely on your provided >> description and the log messages. >> >> Let's start with @EnableClusterConfiguration. >> >> >> 1) First, this annotation only sends Region and Index (both OQL and Lucene >> Indexes are supported) schema object definitions from the client to the >> server. Functions are not included in the cluster configuration push from >> the client. >> >> NOTE: Functions are not supported in the client to server cluster >> configuration push using @EnableClusterConfiguration since this would >> require the Function class definition(s) to be on the servers' classpath >> when the Function(s) get created/registered, which would entail uploading a >> JAR file. Additionally, Apache Geode does not recognize SDG POJO annotated >> Function "implementations" [1] (i.e. with @GemfireFunction). See here [2] >> for more details. >> >> >> 2) Second, the default behavior of @EnableClusterConfiguration when >> "pushing" schema object definitions (i.e. only Regions/Indexes) from the >> client to the server is to invoke "internal", "administrative" Functions >> provided by SDG. E.g. to create Regions on the servers from a client SDG >> uses the o.s.d.g.config.admin.functions.CreateRegionFunction [3]. To create >> Indexes on the servers from a client SDG uses the >> o.s.d.g.config.admin.functions.CreateIndexFunction [4]. Again, these are >> "internal" SDG "administrative" Functions and the default behavior for >> @EnableClusterConfiguration when pushing schema object definitions from the >> client to the server. >> >> Additionally, these internal SDG administrative Functions are not >> automatically registered by SDG on the servers since the servers may not be >> Spring enabled; e.g. they may have been started with Gfsh or via some other >> approach (although, in your case, the servers are Spring enabled since you >> bootstrapped the servers with Spring Boot; +1). >> >> Furthermore, these SDG provided Functions are not recommended for production >> purposes either since their "actions" are not recorded by Apache Geode's >> Cluster Configuration Service [5] (yet). Therefore a user must register >> these Functions manually (although, I am already getting ideas here). For >> instance, see my SDG integration test for @EnableClusterConfiguration based >> configuration [6]. Specially, have a look at the "forked" Apache Geode >> server (Spring) configuration here [7] and here [8]. >> >> >> 3) Third, the alternative to using these internal SDG administrative >> Functions is to set the @EnableClusterConfiguration(useHttp = true), By >> setting the useHttp annotation attribute to true, the SDG >> @EnableClusterConfiguration push will use Apache Geode's Management REST API >> to push the schema object definitions from the client to the servers. >> >> This approach has several advantages. One, the internal SDG administrative >> Functions are not used and therefore are not required to be manually >> registered on the servers in the cluster. Two, by using HTTP to push >> configuration from the client to the server, Apache Geode's Cluster >> Configuration Service will record the "actions" sent from the client. Thus, >> when you add additional members to the cluster, either with Spring Boot, or >> event with Gfsh, they will have the same/consistent configuration. If you >> shutdown the entire cluster and bring it back up, it will retain the >> configuration from before; all compliments of Apache Geode's Cluster >> Configuration Service., which SDG uses when useHttp is set to true. >> >> However, using HTTP does require a full installation of Apache Geode to be >> present on the system where the server is running. When using Spring Boot >> to bootstrap your servers, you can point these Spring Boot >> configured/bootstrapped servers to a full Apache Geode installation by >> setting the GEODE_HOME environment variable to the installation directory. >> >> FYI, though the Management REST API is not publicly advertised (like the >> "Developer REST API" [9]), it does exist because 1) I created it when I was >> working the GemFire/Geode engineering team, and 2) it is the same >> functionality used by Gfsh when you connect via HTTP, like so... >> >> gfsh> connect --use-http --url=http://localhost:7070/gemfire/v1 >> <http://localhost:7070/gemfire/v1> >> >> This "Management REST API" was created for the purposes of managing Apache >> Geode clusters running in a Cloud environment (e.g. AWS, or GCP) so users >> would not have to worry about firewalls when connecting Gfsh locally to the >> Apache Geode Manager in the cluster running in the Cloud. SDG's >> @EnableClusterConfiguration uses this vary Management REST API to carry out >> its function when useHttp is set to true. >> >> >> As for your application-defined Apache Geode Functions using SDG's Function >> annotation support, these will be registered and executable (using SDG's >> Function execution annotation support [10]) as you would expect. >> >> To make all of this more concrete, let me put a quick example together for >> you showcasing your UC. >> >> I will follow up in a short bit once I have completed the example; stay >> tuned. >> >> Cheers, >> John >> >> >> [1] >> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-implementation >> >> <https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-implementation> >> [2] >> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#_implementation_vs_execution >> >> <https://docs.spring.io/spring-data/geode/docs/current/reference/html/#_implementation_vs_execution> >> [3] >> https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html >> >> <https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateRegionFunction.html> >> [4] >> https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateIndexFunction.html >> >> <https://docs.spring.io/spring-data/gemfire/docs/current/api/org/springframework/data/gemfire/config/admin/functions/CreateIndexFunction.html> >> [5] >> http://geode.apache.org/docs/guide/12/configuring/cluster_config/gfsh_persist.html >> >> <http://geode.apache.org/docs/guide/12/configuring/cluster_config/gfsh_persist.html> >> [6] >> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java >> >> <https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java> >> [7] >> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L195-L197 >> >> <https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L195-L197> >> [8] >> https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L207-L220 >> >> <https://github.com/spring-projects/spring-data-geode/blob/2.0.2.RELEASE/src/test/java/org/springframework/data/gemfire/config/annotation/EnableClusterConfigurationIntegrationTests.java#L207-L220> >> [9] http://geode.apache.org/docs/guide/12/rest_apps/book_intro.html >> <http://geode.apache.org/docs/guide/12/rest_apps/book_intro.html> >> [10] >> https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-execution >> >> <https://docs.spring.io/spring-data/geode/docs/current/reference/html/#function-execution> >> >> >> On Thu, Dec 14, 2017 at 5:35 PM, Dennis Dai <[email protected] >> <mailto:[email protected]>> wrote: >> Hello, >> >> I am running a Geode server instance from spring boot (with >> @CacheServerApplication annotation). Then on the client side (also spring >> boot applications with @ClientCacheApplication annotation), I am trying to >> define some regions (with @Region annotation). Together with >> @EnableEntityDefinedRegions and @EnableClusterConfiguration on the client >> side and @EnableGemfireFuctions on the server side, I was hoping the regions >> would be automatically created on the server. But when I ran the client app, >> I am getting an error saying: >> >> org.apache.geode.cache.client.ServerOperationException: remote server on >> 192.168.1.100(SpringBasedCacheClientApplication:50794:loner):55570:4310ba57:SpringBasedCacheClientApplication: >> The function is not registered for function id CreateRegionFunction >> >> On the server side, I got a similar error: >> >> 18:11:40.636 [ServerConnection on port 40405 Thread 1] WARN >> org.apache.geode.internal.cache.tier.sockets.BaseCommand - Server connection >> from >> [identity(192.168.1.100(SpringBasedCacheClientApplication:50794:loner):55570:4310ba57:SpringBasedCacheClientApplication,connection=1; >> port=55570]: The function is not registered for function id >> CreateRegionFunction >> >> From @EnableGemfireFunctions API doc: >> >> Enables GemFire annotated Function implementations. Causes the container to >> discover any beans that are annotated with {code}@GemfireFunction{code}, >> wrap them in a PojoFunctionWrapper >> <https://docs.spring.io/spring-data/geode/docs/current/api/org/springframework/data/gemfire/function/PojoFunctionWrapper.html>, >> and register them with the cache. >> >> I thought that is all I need to get the spring data geode’s functions (e.g., >> CreateRegionFunction) registered on the server side. What am I missing here? >> >> Thanks, >> Dennis >> >> >> >> -- >> -John >> john.blum10101 (skype) > > > > > -- > -John > john.blum10101 (skype) > > > > -- > -John > john.blum10101 (skype)
