I agree with John's points here -- and, by extension, Dan's points. :) On Tue, Apr 25, 2017 at 4:00 PM, John Blum <jb...@pivotal.io> wrote:
> > *Naming convention: Geode's region is a Cache in JSR-107, and Geode's > Cache is a CacheManager. I think it would be too confusing to change the > meaning of cache. Also, how do you document this given that Cache means > different things if you are talking JCache vs Geode.* > > Perhaps something similar to this > <http://docs.spring.io/spring/docs/current/spring-framework- > reference/htmlsingle/#cache-jsr-107-summary> > [1]. > Geode could employ a similar approach with relative ease, but with > terminology. > > Also, I think it is Geode that is quite unique in its naming/terminology, > e.g. a "cache" as Region. Oracle Coherence even names caches, Cache. > > IMO, users will be more familiar with the industry adopted terminology, > which was the basis for JSR-107 defined by the JCP*.* If there were the > makings of a Geode *JCache* API, it would certainly help ease users > transition as it would give them a familiar way to interact with/program to > Geode. > > +1 to Dans points. I too was mainly thinking that the initial > implementation of the Geode *JCache* API would (and should) mostly be a > wrapper around the existing API. This is not unlike *Spring Data Geode* > today. > > Of course, there are going to be things in *JCache* that have no Geode > equivalent, and require a new approach, and might even be costly to > implement. This is fine, we should not be trying to cram a round peg in a > square hole, plus what is the cost of not doing it right. Better not to do > anything at all; complete compliance is not a prerequisite to release early > and garner feedback as the API is being built, which will only help > prioritize the important bits that are needed sooner than later. > > Case in point, EntryProcessors are akin to a database trigger and I have > seen Geode/GemFire users try to emulate these concepts in Geode/GemFire > using a combination of CacheListener (sometimes,CacheWriter) despite the > warning > <http://geode.apache.org/docs/guide/11/developing/events/ > writing_callbacks_that_modify_the_cache.html> > [2]. > It may not be trivial to implement, but it would be nice to (eventually) > have an answer for, and why not a standard one at that. > > Finally, as for have an API around what *Gfsh* does... well, there is the > Management REST API (more like Web API, actually), not publicized. > However, that could also do with proper "Resource" representations of the > commands in order to make it more "RESTful". Second, I would not base any > implementation on parsing or properly formatting a command to, I presume, > pass to the CommandService > <http://geode.apache.org/releases/latest/javadoc/org/ > apache/geode/management/cli/CommandService.html> > [3], > as would be expected by *Gfsh*... YIKES! I realize Wes probably did not > have much of a choice given the lack of an API, or at least a consistent > approach. Many of the commands are implemented either by accessing the > Geode public API (usually on the remote side), invoking a Function or in > some cases using the corresponding Management MBean functionality. But > *Gfsh's* command syntax should absolutely be avoided; that is just a UI, a > DSL of sorts. > > -j > > > [1] > http://docs.spring.io/spring/docs/current/spring-framework- > reference/htmlsingle/#cache-jsr-107-summary > [2] > http://geode.apache.org/docs/guide/11/developing/events/ > writing_callbacks_that_modify_the_cache.html > > > On Tue, Apr 25, 2017 at 3:20 PM, Swapnil Bawaskar <sbawas...@pivotal.io> > wrote: > > > I had looked at the JCache in the past and here are some of the things I > > noted: > > > > Naming convention: Geode's region is a Cache in JSR-107, and Geode's > Cache > > is a CacheManager. I think it would be too confusing to change the > meaning > > of cache. Also, how do you document this given that Cache means different > > things if you are talking JCache vs Geode. > > > > The way to create a Cache in JSR-107 is: > > Cache<Integer, Date> cache = manager.createCache(cacheName, Configuration > > c); > > Where it is upto the implementation to extend Configuration. Given this, > > users will not be able to switch from an existing implementation to ours; > > will have to write new code specially Configuration, making callbacks > > serializable etc. > > > > JSR-107 will not be limited to the client. Server side callbacks like > > CacheLoader, CacheListener etc. will need handle on jsr-107 “cache”. > > > > JSR-107 supports features like an EntryProcessor, which is a function > > invoked atomically on an entry operation. We will have to make invasive > > changes to Geode to support this. > > > > Given these, I don't think supporting JSR-107 is trivial. > > > > On Tue, Apr 25, 2017 at 2:55 PM Dan Smith <dsm...@pivotal.io> wrote: > > > > > What transport are you planning on using? REST, or the current binary > > > protocol? Or is this just a wrapper around the existing java client > APIs? > > > > > > If this about creating a new API, I agree with what John is saying that > > we > > > need reduce the number of APIs we have to do the same thing in java. It > > > seems especially confusing if we end up with different APIs that > support > > > distinct features - like being able to create a region on the server > with > > > this new API but only being able to invoke a function with the old API. > > > > > > The other thing I think we need to avoid is being able to do things > from > > > the client (or gfsh, or xml, ...) that don't have a java API on the > > server. > > > I'm assuming your getOrCreateRegion region is supposed behave like the > > gfsh > > > command and update the cluster configuration? +++1 to Wes's suggestion > > have > > > a public API for executing these gfsh operations. > > > > > > I really think we need to work on having *one* authoritative public API > > > that contains everything that we support on the server. The protocols > we > > > support (REST, binary, ...) and the client drivers that use those > > protocols > > > should just be ways of accessing that API. The Java API on the server > > right > > > now the closest thing we have to this, but there are things you can do > > with > > > gfsh and things you can do with the current client that have no java > API > > > equivalent. I think we really need to fix that! > > > > > > Also, preferably we won't have to hand code a bunch of stuff in every > > > client driver and protocol whenever we add a new feature. For example > if > > > were to add a geode function to invoke each new API we add, that new > API > > > would be accessible from REST, gfsh, the java client etc. Later we > could > > > add syntatic sugar to make the new API prettier, but if we had a low > > level > > > API that automatically exposed all new features that would make adding > > new > > > features much less painful. > > > > > > I do like the idea of adding a reactive API. > > > > > > Supporting JSR-107 might be nice - but that could probably be written > > just > > > as a wrapper around the current API without too much work. I don't > think > > we > > > should do anything for JSR-107 other than provide a JSR-107 compatible > > > wrapper - if someone wants additional geode specific features they > should > > > drop into the existing API. > > > > > > I also like the idea of a smaller client jar and dependencies, but I > > think > > > there is a huge amount of room for improvement in our existing client > > just > > > by refactoring the code a bit more and shinking geode-core into a bare > > > minimum. > > > > > > -Dan > > > > > > On Mon, Apr 24, 2017 at 8:20 PM, Fred Krone <fkr...@pivotal.io> wrote: > > > > > > > That's great, Wes. I will take a look. Thanks. > > > > > > > > John -- good feedback to consider and I was hoping this would come > up. > > > > > > > > "In my mind, there really are only 2 approaches... *Spring* and > > > > non-*Spring*, > > > > or rather PCF and non-PCF, and since PCF is primarily based on Boot > > > (given > > > > Microservices/applications are the new concurrency), then I am really > > > > saying the same thing." > > > > > > > > This would be cover non spring and attempt to give the community > > > something > > > > that had the same standardized experience as JSR 107 -- starting with > > the > > > > Cache interface itself. Although we don't necessarily have to > > implement > > > > Cache, it's method signatures are essentially a (non spring) caching > > > > standard for Java developers at this point. We considered full > blown > > > JSR > > > > 107 implementation but thought it was too robust for the needs > > mentioned > > > > (that's not to say we couldn't get there incrementally). I think > those > > > > needs still exist open-source outside of PCF and don't cannibalize. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 24, 2017 at 7:44 PM, Real Wes <thereal...@outlook.com> > > > wrote: > > > > > > > > > > > > > > Here is a simple Java client _for enterprise use_ that I developed > > for > > > > > Geode and distributed to several enterprises. It has similarities > and > > > > > differences for your goal. This project creates both server and > > client > > > > > regions dynamically. It lists regions, alters regions… really > > anything > > > > > that GFSH can do. It differs in that it calls GFSH to do its work > > > rather > > > > > than a Java interface. You can think of this as a Java API on top > of > > > > GFSH. > > > > > > > > > > You can use this model and keep the similarities while adjusting > for > > > the > > > > > Java interface. > > > > > > > > > > Similarities > > > > > - Client uses SDG and complies with JSR-107 > > > > > [the Cache Manager is here: https://github.com/Pivotal- > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > cluster-management-client/src/main/java/com/geode/cache/manager ] > > > > > - After the server creates its region, client creates its region > > > > > [ see createRegions at: https://github.com/Pivotal- > > > > Data-Engineering/ > > > > > ClusterManagement/blob/master/cluster-management- > > > > > > > > client/src/main/java/com/geode/cache/manager/ > RegionCreator.java<https:// > > > > > github.com/Pivotal-Data-Engineering/ClusterManagement/ > > > > blob/master/cluster- > > > > > management-client/src/main/java/com/geode/cache/manager/ > > > > RegionCreator.java> > > > > > ] > > > > > > > > > > Differences > > > > > Server dynamically calls GFSH to execute commands > > > > > > > > > > How it works > > > > > - Whenever the client calls a region that does not exist, the > client > > > > looks > > > > > for a <region name>.properties file. If the properties file exists, > > the > > > > > region is created with the dynamic properties. > > > > > - If <region name>.properties does not exist, default region > > properties > > > > > are loaded and used to create the region. > > > > > - properties are formed into a GFSH string and passed to a server > > > > function > > > > > - The server function calls GFSH using internal calls. It calls the > > > > > GfshParser, executes the command, and then returns the parsed > > results. > > > > > [see https://github.com/Pivotal-Data-Engineering/ > > > > > ClusterManagement/blob/master/cluster-management-server/src/ > > > > > main/java/com/geode/gfsh/GfshCommand.java] > > > > > > > > > > Limitations > > > > > It uses internal calls to call GFSH. I have made requests to make > > this > > > a > > > > > stable public interface. Andrew Baker had a good idea to return > gfsh > > > > > results in JSON format with a new —results=json property to pass to > > > GFSH. > > > > > > > > > > Strengths > > > > > That is uses GFSH can be viewed as a strength to keep a consistent > > API, > > > > > whether Java or GFSH > > > > > > > > > > TESTS > > > > > You can start by running server tests at: > > https://github.com/Pivotal- > > > > > Data-Engineering/ClusterManagement/tree/master/ > > > > > cluster-management-server/src/test/java/com/geode/gfsh > > > > > > > > > > Client tests are at: https://github.com/Pivotal-Data-Engineering/ > > > > > ClusterManagement/tree/master/cluster-management-client/src/ > > > > > test/java/com/geode/management/client > > > > > > > > > > > > > > > Regards, > > > > > Wes Williams > > > > > > > > > > On Apr 24, 2017, at 6:51 PM, Kirk Lund <kl...@apache.org<mailto: > > klund > > > > > @apache.org>> wrote: > > > > > > > > > > A couple things I'd like to see: > > > > > > > > > > 1) completely new API that doesn't reuse the old API classes (or at > > > least > > > > > not the giant classes such as Cache and Region interfaces) > > > > > 2) separation of API and Impl so that users can compile their code > > > > against > > > > > a dedicated client API jar > > > > > > > > > > On Mon, Apr 24, 2017 at 3:03 PM, Fred Krone <fkr...@pivotal.io > > <mailto: > > > > fkro > > > > > n...@pivotal.io>> wrote: > > > > > > > > > > In an effort to improve Java developer ease of use for caching with > > > > Geode I > > > > > am looking for feedback on going forward with creating a Java > client. > > > > This > > > > > client will allow for server-side region creation and distributed > > data > > > > > caching. This would allow for a thin client that fits with > > > microservice > > > > > caching patterns and also abstracts a cleaner client-server > > experience > > > > > driven interface. > > > > > > > > > > Initially we were going to update the Region interface but were > > > concerned > > > > > with breaking existing applications. We also would like to provide > > > > Region > > > > > creation to a client application and so propose here solving both > of > > > > these > > > > > areas with a Java client. > > > > > > > > > > It would have new project repo for the client. > > > > > > > > > > It would provide new Region interface for clients. The specifics > of > > > the > > > > > API design are too lengthy for this conversation but implementation > > > will > > > > > resemble JSR 107 Cache interface. > > > > > > > > > > It would use the new security framework. > > > > > > > > > > > > > > > *An example*, > > > > > > > > > > The client application simply creates an instance of client and > > points > > > it > > > > > to the locator: > > > > > > > > > > > > > > > org.apache.geode.client.Client client = Client.create(locatorHost, > > > > > locatorPort); > > > > > > > > > > > > > > > Client has the following methods: > > > > > > > > > > package org.apache.geode.client; > > > > > > > > > > > > > > > public interface GeodeClient { > > > > > > > > > > /** > > > > > > > > > > * creates the region on the servers, or gets the region if it > > exits, > > > > > returns a PROXY region > > > > > > > > > > */ > > > > > > > > > > public Region getOrCreateRegion(RegionAttributes attributes, > String > > > > > name); > > > > > > > > > > > > > > > /** > > > > > > > > > > * Returns a PROXY region if the region exists on the server > > > > > > > > > > */ > > > > > > > > > > public Region getRegion(String name); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > MVP > > > > > > > > > > The smallest set of features to test this idea, learn and iterate, > > and > > > > get > > > > > the client into the communities hands for future iterations is: > > > > > > > > > > > > > > > Create a server side Region from a client > > > > > > > > > > Region interface has CRUD on par with javax.cache.Cache (from JSR > > 107) > > > > > > > > > > Calls are asynchronous -- futures > > > > > > > > > > > > > > > > > > > > Also would like feedback on which future functionality would be > most > > > > useful > > > > > from a thin client: > > > > > > > > > > Function execution > > > > > > > > > > Durable clients > > > > > > > > > > User defined serialization > > > > > > > > > > Register interest > > > > > > > > > > Queries > > > > > > > > > > CQ > > > > > > > > > > Near side caching > > > > > > > > > > Create disk stores from client > > > > > > > > > > Region group membership > > > > > > > > > > Client subscription load balancing > > > > > Transactions > > > > > > > > > > > > > > > Thanks, > > > > > -Fred > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > -John > john.blum10101 (skype) >
