Fair comment, let's try to understand the problem domain better first. Regards,
Peter. ----- Original message ----- > Peter: > > Could you expand on “dynamic multiple IP endpoints for clients”? Are > you thinking of load balancing, failover, or what? > > In the case of failover, I’d tend to say “Just let the endpoint fail, > and then the application can lookup a new one that works” - I’ve tried > implementing magical behaviour in smart proxies, and never been happy > with the results. I would tend to say the same with leader election - > just have the elected leader register the service in Reggie. In the > case of a new leader, let the old one fail, then the clients can lookup > the new one. > > I’d like to understand better what you’re suggesting. > > > Cheers, > > Greg Trasuk > On Apr 10, 2015, at 9:06 AM, Peter <j...@zeus.net.au> wrote: > > > And if not, do you have any thoughts on leader election and how JERI > > server side endpoints might communicate state? > > > > JERI could use dynamic multiple IP endpoints for clients. > > > > Regards, > > > > Peter. > > > > ----- Original message ----- > > > Thanks Brian, very interesting. > > > > > > Regarding high availability and self healing, would you reccommend > > > River support using zookeeper? > > > > > > If so, how are you using it? > > > > > > Regards, > > > > > > Peter. > > > > > > > > > ----- Original message ----- > > > > We've built a scalable distributed graph database using river ( > > > > www.blazegraph.com, formally www.bigdata.com). There are > > > > actually two versions that use river. > > > > > > > > 1. High availability and self-healing based on low write > > > > replication. This uses zookeeper for the leader election. It would > > > > be nice if river supported these semantics in the base > > > > distribution. > > > > > > > > 2. Horizontally scaled using dynamic sharding. This implementation > > > > uses river to expose the shards on the data services, to > > > > distribute the evaluation of high level queries over the shards > > > > and services, etc. > > > > > > > > River has made it relatively easy to have services export RMI > > > > interfaces. > > > > > > > > Some possible issues to consider: > > > > > > > > a. better communications about the project (i.e., more marketing). > > > > I think that most of the technical documentation for the river > > > > ecosystem (other than javadoc) is pretty old. It might still be > > > > valid, but there is a perception of stagnation. Maybe a weekly > > > > blog post, project of the month, etc. > > > > b. bindings for other languages (yes, this is not that easy). > > > > c. leader election semantics and similar interesting patterns in > > > > the base platform. > > > > d. design patterns for building scalable applications on river (it > > > > is really not that easy to get started if I remember back to when > > > > I first used the platform in the mid 2000s.) > > > > e. good tutorials for non-multicast environments (EC2). > > > > f. good tutorials for security, including living with firewalls and > > > > constraining ports. > > > > g. maybe a json interface to reggie so applications can easily see > > > > what's in the various registrars? > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > > > > > > > > ---- > > > > Bryan Thompson > > > > Chief Scientist & Founder > > > > SYSTAP, LLC > > > > 4501 Tower Road > > > > Greensboro, NC 27410 > > > > br...@systap.com > > > > http://blazegraph.com > > > > http://blog.bigdata.com <http://bigdata.com> > > > > http://mapgraph.io > > > > > > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra > > > > high-performance graph database that supports both RDF/SPARQL and > > > > Tinkerpop/Blueprints APIs. MapGraph™ > > > > <http://www.systap.com/mapgraph> is our disruptive new technology > > > > to use GPUs to accelerate data-parallel graph analytics. > > > > > > > > CONFIDENTIALITY NOTICE: This email and its contents and > > > > attachments are for the sole use of the intended recipient(s) and > > > > are confidential or proprietary to SYSTAP. Any unauthorized > > > > review, use, disclosure, dissemination or copying of this email or > > > > its contents or attachments is prohibited. If you have received > > > > this communication in error, please notify the sender by reply > > > > email and permanently delete all copies of the email and its > > > > contents and attachments. > > > > > > > > On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk > > > > <tras...@stratuscom.com> wrote: > > > > > > > > > > > > > > Well, here’s what I’ve used Jini for: > > > > > > > > > > Desktop Java applications communicating with a back-end > > > > > implemented as a set of Jini services in “LAN” scope. In this > > > > > scenario, Jini has the following valuable features: > > > > > - Zero-configuration on the client side. The client is able > > > > > to use multicast discovery to find the lookup service(s) and > > > > > then lookup the service it wants to use. There is no need for > > > > > the client app to have a configured url to reach the back-end. > > > > > - If the backend needs to move to a different host, the clients > > > > > don’t need to be reconfigured. > > > > > - The client can export a service endpoint, even without > > > > > publishing it to the service registrar. It can use that > > > > > endpoint to receive event notifications. So the user > > > > > interface can be completely asynchronous. This is not easy to > > > > > do using a request/response protocol like http (i.e. SOAP or > > > > > RESTful services). > > > > > > > > > > Shared Data Store on a LAN. For example, one service scanned a > > > > > programmable logic controller for process data. Other clients > > > > > would display selected portions of that data. For instance, a > > > > > daemon would display the aforementioned process data on overhead > > > > > displays. Another daemon took the process data and logged it to > > > > > a database every eight hours or so. Apart from the > > > > > zero-configuration aspects mentioned above, Jini lets us avoid > > > > > setting up a central data store. You want to find a tag called > > > > > “/cell-006/smt1/placement-defects”? Simply query the lookup > > > > > service for services implementing the “SharedDataStore” > > > > > interface, then select the one that lists a service attribute > > > > > that matches the prefix of the tag you want, then subscribe to > > > > > change events for it. The entire idea of a distributed > > > > > key-value store, along with the Paxos-based consistency > > > > > algorithm and leader-elections that go along with it ——is simply > > > > > not necessary ! —. You just get the data from the source. > > > > > > > > > > Distributed computing framework, roughly based on data flow > > > > > architecture, using JavaSpaces in a leader/follower pattern. > > > > > Like any tuple-space system, the architecture has flexible > > > > > scaling of resources (just add more followers as required). > > > > > As well, you get automatically optimal dynamic load balancing > > > > > (since faster processors finish a packet of work faster, they > > > > > simply pull work out of the JavaSpace in proportion to their > > > > > speed. Being JavaSpaces, the strong typing and dynamic class > > > > > loading turns out to be useful in a number of ways, mainly that > > > > > you can distribute code to the followers without any extra > > > > > effort. > > > > > > > > > > All of the above had aspects of what has recently come to be > > > > > known as “micro-services architecture” (I’ve been doing this > > > > > kind of architecture for 30 years, but that’s a different story, > > > > > and the kids today won’t believe it anyway). In all cases, > > > > > the ability of Jini services to easily discover and coordinate > > > > > with other services, with very little static configuration makes > > > > > the system very flexible. > > > > > > > > > > Which leads to my thoughts for the future of River - service > > > > > integration in the cloud/data centre. I look at the > > > > > convolutions that people are going through to get service > > > > > discovery working in a Docker environment (e.g. > > > > > https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores), > > > > > and I think that Jini has solved this problem already. The > > > > > dynamic discovery and zero-configuration nature of Jini, not to > > > > > mention the inherent fault-tolerance that goes along with > > > > > leasing, etc, makes Jini perfectly suited to a > > > > > dynamically-scalable environment. We just haven’t made it > > > > > easy to get started. Also, in the past, people were often > > > > > left with the impression that Jini was too complex. I think > > > > > that people have come around to the idea that the problem-space > > > > > for distributed computing is complex, so the solution-space is > > > > > necessarily complex as well. > > > > > > > > > > Cheers, > > > > > > > > > > Greg Trasuk. > > > > > >
