Peter, It seems that the a key issue with evolving APIs for river will be interoperability in a heterogeneous deployment environment. I would suggest that we do not break interoperability within a major revision (3.0, 4.0, etc.) and that we document changes that do break interoperability for major releases. And that we try for backwards compatibility where that makes sense, but use major releases to allow breaking changes.
Bryan ---- Bryan Thompson Chief Scientist & Founder SYSTAP, LLC 4501 Tower Road Greensboro, NC 27410 br...@systap.com http://blazegraph.com http://blog.bigdata.com <http://bigdata.com> http://mapgraph.io Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints APIs. Blazegraph is now available with GPU acceleration using our disruptive technology to accelerate data-parallel graph analytics and graph query. CONFIDENTIALITY NOTICE: This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments. On Fri, Sep 4, 2015 at 4:40 AM, Peter <j...@zeus.net.au> wrote: > Threats to development and collaboration (as I see it): > > 1. The Jini standards are sacrosanct. > 2. River is an implementation of the Jini Standards. > 3. River has no public api other than the Jini standards. > 4. Although public API can be improved in a backward compatible > manner, the mandatory Jini Standards should not be. > 5. The Jini standards are static; there is, at present, no process > for standards review or replacement. > > This is the message I’m receiving, is this the message we want to send > would be River developers? > > Sun Labs was isolated from the rest of Sun, to ensure developers were able > to innovate, how can River innovate now? > > Here’s an example of a problem with the Jini public api, there are many as > such, I’d like to fix, but for the sake of harmony on river-dev, I don't > discuss them, instead I document and work around them if I can, I only > discuss issues I can't work around: > > The following is an implementation comment from > org/apache/river/impl/lease/AbstractLeaseMap.java > /** > * AbstractLeaseMap is intended to work around some minor design warts in > the > * {@link Lease} interface: > * > * In the real world, when a Lease is renewed, a new Lease contract document > * is issued, however when an electronic Lease is renewed the Lease expiry > * date is changed and the record of the previous Lease is lost. Ideally the > * renew method would return a new Lease. > * > * Current Lease implementations rely on a {@link Uuid} to represents the > lease, > * the expiry date is not included the equals or hashCode calculations. For > this > * reason, two Lease objects, one expired and one valid, may be equal, this > * is undesirable. > * > * The Lease interface doesn't specify a contract for equals or hashCode, > * all Lease implementations are also mutable, previous implementations > * of {@link LeaseMap} used Leases as keys. > * > * AbstractLeaseMap uses only the {@link ID}, usually a {@link Uuid} > * provided by a Lease for internal map keys, if {@link ID} is not > implemented > * then the Lease itself is used as the key. > * > * Both Lease keys and Long values are actually stored internally as values > * referred to by ID keys, allowing Lease implementations to either not > override > * hashCode and equals object methods or allow implementations that more > * accurately model reality. > > Documentation from the Map interface states: > > Note: great care must be exercised if mutable objects are used as > map keys. The behavior of a map is not specified if the value of an > object is changed in a manner that affects equals comparisons while > the object is a key in the map. A special case of this prohibition > is that it is not permissible for a map to contain itself as a key. > While it is permissible for a map to contain itself as a value, > extreme caution is advised: the equals and hashCode methods are no > longer well defined on such a map. > > You can't tell me this is well designed, it might be standardised, but > it's also flawed. > > The first issue is, if I make an attempt to address this issue, there will > be strong resistance to doing so. Who remembers what happened when I tried > to create a Startable interface for starting services? It's now only an > implementation detail? It was a very frustrating discussion, one developer > hasn’t returned, yes I didn’t handle it well, at the time, I deliberately > drove that developer away out of frustration. > > But the outcome has not been beneficial, there were no winners, not only > did we lose a community member, even experienced developers are still > exporting from constructors, new users will read our examples and unsafely > export their services from within constructors and if they're using River > 3.0, they're going to experience problems, how's this going to help > adoption? River 3.0 can hammer a service with multiple threads, running at > native socket speeds, if there's a concurrency bug, River 3.0 will expose > it. With River 3.0, all hotspots are native methods. > > I think with River 3.0, we need a statement in the README file that says, > River 3.0 does not support exporting services from object constructors. > > It's not so much a question of is River 3.0 ready for release, but is the > world ready for River 3.0? > > Too much time is consumed debating and analysing, too little on > development, we are suffering from standards concensus paralysis. It's fair > to say that this has a negative effect on developer motivation. We spend > days arguing about something that takes 20 minutes to implement, meanwhile > development stalls and people complain of slipping release dates. > > During project incubation, we had a philosophy of doers decide, so if you > wanted to veto something you needed to implement something else that solved > the original issue. But now we have returned to a concensus model and we > lack the ability to make progress when we cannot achieve concensus. > > It sounds like Git could help us a lot with our development collaboration > problem. > > With Git, we could have an “Official Jini Standards release branch.” For > those of us that bump into the limitations of the Jini Standards API, we > need another branch: “Jini standards with extensions”. > > In this extensions branch, we can have innovation... If parts of this > branch gain concensus, we then integrate these stable components into the > Jini Standards, without requiring a namespace change that breaks > compatibility. > > So think of one branch as the concensus branch and the other is the doer's > decide branch. > > Those of us who want to further River aren't a threat to the existance of > existing implementations. Git will enable us to maintain a shared codebase. > > For example the Lease interface could contain a default method: > > public interface Lease { > > final long FOREVER = Long.MAX_VALUE; > final long ANY = -1; > final int DURATION = 1; > final int ABSOLUTE = 2; > long getExpiration(); > void cancel() throws UnknownLeaseException, RemoteException; > void renew(long duration) > throws LeaseDeniedException, UnknownLeaseException, RemoteException; > void setSerialFormat(int format); > int getSerialFormat(); > LeaseMap<? extends Lease, Long> createLeaseMap(long duration); > boolean canBatch(Lease lease); > > default Lease renewal(long duration) throws LeaseDeniedException, > UnknownLeaseException, RemoteException > { > renew(duration); > return this; > } > > } > > The next thing would be to write a contract for Lease equals and hashcode > methods. > > Q: How does this help? > A: It reduces maintenance, simplifies debugging, reduces mutable state and > clarifies ambiguity in implementing the existing interface. > > All River’s Lease implementations in the Jini standards extended edition, > would be immutable and override the new method and return a new copy, the > renew method would be implemented so the Lease would replace itself in the > LeaseMap. LeaseMap implementations would replace the existing Lease with > the renewed Lease. The implications are subtle, but it simplifies the > implementation of Leases greatly. > > For those that remain sceptical, existing third party Lease > implementations and the strict Jini standards edition would still be > interoperable, although less defined, this would be an example of stable > backward compatible api evolution. If there was concensus, this change > could be incorporated into the Jini standards. The Jini standards extended > edition would be fertile ground for developers to innovate without > threatening the existing user base. > > An example of hampered development is my recent research into River > service security. I found that River still depends on Serialization for > security. > > No other software today relies on Serialization for security? Do we really > want this? > > The results of my investigation indicated that the serialization protocol > wasn’t at fault, but the input stream needed to be filtered much like a web > server filters input. Securing serialization by filtering input, would > allow bootstrap proxy’s to be securely obtained from lookup services by > clients. > > Unfortunately due to ObjectInputStream's api, it isn't possible to have > pluggable input filters like a web server, but it was possible to use the > existing protocol and remain serial form compliant. > > Now this atomic serialization was an implementation fix, relevant only to > Entry’s and bootstrap proxy’s, it's selection would have been by > configuration and security constraints. No users had to implement it; it > wasn’t part of the api, but it would provide them with secure bootstrap > proxy’s. If users wanted to use this in their own services, they could have > done so, with significant performance, evolution and security benefits. > > The consensus on river-dev was that we should limit the amount of data > that could be downloaded through an input stream, but serialization wasn’t > a River project concern. The sad reality is that you can’t limit the amount > of data through the input stream, because it breaks once you hit that > limit. The recommendation can’t be implemented, but I figured the odds of > acceptance were low and let it go. It was shut down before I had > opportunity to present the code for peer review. > > Why isn't the message: "Ok, lets have a look at the code, can you explain > more?" This was how the OpenJDK project responded. Due to other > constraints, it couldn’t be fully adopted by OpenJDK, but at least they > listened and implemented some of the functionality. > > So now we live with a legacy; we have this broken proxy trust model that > burdens users. I would still like to fix it. > > As a River developer, I’m being boxed in by Jini Specifications, I can > only fiddle with the implementation and fix bugs. I can build software > around it, but I’m prohibited from fixing fundamental design flaws. > > I had to pause and think before sending this email, I don't want to stir > up arguments on the list, but then I also feel the need to talk about it. > > I thank you for your honesty and hope you'll respect me for mine, perhaps > we can reach a compromise that has mutual benefits, I understand we can't > all agree, but we need to find a way to make progress when we don't. > > Thank you, > > Peter. > > >
