Since I have pretty much given up on trying to work on proton, I'm not sure that my opinion counts for much anymore... however I suppose I'm a little curious how the JNI binding differs in "extra work" vs. PHP or Ruby or such. Similarly why the API distinction is "hasty" when again it should (modulo some language idiom changes) reflect the API that the C is presenting to Ruby/PHP/Python/etc.
To me the fact that APIs aren't defined but that instead the only way of knowing what a particular call should do seems to be 1) look at the code in C and, if it's not clear, then 2) ask Rafi; is why I no longer feel it productive to try to contribute. As such, it seems that this change is more about vertical scaling (making it easier for Rafi, and those sitting very near him, to work more productively), than horizontal scaling (allowing people who are not Rafi or with easy access to him to work on it). Overall I don't care very much about Messenger in Java (either pure Java or JNI), but I do care about the Engine. When the Proton Engine was first discussed it was always said that the C code should be able to be called from within Java as an alternative to a pure Java impl. - has this changed? Going back to the horizontal vs. vertical scaling... If the project were structured differently I would be very happy to contribute to the engine side of things in Java. However I think that would require us to be organised differently with a recognition that the Engine and Messenger are conceptually separate (with Messenger depending on the Engine API, and having no more authority over defining/testing the engine API than any other client - such as the C++ broker, ActiveMQ, or the upcoming JMS clent, etc.). As such I would rather see us fix the definition of the API (if you believe it to be "hasty") rather than simply try to remove any notion of their being an API which is distinct from the implementation. -- Rob On 19 December 2013 14:49, Rafael Schloming <r...@alum.mit.edu> wrote: > Hi Everyone, > > I've been struggling with some of the cross language maintenance aspects of > proton for a while, and I believe we need to take some steps to improve the > situation. I'm one of a tiny number of people (two possibly) who regularly > commit changes to both the Java and C codebase and attempt to keep the two > in sync and at feature parity. Because of this, not everyone is necessarily > aware of the process, but to summarize the issue, currently there are far > too many moving parts and layers of indirection involved. This is not only > a significant drag on my personal productivity, but perhaps more > importantly it is a significant barrier to growing the number of proton > contributors in general as all those moving parts and layers of indirection > need to be understood before being able to make complete contributions. > > To get an idea of what's going on here I think it helps to look at what's > involved in a simple change. Recently I noticed an edge case around status > updates in the messenger interface. The fix involved adding another value > to messenger's status enum and making a trivial logic change to make use of > that value under the appropriate circumstances. You can look at the full > commit here[1] if you like, but it breaks down in the following way: > > Changes to C implementation (.h and .c file): 8 LOC > Changes to the python test shim for C: 3 LOC > Changes to the Java API: 2 LOC > Changes to the pure Java implementation: 6 LOC > Changes to the JNI binding: 4 LOC > Changes to the python test shim for Java: 3 LOC > Changes to the python test suite: 2 LOC > > Now obviously from a personal productivity perspective it is at a minimum > annoying to have to touch so many different parts in order to make even a > trivial change, but setting that aside for the moment, what is really > sobering about this is that each one of the above parts involve a non > trivial learning curve on their own, and while it's true that only a few > lines of code are needed in each area, it is necessary to understand each > piece before being able to write the correct few lines of code. This > presents a pretty daunting hurdle for bringing new contributors to the > codebase up to the level needed to make even a trivial change like the one > above. > > The JNI binding and the python test shim for Java both exist to serve > similar purposes, namely to facilitate running a common test suite against > both implementations in order to ensure common behaviour. The python test > shims allow the python test suite to run against both proton-c and proton-j > (via jython), and the JNI binding allows pure Java tests to be run against > proton-c as well as proton-j. > > Currently by my count there are about 19 such tests in Java. By comparison > there are about 266 tests in the python test suite. Also, judging by > commits, the python test suite is growing/actively maintained, and the java > system tests (the subset of java tests that are run against both > implementations) are neither. On top of this the JNI binding itself has > suffered significantly from bit rot. As far as I know it is not made use of > outside of our own test infrastructure, and currently about 50% of the > tests run against it are skipped because it is only minimally maintained > when necessary to get the tests to pass. > > Because of all this I'm proposing that we remove the JNI binding and roll > back the Java API/impl split that was (hastily) done to facilitate the test > infrastructure. This should significantly simplify the development process > relative to what it is now, and while there is still more learning curve > than desired with the python shims, I believe this will put us in a > position to improve the shims, remove duplication, and minimize the > overhead associated with them, ultimately allowing us to make the codebase > more transparently communicate its design and hopefully lessen the > learning curve for new contributions. > > I would love to hear thoughts and/or alternative ideas on how to improve > things. I would like to start addressing this early in the 0.7 development > cycle. > > [1] http://svn.apache.org/r1551950 >
