> On 15 Oct 2015, at 07:58, Adrian Crum <[email protected]> > wrote: > > Keep in mind that much of David's code in OFBiz has been rewritten. So yes, > we CAN do a better job than him.
I think there’s a name for this logical fallacy… > Also keep in mind that Moqui duplicates some of the problems I listed - so by > using Moqui, we keep the same problems instead of fixing them. Could you be more specific, other than the type conversion stuff you mentioned many years ago (which I fully disagree with)? -David > On the other hand, it is David's responsibility to fix them, not ours. > > If we created a sub-project, then we would have the opportunity to review > committer permissions and perhaps restrict access to the new code. > > Adrian Crum > Sandglass Software > www.sandglass-software.com > > On 10/15/2015 6:34 AM, Al Byers wrote: >> I was waiting for someone to bring this up. David Jones created Moqui with >> the same end in mind that the rewrite is meant to accomplish. Do you think >> that you will do a better job than him? Even if you could, would it be so >> much better that it warrants the effort that it would take? >> >> Is this a political thing? I don't know that David would give up control of >> Moqui Core and I, for one, would not want him to. So many of OFBiz's >> problems are the result of ineptitude on the part of committers who did not >> know what they were doing (like me.) But I don't know that the Core will >> change all that much going forward and there should be places in the Mantle >> to contribute and, most certainly, in the Crust. >> >> I know that there were some valid questions about licensing and project >> management, but I would think that they could be worked out. >> >> On Thu, Oct 15, 2015 at 4:46 AM, Hans Bakker <[email protected]> >> wrote: >> >>> Why not skip this step, using moqui which is already up and running and >>> then as Adrian states as a next step: applications could be pulled down and >>> adapted to it. >>> >>> Hans >>> >>> >>> >>> On 15/10/15 16:51, Jacques Le Roux wrote: >>> >>>> I'm in the same mood than Paul and Scott. So a sub-project could indeed >>>> be a solution. >>>> >>>> Jacques >>>> >>>> Le 15/10/2015 03:11, Adrian Crum a écrit : >>>> >>>>> I agree that a sub-project would be nice to have. Once the new framework >>>>> is up and running, applications could be pulled down and adapted to it. >>>>> >>>>> Adrian Crum >>>>> Sandglass Software >>>>> www.sandglass-software.com >>>>> >>>>> On 10/14/2015 5:53 PM, Ron Wheeler wrote: >>>>> >>>>>> This seems to be very good advice. >>>>>> A completely separate sub-project under OFBiz with its own mailing lists >>>>>> would keep the people together yet all allow the new framework the >>>>>> flexibility to move forward. >>>>>> >>>>>> Ron >>>>>> On 14/10/2015 8:27 PM, Scott Gray wrote: >>>>>> >>>>>>> My advice is as follows: >>>>>>> 1. If people are interested, then get them together and start working >>>>>>> on it. >>>>>>> 2. Find somewhere to do the work. I don't think a branch is >>>>>>> appropriate >>>>>>> because it's completely new development rather than a refactoring. I >>>>>>> don't >>>>>>> have any objections to it being done under the ASF OFBiz umbrella >>>>>>> (although >>>>>>> I don't really see the need either). >>>>>>> 3. Set up a separate mailing list for discussion. Generally I'd try >>>>>>> and >>>>>>> keep quiet about it in the early stages on the dev/user lists or other >>>>>>> marketing channels because it could potentially harm adoption of our >>>>>>> existing framework (think AngularJS 2.0). >>>>>>> >>>>>>> There really isn't any need to get early stage sign-off from the PMC or >>>>>>> anyone else in the community. You only need enough PMC approval to >>>>>>> get the >>>>>>> required infrastructure sorted, which I don't think would be an issue. >>>>>>> >From there, it's really up to the community to decide whether or not >>>>>>> the >>>>>>> thing will fly. >>>>>>> >>>>>>> Regards >>>>>>> Scott >>>>>>> >>>>>>> >>>>>>> On 15 October 2015 at 08:21, Adrian Crum >>>>>>> <[email protected] >>>>>>> >>>>>>>> wrote: >>>>>>>> I understand that Sharan brought up the framework rewrite subject at >>>>>>>> ApacheCon, and some attendees felt that the framework is fine and no >>>>>>>> action >>>>>>>> needs to be taken. >>>>>>>> >>>>>>>> In this message, I will try to give a detailed explanation of why a >>>>>>>> framework rewrite is necessary. I don't plan to take any further >>>>>>>> action on >>>>>>>> this subject, because I've brought it up before without success, and >>>>>>>> I'm >>>>>>>> tired of discussing it. It is my hope that the light bulb will click >>>>>>>> on in >>>>>>>> someone's head and they will take action. >>>>>>>> >>>>>>>> My Background >>>>>>>> ------------- >>>>>>>> >>>>>>>> I became a member of the OFBiz community in 2004. I immediately >>>>>>>> started >>>>>>>> making contributions to the project by supplying patches to the issue >>>>>>>> tracker. In 2007, I became a committer. Most of my initial work was >>>>>>>> on the >>>>>>>> UI and some work in the applications (mainly Asset Maintenance and >>>>>>>> Work >>>>>>>> Effort). I stayed away from touching the framework code because it was >>>>>>>> deep, dark, and scary. >>>>>>>> >>>>>>>> Eventually, I started to understand how the framework code works and I >>>>>>>> made some minor modifications. As my understanding grew, I progressed >>>>>>>> to >>>>>>>> rewriting large swaths of framework code - making it thread-safe, >>>>>>>> fault >>>>>>>> tolerant, efficient, and easier to use. >>>>>>>> >>>>>>>> I will list some of my contributions here, so everyone can have a >>>>>>>> clear >>>>>>>> understanding of my experience with the framework code: >>>>>>>> >>>>>>>> New Features >>>>>>>> >>>>>>>> User Preferences >>>>>>>> >>>>>>>> Visual Themes >>>>>>>> >>>>>>>> Custom UI Label XML File Format >>>>>>>> >>>>>>>> Temporal Expressions >>>>>>>> >>>>>>>> Data Type Conversion Framework >>>>>>>> >>>>>>>> Screen Widget Boundary Comments >>>>>>>> >>>>>>>> Metrics >>>>>>>> >>>>>>>> Integrations >>>>>>>> >>>>>>>> UEL >>>>>>>> >>>>>>>> iCalendar >>>>>>>> >>>>>>>> JSR 223 >>>>>>>> >>>>>>>> WebDAV >>>>>>>> >>>>>>>> LDAP >>>>>>>> >>>>>>>> Refactorings/Improvements >>>>>>>> >>>>>>>> FlexibleStringExpander >>>>>>>> >>>>>>>> FlexibleMapExpander >>>>>>>> >>>>>>>> FOP Integration >>>>>>>> >>>>>>>> FreeMarkerWorker >>>>>>>> >>>>>>>> Date-Time Handling >>>>>>>> >>>>>>>> Mini-language >>>>>>>> >>>>>>>> Job Scheduler >>>>>>>> >>>>>>>> In addition, I have performed innumerable framework bug fixes. >>>>>>>> >>>>>>>> So, the contents of this message come from years of experience mucking >>>>>>>> about in the framework code. >>>>>>>> >>>>>>>> Okay, let's get started... >>>>>>>> >>>>>>>> Initial Problem Statement >>>>>>>> ------------------------- >>>>>>>> >>>>>>>> In 2009, David Jones started a framework rewrite in a branch: >>>>>>>> >>>>>>>> >>>>>>>> https://svn.apache.org/repos/asf/ofbiz/branches/executioncontext20090716 >>>>>>>> >>>>>>>> At the time, there was some agreement that a rewrite was necessary, >>>>>>>> but >>>>>>>> there was disagreement as to how the rewrite should be incorporated >>>>>>>> into >>>>>>>> the project: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/200908.mbox/%[email protected]%3E >>>>>>>> >>>>>>>> >>>>>>>> There were concerns that a rewrite would break backward compatibility. >>>>>>>> Work on the rewrite branch stopped. Eventually, Jacopo suggested the >>>>>>>> community be more accepting of backward-incompatible changes: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://mail-archives.apache.org/mod_mbox/ofbiz-dev/201004.mbox/%[email protected]%3e >>>>>>>> >>>>>>>> >>>>>>>> Despite an effort to convince David to proceed with the framework >>>>>>>> rewrite, >>>>>>>> he ended up doing it in a separate project: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://mail-archives.apache.org/mod_mbox/ofbiz-dev/201104.mbox/%[email protected]%3E >>>>>>>> >>>>>>>> >>>>>>>> This page describes differences between OFBiz and Moqui, and within >>>>>>>> it you >>>>>>>> can extract information on the problems David was trying to solve: >>>>>>>> >>>>>>>> http://sourceforge.net/p/moqui/discussion/1086127/thread/4c52f240/ >>>>>>>> >>>>>>>> There was an email he sent out on the OFBiz dev list where he listed >>>>>>>> the >>>>>>>> problems he saw in the framework, but I can't find it. The rest of >>>>>>>> this >>>>>>>> message will include the issues he mentioned (the ones I remember). I >>>>>>>> was >>>>>>>> in agreement with him at the time, and I still agree that a framework >>>>>>>> rewrite is necessary. >>>>>>>> >>>>>>>> The Problems >>>>>>>> ------------ >>>>>>>> >>>>>>>> Code is scattered everywhere - due to an initial effort to make the >>>>>>>> framework modular. This causes serious problems. The mere fact that >>>>>>>> components like entityext and securityext EXIST makes it clear that >>>>>>>> there >>>>>>>> are problems - those components should not be there. Also, we run >>>>>>>> into the >>>>>>>> recurring problem of circular dependencies (component A will not build >>>>>>>> unless component B is built, and component B will not build unless >>>>>>>> component A is built). >>>>>>>> >>>>>>>> Bad separation of concerns. There are far too many examples of classes >>>>>>>> that try to be everything to everyone. This makes debugging >>>>>>>> difficult, and >>>>>>>> it makes maintenance/improvements a nightmare. [Using an analogy, >>>>>>>> consider >>>>>>>> an automobile design where a spark plug is not separate from a >>>>>>>> transmission. Instead, the automobile uses a spark-plug-transmission. >>>>>>>> So >>>>>>>> when the engine is running rough because the spark plug is bad, you >>>>>>>> have to >>>>>>>> replace the spark plug AND the transmission.] A good framework >>>>>>>> example can >>>>>>>> be found in my rewrite of the mini-language code. Originally, the >>>>>>>> models >>>>>>>> AND the script execution context both contained script behaviors - >>>>>>>> making >>>>>>>> debugging/improvements difficult. I changed it so only the models >>>>>>>> contain >>>>>>>> script behavior and the script execution context contains only the >>>>>>>> script >>>>>>>> execution state. >>>>>>>> >>>>>>>> Lack of good OO design. There are many places where a bit of framework >>>>>>>> functionality is contained in a single method that is hundreds or >>>>>>>> thousands >>>>>>>> of lines long. There is a term for that: Brittle Code. Code isn't >>>>>>>> reused. >>>>>>>> Instead, it is copy-and-pasted all over - so when a problem is found >>>>>>>> in the >>>>>>>> C&P code, it has to be fixed in many places instead of one. >>>>>>>> >>>>>>>> Fail-slow design. There are a lot of places in low-level code where an >>>>>>>> error condition is encountered, but instead of throwing an exception, >>>>>>>> the >>>>>>>> error is ignored and maybe it is logged, or the code tries to "guess" >>>>>>>> at a >>>>>>>> solution and then provide an arbitrary default behavior. I've seen >>>>>>>> many >>>>>>>> developers struggle with debugging a problem because they didn't look >>>>>>>> at >>>>>>>> the logs, or because the error was not logged and there is no way of >>>>>>>> knowing what caused it. They end up spending hours single-stepping >>>>>>>> through >>>>>>>> code until it reaches the error. >>>>>>>> >>>>>>>> Out-of-date code. A good example is the use of Javolution. That >>>>>>>> library >>>>>>>> was beneficial in the Java 1.4 days, but it is not necessary today >>>>>>>> because >>>>>>>> of improved garbage collection. Another good example is DCL code. DCL >>>>>>>> was >>>>>>>> used extensively in OFBiz, but it is clearly documented to be an >>>>>>>> unreliable >>>>>>>> design (I can get it to fail 90% of the time). Some DCL code has been >>>>>>>> replaced, but a lot of it is still there. >>>>>>>> >>>>>>>> Portions of the API are overly complicated. Some methods require a >>>>>>>> collection of user-specified artifacts/arguments, which makes client >>>>>>>> code >>>>>>>> complicated and verbose. (David solved that problem with his Execution >>>>>>>> Context.) Portions of the API are cluttered with unnecessary >>>>>>>> "convenience >>>>>>>> methods" - making the API harder to learn and memorize. In some >>>>>>>> places, a >>>>>>>> domain-specific API is spread across instance methods and static >>>>>>>> methods >>>>>>>> and across different classes - making the API hard to understand and >>>>>>>> use. >>>>>>>> Yes, there can be good designs that require something like that, but >>>>>>>> in the >>>>>>>> OFBiz framework, it exists because of a bad design, not a good one. >>>>>>>> >>>>>>>> Use of thread-local variables. This makes multi-threaded design >>>>>>>> impossible. The J2EE specification and the Servlet API require one >>>>>>>> thread >>>>>>>> per request (and most J2EE libraries depend on that behavior), so the >>>>>>>> current design makes sense from a J2EE perspective, but what if I >>>>>>>> don't >>>>>>>> want to run the framework in a J2EE container? Which leads to the next >>>>>>>> problem... >>>>>>>> >>>>>>>> Dependence on J2EE designs/APIs/libraries. There are developers in the >>>>>>>> Java community (myself included) who are beginning to question if >>>>>>>> J2EE is >>>>>>>> really necessary to run web applications. The folks at Atomikos are a >>>>>>>> good >>>>>>>> example. OFBiz does not use EJBs, so tying the framework to J2EE does >>>>>>>> not >>>>>>>> make sense. It would be better if the framework was designed to run >>>>>>>> outside >>>>>>>> a J2EE container, and then have container integration as an option. >>>>>>>> >>>>>>>> Configuration files are scattered everywhere. Anyone who has deployed >>>>>>>> OFBiz in a production environment will agree this is a problem. Try >>>>>>>> changing the HTTP/HTTPS and port settings - it is a nightmare. Some >>>>>>>> configuration settings are in nonsensical places. >>>>>>>> >>>>>>>> An abysmal lack of unit testing. I don't have an exact figure for code >>>>>>>> coverage, but my gut feeling is coverage is less than 10%. Basically, >>>>>>>> we >>>>>>>> all have our fingers crossed - hoping that the framework code works as >>>>>>>> expected. This was made painfully obvious a while back when I was >>>>>>>> looking >>>>>>>> at some entity caching code and thought to myself "this code can't >>>>>>>> work." >>>>>>>> So I wrote some entity cache unit tests and confirmed that the entity >>>>>>>> cache >>>>>>>> had serious problems. Think about that - years passed with no entity >>>>>>>> cache >>>>>>>> unit tests and consequently we had no idea it wasn't working. >>>>>>>> >>>>>>>> Fix Versus Rewrite >>>>>>>> ------------------ >>>>>>>> >>>>>>>> Jira issues could be created for these problems and teams of >>>>>>>> developers >>>>>>>> could work to fix them. >>>>>>>> >>>>>>>> Or, we could create a branch and start over from scratch. This time >>>>>>>> around, there should be less push-back from people concerned about >>>>>>>> backwards compatibility. A rewrite offers the advantage of >>>>>>>> reconsidering >>>>>>>> everything - like API design, general problem solving, and new >>>>>>>> features. >>>>>>>> >>>>>>>> I created a Wiki page for a framework design: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> https://cwiki.apache.org/confluence/display/OFBADMIN/Another+Framework+Vision >>>>>>>> >>>>>>>> >>>>>>>> but there hasn't been much interest in it. If the community decides >>>>>>>> to go >>>>>>>> ahead with a rewrite, then please feel free to use the Wiki pages as a >>>>>>>> guide. >>>>>>>> >>>>>>>> Sandglass Foundation >>>>>>>> -------------------- >>>>>>>> >>>>>>>> Like David, I came to the conclusion that a framework rewrite would be >>>>>>>> easier outside the OFBiz community. So, I created my own library >>>>>>>> called >>>>>>>> Foundation: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> http://www.sandglass-software.com/products/sandglass/documents/Foundation_Brochure.pdf >>>>>>>> >>>>>>>> >>>>>>>> (PDF) >>>>>>>> >>>>>>>> and I only mention it here to stress how wonderful it can be to start >>>>>>>> with >>>>>>>> a clean slate and design an API that is concise yet powerful. (Please >>>>>>>> do >>>>>>>> not discuss Foundation here, contact me privately if you want more >>>>>>>> information.) >>>>>>>> >>>>>>>> Some examples of what can be done with a rewrite: >>>>>>>> >>>>>>>> A single configuration file >>>>>>>> Use ANSI/ISO SQL SELECT statement strings instead of constructing >>>>>>>> complicated Java structures >>>>>>>> Simultaneous asynchronous queries >>>>>>>> Relational integrity across multiple datasources >>>>>>>> Multi-table SELECT across multiple datasources >>>>>>>> Automatic and transparent row version control >>>>>>>> Automatic and transparent multi-language datasource support >>>>>>>> Abstract entities (similar to SQL user types) >>>>>>>> Service engine throttling (protects against server >>>>>>>> over-utilization) >>>>>>>> Simplified security (authorization) ( >>>>>>>> >>>>>>>> https://cwiki.apache.org/confluence/display/OFBIZ/OFBiz+Security+Redesign) >>>>>>>> >>>>>>>> >>>>>>>> Pure interface-based API - so developers are free to modify >>>>>>>> framework >>>>>>>> behavior by using decorators >>>>>>>> Thorough unit tests >>>>>>>> >>>>>>>> Benefits of a rewrite: >>>>>>>> >>>>>>>> Reduced resource requirements (lower hosting fees) >>>>>>>> Reduce application development time - due to a simplified API >>>>>>>> Easier framework code maintenance >>>>>>>> Better reliability >>>>>>>> >>>>>>>> Conclusion >>>>>>>> ---------- >>>>>>>> >>>>>>>> Like I said at the start, this is all I will say about the subject. >>>>>>>> I'm >>>>>>>> done trying to convince everyone. I hope someone agrees with me and >>>>>>>> they >>>>>>>> are able to build support for the idea. >>>>>>>> >>>>>>>> -- >>>>>>>> Adrian Crum >>>>>>>> Sandglass Software >>>>>>>> www.sandglass-software.com >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>> >>
