Stefano Mazzocchi wrote:
Daniel Fagerstrom wrote:
IMO our huge community problem is what is most important. No, I'm certainly not talking about our excelent existing community, I'm talking about the fact that we only elected two new committers during 2004, compare that to around ten during 2003, and probably similar figures the years before that. Are our requirements on new committers to high? Maybe, maybe not, I think the standards have been high all the time.
Daniel is right in identifying a transition in our development model and I think I know why this is and has very little to do with block (even if that's important too).
<snip/>
Scripting taught me (spoiled me?) with incredible high productivity
I started with Basic and some assembler and Forth. Then I wrote everything in Common Lisp, Prolog or obscure declarative research languages when I was a computer science student. Oh what I hated it when I had to use C, C++ and Java when I started to work in the industry. My productivity decreased with about a magnitude. Using Mathematica (which is based on concepts from functional and logic programming language) for my applied mathematics research reminds me occationally that there is a decent, efficient and modern way to solve problems with computers.
When I started to do webapp development I loved XSLT despite its awfull syntax, at last I was able to be productive and work in a functional language again. Cocoon made it even better as I was able to use the pipeline pattern again, something that was central in my use of Lisp.
but I think there is a problem when people are not able to go back and fix the underlying object model that the scripting is a glue for... because it forces people to hack things in their glue layer, which reduces reuse and *prevents* contributions to get back in the system!!!
I think that is a key observation.
I'd like to add to that flowscripts is used for three different concerns:
* multipage flow control (obviously)
* scripted side effects and business logic glue
* injecting pipeline data in your business objects (what I called input pipelines in RTs some year(s) ago)
Maybe it can be divided along other concern dimensions. The important point is that we didin't made a proper concern analysis of the vast area of webapps, but instead decided that it is outside the scope of Cocoon.
If we instead identify the involved concerns (the three above IMO) we can build framework support for them with well defined component interfaces. And only then people will start to contribute reusable components.
We are all (rightefully) to ashamed about the mix of concern mess we create in our flowscripts, to contribute anything back.
For DHTML there is XHTML/XUL+CSS+Ecmascript+XPCOM and believe me, they work great, they are a wonderful platform to work with and our latest experiments prove that very well (see http://simile.mit.edu/piggy-bank). Linotype had a kick-ass, best-of-breed WYSIWYG DHTML editor, but I never was able to figure out how to modularize it enough for people to reuse (or include into cforms, which is probably the best solution).
For the sitemap, our object model is not super clean (actions should go, matchers and selectors should be unified, views should be unified with virtual pipeline components) but it's solid and well understood. I can live with the little nuances. The number of existing components is stunning and incredibly solid, the need to write custom pipeline components is very small... yet is that need emerges, people just prefer to hack around it in flow... but knowing how hacky their solutions are, they are never contributed back!
Exactly!
No framework => no user contributions!
[sound of many people sighing around the virtual room]
:)
The biggest and most continous patchflow over the last year came from the portal system and cform: those are the systems were the underlying infrastructure has to be able to support the load of the systems on top *and* the people working on it (Sylvain and Carsten) are the only few people left that have no problems in touching the core.
Everybody else, myself included, is either scared to touch anything anymore in the core, or too spoiled by the scripting hacks around it to do anything about it.
I'm not affraid about the core as such, I have been afraid about introducing back incompabilities without noticing it. Time to stop worrying so much about that and start to actually take responsibility and do something!
Vadim will spot the incompatibilities and complain anyway ;)
Scripting spoiled me so much (and my life habits changed too with my new job) that my attention span in coding is very much reduced: spending hours without seeing anything solidifying will very likely force me to leave it unfinished and uncommitted... then the next day something else comes along... and when I go back to it I forgot where I was (or the codebase would have already changed!) and I have to start over again!!!
I can't count how many times that happened!
This is not only about scripting. What you are describing is the harsh reallity of living the life of a grown-up ;) Work, family and habits makes it so much harder to spend hours, days and weeks focusing on a complicated problem (like a large refactoring or a framework design). We all try and maybe spend some weekends and late nights on doing stuff. But after a while you became worn out from the lack of sleep and from not taking responsabillity for all the rest parts of your life :/
I'll guess this is the case for most of you. And that is one of the weaknesses in our current community:
We lack diversity, we have so much in common.
Where are the age 24 Stefano with friends that could spend weeks hacking and beeing obsesed with basic contracts today? Not in our community, but somewhere else.
I'm in *desperate* need to reduce that try/fail cycle in cocoon or I simply won't be able to touch its code anymore. Which is very frustrating, given the amount of things I could get done if I only could go back to it.
I'm saying "unit tests" as everyone else. I have used that while refactoring JXTG and it makes the work much more efficient.
Also we need to improve the internal design and contracts to make it easier to understand whats going on and do support work. But I doubt that there is an easy way towards that. The refactoring with unit tests way is the only I know about.
Moving the blocks out of the way and implementing a compile-time block deploying tool would do wonders in reducing the size of the codebase and making people think that cocoon is a more tractable problem... it might well increase the birth of an ecosystem of block creation (I really hope so!)
So do I.
Then we should mercelessly remove everything that even slightly smells application from core so that it becomes mean and lean.
Then we need to attract framework developers for core development. They are a different kind relative to application developers. There where more of them in early Cocoon. Avalon was mainly populated by them. Some of us are former framework developers who have grown fat, old, lazy and even worse "pragmatic".
Geting the core mean an lean is not about consensus and cozyness, its about being obsessed by geting the contracts "right".
An example: there have been an overwhelming response in general discussions about templates, expression languages, user implemented tags or not and object models. But when I have asked for feedback on interfaces for e.g. expression languages there have been nearly no response. Is that because:
* it is irrelevant and you didn't care
* you didn't have time
* you found the design perfect and had nothing to add
* you trusted my judgement
* you thought it sucked badly, but didn't feel like discussing that with me
* WTF, obsessing about interfaces is only for academics, in the "real world" there are more important things
but it won't solve the issue that cocoon *itself* is exhibiting: how scripting spoiled us.
Java is an incredibly verbose language, but Eclipse fixes that during writing and verbosity helps in reading other people's code, so it's not a bad language after all (and 1.5 will reduce a lot of that verbosity which is ever better)
We have a catch 22 situation. You must be a really god Java developer to be able to contribute enough to Cocoon to become a committer. But many really good Java developers have the irritating idea that it is much easier to develop things in Java, that "serious" things must be developed in Java and that scripting sucks ;)
Avalon is a very intrusive framework and a lot of the code feels like it's unnecessarely complex but its death allowed us to feel free about modifying our own container.. which is leading the innovations that Sylvain and Carsten introduced. This feels refreshing and has very nice promises for the evolution of our internal architecture (toward a simpler one, learning from what the other CoP containers did)
What we need is to help people write java code again and the only way to achieve this is by reducing the try/fail cycle.
If we don't do this, I fear that our layers and layers of scripting glue won't hold the pressure and collapse, leading to the need for smaller and more maintainable engines, eventually a code fork of the core. This would be very energy consuming, both technically and socially.
This is a wake up call people: our foundations are becoming old and are in danger of losing the knowledge that allows us to maintain it. (what happens if Sylvain and Carsten leave! think about it!)
An old foundation, no matter how solid at one point, becomes fragile and deteriorates if not actively maintained. We cannot allow to lose or deteriorate the collaborative knowledge on that, or we are doomed to fragmentation.
Yes and to me this means that we (not only the few brave) must start to care about the inner design of core again. With care I mean being active both in coding and design discussions. IIRC there was a common interest in such things when Cocoon was mainly a publishing framework but when it transitioned to become a webapp framework, the community involvement in basic interfaces declined.
This is why I think we should make an effort to think about how we can solve this time/cognitive-energy discrepancy between the scripting and the java try/fail cycle.
That is important, but as I described above I think there are other important factors as well.
/Daniel
