On Wed, 25 Mar 2009 20:31:23 +0900, Martin Aspeli wrote: > Balazs Ree wrote: >> On Wed, 25 Mar 2009 11:12:47 +0900, Martin Aspeli wrote: >> >>> Matthew Wilkes wrote: >>>> Hello everyone! >>>> >>>> It's time for us to write up our ideas list, for now, can everyone >>>> who has any ideas please reply to this email with a title and maybe >>>> an explanatory sentence of anything they think is a good idea. >>> Here's a fun and experimental one: >>> >>> * Server side KSS compiler >>> >>> Ever since our adoption of jQuery, we have some overlap between the >>> KSS core and jQuery in things like event binding, AJAX request making >>> and DOM manipulation. KSS has a really nice programming model, and >>> fits a certain user base (the "I'm afraid of JS and I want structure >>> and testability on the serverside" type) well. However, this >>> duplication has some performance implications in terms of page weight >>> (we send a lot of JS with each request) and the fact that both jQuery >>> and KSS need to scan the DOM, bind events and so on when the page is >>> loaded. >>> >>> Martijn Pieters suggested it should be possible to parse a KSS file >>> server-side and "compile" it to equivalent jQuery operations for >>> binding event handlers and performing client side manipulation or AJAX >>> requests. That is of course only half of it: we also need something to >>> convert server-side (or parse client-side) the response payloads that >>> come back from KSS actions so that they do something when returned. >>> >>> The aim here would be to find out if we can avoid sending all those >>> kilobytes of KSS client-side code with each request, whilst still >>> retaining the nice programming model that KSS offers. Effectively, >>> this would be an alternative implementation of the KSS protocol on top >>> of jQuery. >> >> I was giving some thought to this and came to similar conclusions. I >> see a first step that's feasible to do and brings an immediate benefit. >> Therefore I propose to give focus to this. This would be the building >> the server side parser generator for .kss files. > > Why do we need a *parser* *generator*? That sounds very recursive to me. > Why not just a *parser* and a jQuery *generator*. :)
By "parser generator" I mean a tool that takes a formal syntax definition (like BNC), in this case for the .kss syntax, and generates a parser based on this. This parser then, will be able to parse and compile kss files and generate whatever output desired. To get a better idea of what a "parser generator" is, please look at the following example: http://www.antlr.org/ There are also other approaches to build the parser. There are several tools and libraries made for this purpose, and one of them can be chosen to be applied - but you can do without them too, since indeed the functionality they provide can be just programmed in plain python. The reason why I am interested to use existing tools and libraries for this purpose (be it a "parser generator" or not) is that I have bad experience with "custom made" parsers, including our very own parser for kss. >> Based on an ideally BNC notation of .kss syntax, this parser generator >> would be capable of generating a parser in both python (server side) >> and javascript languages (client side). > > Why do we need a JS parser? Don't we have one of those already? If we'd keep on using two parsers, it would be an impossible task to keep then in sync. They would deviate already at the time of creation. But if we use the parser generator approach, we get the js parser together with the python parser at minimal cost. And the even bigger advantage will manifest when there will be no need to maintain and retrotest two different parser implementations (in two different languages even). If the parser generator approach is not used, then I agree it may be more difficult or impossible to get a python and js parser from the same source, so then I would not consider replacing our existing js parser, but in that case the end result would be that kss looses the ability to work on non-python platforms, since we would eventually drop maintaining the js lib and keeping it in sync. >> The server side (python) parser would parse a kss resource and would >> generate a javascript compiled version. This would be linked in the >> page as a compiled kss resource (replacing the original .kss it >> compiled). The operation of the server side parser could be governed >> through the portal_kss tool first. Optionally, on long term, it could >> make a wsgi middleware that compiles the kss resources found in the >> pages, doing the compilation and replacing the links to the compiled >> versions transparently. > > I hadn't thought about the WSGI approach. This may in fact be easier and > more interesting. > > I think in the short term, though, the aim should be to just make the > generator work. Finding out where to hook it in is not such a big deal. Agreed, actually I think making the wsgi middleware is not such a big task (it may be easier then hook it to plone_kss, wink), but it would be more difficult for people to deploy on today's Plone, plus it really does not matter that much for the end goal. Could be added later. >> When during kss bootstrap at pageload, the client observes the compiled >> kss code in the html page (included or linked in), it simply executes >> it. In the first version this could execute calls on the existing kss >> api. Which would do the right thing that would have resulted if the >> client does the compilation from the original .kss resource. This makes >> it possible to deploy this solution with existing kss right away. > > But then what's the point? If we need the full KSS client-side library > anyway, we don't really need it. My observation and analysis disagrees on this point: the biggest performance bottleneck with kss is pageload time. This is not so much a result of the code size (with modern compression applied, kss's javascript is quite small, and in a modern website, gives only a small part of the full js code) but the bootstrap sequence (according to my benchmarks). For some reason (hint: no native domload event in browers) the delay during pageload is much bigger than what should be necessary by the amount of CPU used. With the server-side parsing implemented in itself, calling on the current kss, the whole problem would be circumvented and pageload time would reduce to zero. (What would remain is what your js code would spend by setup anyway, but kss would not add to it.) In my eyes this is a big enough gain, it would take away the original "why not use kss" reason. On the other hand, if we are over with the pageload, execution speed of kss is currently acceptable since there is nothing CPU hungry it's doing there. If not only the pageload/bind but the execution scheme of kss would also be compiled to javascript, that would not bring a huge additional speedup to that gained during the pageload compilation. It would further decrease code size (to zero if you want, although see below my note about libraries) and it would make the use of kss and native javascript code together much more easy and accessible which in my opinion would be the biggest goal. So while I agree on the long path, the reason why I suggested this as a direct focus of the GSoc topic is because of the direct benefit it brings. I also understand that you have different opinion about the "parser generator" approach and I accept this. Obviously it is not the only way. I think there are many different approaches that may be equally fruitful: my preference for solutions come from my experience (and challenges) with our custom made parser, and research that I did for solutions. > I think the end result should be a JS file full of jQuery, including > event bindings and the like. This is then executed as any other, > hand-written file full of jQuery. > > We would need jQuery equivalents of the current client-side KSS plugins, > of course. Many of those will be available natively in jQuery though. True, but let me rephrase in an other way: the generated output will contain javascript code _that calls other libraries_. Of those libraries there will be calls to the jquery lib, from plugins you will also call other (possibly non jquery core) libraries. Sometimes you will not be able to generate all code with just javascript and jquery library calls. You may also need some specific libraries to carry out a certain kss feature. You will have the choice of (a) using a library from kss (that first needs to be factored out for the purpose), or (b) write that functionality yourself, or (c) drop that feature from kss. I feel that you believe that the kss libraries can go away altogether, but it can happen only in one of the above mentioned ways. Some functionality of kss can be dropped and simplified, some needs to be used, and yes the code will be much much shorter in the end but there will be some. None of the code that is in kss, is there without a reason, it's all doing something. On the other hand, if the parser is written in a way that the code it emits can be customizable, we can hook up the parser to emit code interfacing with existing kss and we immediately get the (alleged) advantage of zero pageload time in kss and in plone with kss. After this the same parser can have more branches where only the emitted javascript differs. One version can work with the released version of kss, and there can be one or more branches that aim to arrive at full compilation. >> To summarize what this will immediately result, let me add a subtitle >> for the idea: >> >>> * Server side KSS compiler >> >> (Zero pageload overhead and smaller code for KSS) >> >> I would emphasize "zero pageload overhead", as this will be a >> substantial and immediate benefit. > > +1 > >> The parser generator would also generate a client side (javascript) >> parser from the same definitions. This would replace our existing >> parser and permit to maintain (and regression test) a single parser >> instead of two. > > I see the benefit, but writing a JavaScript code generator of that > complexity sounds pretty difficult. I think that would be a distraction > from the most immediately useful parts of this GSoC project. That really depends on the goals set and what you consider as most immediately useful parts. Note that I did not once disagree with the full path, just trying to higlight an opportunity here. p.s. I will reply to the rest of the posts continously, sorry if my work in current days causes a delay. -- Balazs Ree _______________________________________________ Kss-devel mailing list Kss-devel@codespeak.net http://codespeak.net/mailman/listinfo/kss-devel
