#2477: V3: Packager
------------------------------------+---------------------------------------
Reporter: fredck | Owner:
Type: New Feature | Status: new
Priority: Normal | Milestone: CKEditor 3.0
Component: General | Version:
Keywords: Confirmed V3ProtoOther |
------------------------------------+---------------------------------------
I've worked a few days to come out with a new packager. I've started it as
just a research thing, but the coded evolved into a real solution. For
now, the code has been committed into the "_dev/packager/ckpackager"
folder in the CKEditor prototype.
The basic idea is stop doing string manipulations to the source code with
miraculous regular expressions and use a real JavaScript parser. I've
chosen [http://www.mozilla.org/rhino/ Rhino] for it, because it also made
it possible to code all that stuff in our preferred language: JavaScript.
Rhino is a full JavaScript parser, interpreter and compiler. I was more
interested on the first part of it, the parser.
The results are pretty nice. Rhino gives us a "token tree". We then walk
over this three and re-write the source, token by token. So, for example,
if we find a VAR token, we just write "var" to the output. Under it, we
have the NAME token, which contains the variable name, so we rename and
write it to the source too. The results is "{{{var a;}}}", for example.
Another interesting fact is that the parser gives us the "interpreted"
token three. We don't have the exact same thing we had in the original
code. A simple example; if we have "{{{var a = 1 + 2;}}}" in the source,
the final result will be "{{{var a=3;}}}", because Rhino automatically
summed the numbers for us. Another example, "while(true){}" and
"for(;;){}" give us the same token three with Rhino, so we just output it
in our preferred way, as we really don't know how exactly the source was.
There are also several code enhancements that are done by this new
packager. Things we would never be able to achieve with the previous
packager. Check out the "test/test.js" file for a long list of examples.
Btw, this implementation contains a basic automated test system, so we can
be sure we are not breaking things on changes.
In the "ckpackager/_dev" folder, you will find two batch files. Both of
them are configured to act over the code available in the script.js file,
present in that folder also. Just run dump.bat to have a visual
presentation of the token three representation of that script. The
compress.bat file will instead print the compressed result.
There are several positive things with this packager. One of them is that
'''all''' pending tickets for FCKpackager have been solved with it, and no
ticket regressions have been found. Also, there are no special
requirements when coding. No problems with missing semicolons, or even
[ticket: 1242#comment:2 special ways to write things], like regular
expressions and division signs.
There are though a few negative things that need attention. The first is
the performance. It is twice as slow as the previous packager. This is not
a critical thing, and there is certainly room for enhancements here.
The other problem is that, this packager has been written based on coding
patterns. I think I have handled all syntax situations in the current
implementation, but, for example, I've written things like "{{{(success ?
completed : failed).push( 'thing' )}}}" yesterday, and this pattern was
not been handled. The code got broken. Fortunately I had some intuition
that it may happen, so I've just added a test case for that coding line
and coded the packager to make it work with it. So, to summarize, we may
find that the packager is breaking our code in the future, and we'll be
working to add all necessary cases to it.
For the configuration file, it is not using anymore an XML file. It uses
instead a JavaScript object literal like syntax. For CKEditor, the
"ckeditor.pack" file has been added to the root of the project. It
exemplifies the package file syntax.
Also, the _dev/packager/packagefilegen.html file has been updated to
generate both the old and the new package file contents automatically.
Just run _dev/packager/package_2.bat to execute the new packager over the
CKEditor code.
Regarding the deployment... Right now, the entire ckpackager folder is
needed to run it. Java is required. This is another negative thing, as we
just have a single file to be used with the previous packager to make it
work.
I haven't investigated it well, but I'm sure we are able to compile all
that stuff in a single .jar file for the deployment, and possibly even
generate an exe for it. But, this is something to understand yet.
To conclude, here are the numbers I have, by running both packagers over
the current CKEditor prototype (second run):
* FCKpackager:
* ckeditor_basic.js: 6,449 bytes (18.72% of original)
* ckeditor.js......: 45,821 bytes (21.38% of original)
* total time.......: 1.55 seconds.
* CKPackager 2.0:
* ckeditor_basic.js: 5,813 bytes (17% of original)
* ckeditor.js......: 44,199 bytes (21% of original)
* total time.......: 2.97 seconds.
If we think this new packager is the way to go for us, I'll move it to its
dedicated SVN three as a separated project and put an SVN external in the
prototype three pointing to it. I'll wait the ticket review for it.
--
Ticket URL: <http://dev.fckeditor.net/ticket/2477>
FCKeditor <http://www.fckeditor.net/>
The text editor for Internet
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
FCKeditor-Trac mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/fckeditor-trac
