On 10/30/2012 06:48 AM, Endi Sukma Dewata wrote:
On 10/29/2012 4:27 AM, Petr Vobornik wrote:

I would like to make a bigger change in Web UI. Basically I think Web UI
would benefit from using a Dojo toolkit, a JS framework. I would like to
know if I can proceed with incorporating it. I think this is the best
time (end of 3.0/3.1 and a beginning of 3.2) to do such change.

I don't know about the timing, but I agree we need a better framework.
This was also discussed some time ago with Adam.

Why Dojo?
It may solve several existing issues in Web UI architecture/code

TLDR version:
* incorporates a lot of functionality we don't have to write
* is considered to be a framework for enterprise class web applications
* reuse an establish library which may be known to more developers then
custom build in-house solution
* makes code more maintainable
* opportunity to separate Web UI framework and IPA specific parts

Some functionality of Dojo overlaps with jquery which is not good, but
the reason to use Dojo are the features jquery lacks.
I also look on other frameworks or independent libraries: AngularJS,
backbone.js, closure js framework and several single-purpose libraries
(crossroads, hasher, underscore, amd-utils,. IMO Dojo is the best fit
for us.

I have not used Dojo or any of the above frameworks so I can't really
say anything good/bad about them, but backbone.js also seems to be a
popular lightweight alternative if we aren't planning to use the Dijit.
We should consider whether a framework will work nicely with others or
whether it will require us to use it for everything.

I fear that with backbone.js we would have to rewrite the whole UI to not bend the framework much. From looking at examples IDK how would we incorporate it. I think it would be a nice candidate if we would rewrite UI from scratch.

I like Dojo because we can make the changes gradual. In theory we can just utilize the things from dojo which we want and leave the rest untouched.

If possible, we should try to address the issues below independently
from each other (even if we end up using Dojo for everything) while
we're still learning about it. Some of the issues are actually general
design issues, so it won't be solved simply by converting the framework.
Try to avoid refactoring the code and converting the framework at the
same time because the patch could become very big and hard to review.

I agree. Originally I was just looking if there is some small library which has a reasonable implementation for bindable object store. To keep with saying: "best code is the one you don't have to write". I ended with Dojo because it can do a lot of stuff we might need in time. Of course all of dojo functionality can be replaced with other libraries but I think it's better to use one (and because of AMD modules, we can create small custom build) and not to worry about making them all work together.

 From Dojo I would like to use:

### 1) Build system and AMD modules
Web UI doesn't use any JavaScript optimizer therefore we need to keep
low file count and minimum size of files. We share one file for multiple
components to keep the file count low. It leads to very long files. It's
harder to maintain than separate files for each component. AMD modules
and a builder can solve it.

AMD modules [1] encourages to use one file per component. Then a dojo
builder can be used to modify the declaration of a module for file
concatenation then it can be sent to JavaScript minifier/compiler.

Dojo builder [2] is quite powerfull, it can define 'layers' which serves
as a part of application - basically a sets of component so an
application with ie. 150 files can be compile to one or more (if needed)
.js. Dojo loader takes care of the loading.

AFAIU build system can be run at rpm build time using rhino (version
1.7R3-4 needed). Uglify.js can be used as a compiler, it's JS library
and can be run in rhino too.  The builder and uglify.js would be needed
to budle in our source tarball but they won't be in output rpms so they
shouldn't break Fedora packaging rules. Both are BSD licensed. Builder
would need a patch to support Uglify (it's commited in dojo upstream
trac but I want to incorporated it in latest stable version).

[1] http://dojotoolkit.org/documentation/tutorials/1.8/modules/
[2] http://dojotoolkit.org/documentation/tutorials/1.8/build/

I suppose now if we want to test the static page without web server we
will need to build the pages first?

It's not necessary. For developing purposes it is better to use uncompiled files. Loader can load each separate file (if it's AMD module) if it isn't in bundle.

If we use Dojo do we need to include Dojo js files into IPA source, or
can we use the ones from Dojo rpm and only include it at build time?

I'm for including it because we can create a build with the parts we actually use. Dojo builder isn't in the rpm.

AFAIK the dojo rpm contains already built files. It has a core dojo.js file which contains basic modules to work but the rest of modules are separated. IDK if it can be used in a build process. I have to test it.

There's also RequireJS (http://requirejs.org/).

require.js is just AMD loader but it uses r.js as optimizer. r.js seems more lightweight than dojo builder. I would stick with dojo loader and builder if we use other dojo features.

### 2) Object store/model
At the beginning Web UI was quite simple - a command filled widgets,
widgets got edited, update command was constructed from widgets, it was
sent to  server and at the end widgets were updated with new values.

Now various parts of displayed page (action lists, panels, status
widgets, facet header) needs to use loaded values too or influence
others. This issue was often discussed in patch reviews. As a solution,
some model object with bidirectional binding capabilities would be
greatly beneficial.

Dojo provides a extensible object store [1]. I would like to use 'memory
store' and 'observable' and extend it with 'dirty-check' capabilities.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/intro_dojo_store/

Backbone also has similar mechanisms for events and sync.

### 3) Class model
Yes, in IPA Web UI we are using some class model, but it has flaws,
mainly in initialization of a object (some initialization code of parent
class may be called before inheritance is finished).

Dojo has a class system [1] which is easy to use and support
inheritance, mixins and overrided function calls.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/declare/

At first glance Backbone seems to be better in this area, but Dojo might
also be sufficient for our purposes.

Backbone seems to have advantage in calling super class' constructor. It is not recommended in Dojo.

There's going to be some challenges converting the current class
framework. Like you said, the current initialization isn't well
structured, so we might need to fix that first before converting the
class framework. And then we use 'that' instead of 'this', and sometimes
both, and they get confusing, so we'd have to be very careful when
converting them. Also, the current way to call a method in super class
needs to be fixed.

We can come with some strategy to keep patch reviews simple.
* move initialization stuff to init method (or more fine-grained) and use builder to create and instance (in some parts its already done), this way we eliminate statements in class declaration (except method declaration and overriding) * change to the classes: change that to this/self(to keep context in closures)
* eliminate method overrides with inherited() calls.
* put init calls to constructor
* change builder again

### 4) Localization
Web UI and pages in config, migrate and error folder are not localized
until successful login. It's because Web UI gets localized strings by
API call which needs prior authentication. We should use other method to
provide localized config, error and login pages.

Dojo has a easy to use localization solution [1]. Basically strings are
defined in .js files as modules, translations in subfolders. Dojo loader
and i18n framework take care of loading of proper string depending on
user's local. It doesn't need communication with JSON-RPC so can be used
in config and login pages.

We would  have to make a script to translate the format to a format
usable by transifex.

[1] http://dojotoolkit.org/documentation/tutorials/1.8/i18n/

A possible alternative is jquery-i18n-properties

### 5) Navigation
Our navigation code is limited to IPA entities. Introducing non-entity
page like log (#3040) or help is problematic. Routing code should be
improved. Separate entity initialization and menu definition. As the
re-factoring will be at some point done, we might consider to use
dojo.router and dojo.hash instead of $.bbq. This is really optional and
not strictly related to Dojo, but I wanted to mention it.

Better handling of pkeys may fix `#2741 [ipa webui] Intermittent errors
- 'cn' is required; limits exceeded for this query`

Backbone also has router and history.

### 6) Easier unit tests
By utilizing AMD modules, Localization, Model binding we might be able
to write unit tests of widgets easier. Now we have to simulate entity,
facets, fields, strings loaded from JSON-RPC...

Yes, that would be great.

### 7) Separation of framework and IPA
We often talk about separating Web UI framework and IPA related stuff.
We can do it along with this refactoring.

Yes, but it might need to be fixed separately.

I don't want to this all stuff in one patch :).

What I don't want to do
* rewrite whole UI, changes should be gradual
* replace jQuery. Most of our widgets are using jquery, I don't want to
change this code, so I don't want to replace it with Dojo widget system
- too big effort. So in the result the HTML output should not be changed
much - not to affect automated tests.

### Steps how could be Dojo incorporated
It should be done gradually.
1) make the build system working. Without it a lot of .js files would be
transmitted to browser which is really bad.

This is probably the easiest part. The next steps could be difficult to
untangle. So be ready for a long haul. :)

The harder part is not to populate git and rpm with a lot of third-party stuff. We need to convince dojo builder to use uglify.js under rhino to avoid using of closure compiler or shrink-safe which are not and probably will not be bundled in fedora and RHEL and definitely we don't want to bundle them.

I'm thinking about making a script which would create WebUI-devoloper enviroment:
 * checkout dojo/dojo and dojo/util git repository to 1.8.1 tag
 * create a symbolic links to the tag
 * apply custom patches on dojo/util (there will definitely be one)
* include in FreeIPA git only builded dojo.js (in the case we have separate files for dojo.js and freeipa.js (just a suggestion of name)) * will investigate if we can build the builder and include it in git for rpm builds.

We can of course include into our git a whole 1.8.1 dojo/dojo and dojo/util but I would rather avoid it because it's a LOT of files.

2) separate components to individual files and transform then into AMD
modules. We might change namespaces and dependencies to separate Web UI
framework and IPA related stuff.
3) change class system of entities and facets
4) implement a 'model'
At this point it depends on priorities and new tickets with higher
priority may appear.

Petr Vobornik

Freeipa-devel mailing list

Reply via email to