At 02:06 PM 11/1/2001 -0800, Tavis Rudd wrote:
>THE PROBLEM
>-------------------------------------------------------------------------------------------------------
>Webware's package layout / component system needs refactoring for
>the following reasons:
>
> * it relies too heavily on a particular directory structure, which
> will cause problems with a system-wide install via distutils.
The only real problem I'm aware of here is the mixture of directories that
get written to (Cache/ Sessions/ etc) into the code directory. I think
MakeAppWorkDir addresses some or all of that, but clearly some clean up
and/or better docs in this area would be nice.
> * it will lead to namespace pollution of PYTHONPATH when the
> transition is made to system-wide installs.
I'm not aware that individuals will need to modify their PYTHONPATH
environment variable. See more below.
> * it dictates too much about how components/plugins should be
> packaged and documented.
I disagree vehemently. The whole idea behind Webware components is that you
would know the answers to all of these questions immediately:
- where are the docs?
- where are the examples?
- where are the tests?
- what are the properties (version, author, dependencies, etc.)?
- will this package load on this platform?
...while still maintaining valid Python packages. e.g., even with all the
conventions I have added on, you still just dump your *.py code in a
directory with an __init__.py and then do this:
from MiddleKit.Core.Attr import Attr
> * some of the kits should really be part of WebKit's core. For
> example, COMKit functionality can easily be coded directly into
> the AppServer (or MultiPortServer in the redesign code I've
> been working on). It took 8 lines in the redesign code, rather
> than 93 lines in 2 extra files and 4 directories with COMKit.
> Ditto for TaskKit. The code for managing SessionStores in the
> redesign code was more compact, more efficient and much simpler
> without TaskKit.
I don't have a strong opinion on COMKit either way. Maybe Geoff, who wrote
it, does.
I strongly disagree about rolling TaskKit into WebKit. How would someone
use TaskKit independently of WebKit, if it were rolled into WebKit? I think
the answer is that they couldn't. Ditto for MiddleKit and UserKit.
Also, I'm not aware of anyone having difficulties using Webware due
to TaskKit being a separate package.
I will not allow WebKit to become a monolith. It's an extensible app
server. Things like task scheduling and object-relational-mapping will be
separate packages that focus on having their own docs, tests, examples,
independent usability, and WebKit integration.
> * my old objection to the name WebKit: for marketing and ease of
> introduction reasons it is better to position Webware as
> "A Python-Powered Application Server, that comes with a
> object-oriented library of useful components" than as
> "a suite of software components for developing object-oriented,
> web-based applications." WebKit is the heart of Webware, so
> why not get rid of the extra name to explain? Also see my
> proposed package structure below for more reasons.
The name WebKit does not prevent our marketing Webware in a certain way.
The name is final.
For an example of the discontinuity between marketing and reality, see
Microsoft. :-)
>Its sounds like Webware will eventually be packaged using distutils.
>At that time all the Webware components will be sitting in
>/usr/local/lib/Python-x.x/site-packages. Think of how MiscUtils,
>WebUtils, etc. will look there. Chuck has mentioned the use of a
>Webware.pth file to group all the Webware components into a 'Webware'
>sub-directory that gets added into PYTHONPATH; thus requiring no
>major change to the existing Webware directory structure. I'd argue
>that this is hack to avoid. Sure it's a quick solution, but it's
>really no different than putting all the Webware subdirs directly
>into site-packages. It solves none of the issues listed above.
I thought the *.pth was a feature of Python, not a bug. Other than your
personal distaste for it, I don't see any compelling reasons why it must be
avoided.
>Instead of just having 'Webware' in the PYTHONPATH namespace we'll
>have:
>[internally developed components]
>- WebKit
>- MiscUtils
>- WebUtils
>- UserKit
>- COMKit
>- TaskKit
>- MiddleKit
>- PSP
>[externally developed components]
>+ Cheetah
>+ FunFormKit
>+ etc.
I agree that after a site wide install, you should be able to do this:
from MiscUtils import NoDefault
and NOT this:
from Webware.MiscUtils import NoDefault
My plan is still to accomplish this with Webware.pth. Having the Webware
packages sit next to each other *is key*. It allows programs like WebKit's
Launch.py and MiddleKit's Generate.py to ensure that the MiscUtils and
other packages they use come from their own Webware package and not from
some other location.
For your average user, they don't care, but also won't be bothered or
burdened. For developers like me who may have more than one Webware on a
box, this is essential.
>THE SOLUTION
>-------------------------------------------------------------------------------------------------------
>1) Refactor the directory/package layout:
>Make all the components that are developed in the main Webware CVS
>subpackages of the main 'Webware' package:
>
>[internally developed components]
>- Webware (with all the WebKit classes at the top-level)
WebKit will NOT be put at the top level. The name will NOT be removed.
I'm final on that.
> Webware.MiscUtils
> Webware.WebUtils
> Webware.PSP
> Webware.UserKit
> Webware.MiddleKit
The names like "MiddleKit" will be accessible without "Webware", as in:
from UserKit import UserManager
This allows:
* easy assimiliation of SomeKit into Webware that was previously external
* easy break out of SomeKit into a separate project, if that is ever desired
>[externally developed components]
>+ Cheetah
>+ FunFormKit
>
>2) Refactor the component/plugin loading system:
>It currently operates through the file system. It is better to have
>the plugins loaded as live Python classes and leave where they come
>from upto the launcher script, or whatever is loading the system.
>This is much more flexible and suited for growth!
We already have this today. The settings in AppServer provide _you_ the
ability to dictate where plug-ins come from.
Regarding them being classes, plug-ins are Python packages. Nothing wrong
with that at all. Python packages are...well...Pythonic!
>3) Merge COMKit into the core as I've done in the redesign code I've
>been working on.
I'll let Geoff comment on that.
>4) Get rid of TaskKit. It's better to do this sort of stuff on a
>case-by-case basis, as illustrated by the implementation of
>SessionStore in the redesign code. Simpler, faster, smaller.
Sometime a case-by-case basis can be convenient, especially for the quick
crank. However, the Python package with docs, examples, tests, etc. tends
to promote maturity and group focus. But if you want to start a thread on
the virtues of using TaskKit vs. doing-it-yourself, feel free. I didn't
write TaskKit, haven't use it yet and didn't integrate it into WebKit so I
don't have much to say. However, the folks who did seemed to discover some
non-trivial issues and come up with a design they really liked.
I think that's mostly Jay and Tom. I think people at Geoff's company have
used TaskKit.
>ADVANTAGES
>-------------------------------------------------------------------------------------------------------
> * no namespace pollution: only 1 name is exposed in PYTHONPATH
> rather than 8!
I don't think names like "MiddleKit" and "UserKit" are pollution. Again,
this enables us to take out or put in new packages into Webware without any
pain. And who wants to type the extra long "from
Webware.MiddleKit.Core.Attr import Attr"? Not me.
BTW I thought it was a pain when "DateTime" became "mx.DateTime", for example.
> * simpler to explain and document
Well, now that depends on who you are. ;-)
When people have had trouble using UserKit and MiddleKit it has never been
due to them sitting next door to WebKit instead of underneath it.
> * more flexible as it uses Python's package system rather than
> relying on a particular directory layout.
Webware components *are* Python packages. If you don't believe me, start an
interactive prompt and import one!
But Python packages don't specify a standard for things like willLoad(), so
Webware _adds_ to the package. This does not burden Webware users.
> * easier to customize Webware for new uses
Only with regards to putting more focus on having Webware write it's
sessions, logs, etc. to a different location. Or at least an umbrella
directory like "WebKit/var/" but with a better name.
> * no mangling of PYTHONPATH or sys.path is required.
That's just not a big deal. If everyone had to modify their environment, I
would feel the pain. But the sitewide install should handle this transparently.
>QUESTIONS & ANSWERS
>-------------------------------------------------------------------------------------------------------
> Q: But won't this break existing code?
> A: No! I've written some code to make sure that import statements
> that rely on the existing structure will continue to work.
>
> Q: Where would the docs and examples go for the components?
> A: For the internally developed components they would go either in
> the main Users' Guide (my pref), or in the package dir of each
> component. NB: I've written some distutils code that allows
> the docs to be installed centrally in site-packages as
> 'data-files'.
>
> For externally developed components, such as Cheetah, they
> would go wherever the components' developers want to put them.
>
> Note, I'm also advocating that the Webware docs be written in
> LaTeX, like Python's and Cheetah's, so they can be output in
> multiple formats. Rather than having separate docs for each
> component, I'd suggest having a Beginners' Guide, an Users'
> Guide, and a Developers' Guide. Each internally developed
> component would have a chapter in the relevant Guides.
Again you are debunking the notion that these components can be used
independently. If you're concerned that people don't have easy access to
all Quick Starts and all User Guide's, then we can write an index page that
puts that together.
> Q: But won't it take too much effort to change?
> A: No, most of the work is already done. Besides a little effort
> now will pay off big-time in the future as Webware grows. It's
> better to do this sort of refactoring while Webware is still
> young.
We've discussed most of the above before. Of course, everyone is free to
discuss as much as they want, but at some point an official decision should
be made so we know where things will stand.
As the project manager for Webware, here are some final decisions:
These status quo items all stay:
- the name Webware
- the name WebKit
- WebKit as an app server focused on easy of use and extensibility
- Webware as an umbrella for Webware components
- a Webware component is simply a Python package with some additional
conventions
- a WebKit plug-in is simply a Webware component loaded at launch time
- if a WebKit plug-in needs to so some special set up, it does so via
__init__.py like any good Python package does
- It's "from SomeKit import SomeModule" and will not be "from
Webware.SomeKit import SomModule"
I agree that:
- Webware should have a setup.py
- Webware should make site-wide installation easy
- Webware should make virtual hosting easy
- Webware can always use better marketing including some good ideas from Tavis
- WebKit should be more amenable and encouraging to writing all output to
an umbrella directory that can easily live outside of WebKit/
- Webware needs a "Building a site" tutorial that takes the user through
the steps of building a site, starting with WebKit and incorporating
MiddleKit and UserKit. Basically, showing the canonical way to do things.
I think that:
- Webware programs like Generate.py and Launch.py need to use their own
Webware components (I know this from experience). The Webware/ umbrella
directory facilitates this.
- Users who develop sites are not feeling burdened by anything other than:
- lack of easy site wide install
- lack of virtual hosting
- the usual need for more docs, features and regression tests
- Rearranging the directory structure wrt to WebKit vs. Webware and
modularity, will not improve a Webware user's productivity.
I'm open to:
- more discussion on COMKit being external vs. internal to WebKit
- more discussion as to the virtues of TaskKit
^ I suggest these become discussion threads in their own right.
- focused, preferably-separate contribs like:
- setup.py
- WebKit's output directory is one, configurable umbrella directory
You already have:
- the ability to dictate what plug-ins are loaded or not loaded via
WebKit/Configs/AppServer.config
- the ability to develop kick ass sites with Webware (vorbis.com,
foreclosures.lycos.com) :-)
-Chuck
_______________________________________________
Webware-discuss mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/webware-discuss
