Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-21 Thread Chris Withers

Hi Stuart,

Stuart Bishop wrote:

What were the problematic bits?


# Disgusting hack to use our extended config file schema rather than the
# Z3 one. TODO: Add command line options or other to Z3 to enable overriding
# this -- StuartBishop 20050406
from zdaemon.zdoptions import ZDOptions
ZDOptions.schemafile = os.path.abspath(os.path.join(
os.path.dirname(__file__), 'lib', 'canonical',
'config', 'schema.xml'))

Also, there is only one schema.xml so multiple components can't each insert
their own blob of configuration information into the global schema.


Okay, so I can see two potential problems here:

1. Zope 3's schema.xml has the same problem that Zope 2's used to have - 
no generic multisection where other frameworks/products/whatever can 
insert their own bits of configuration. I suspect fixing that schema.xml 
(which involves inserting one or two lines ;-) would remove the need for 
your monkeypatch above.


2. Are you aware of the component.xml stuff that Dieter referred to?


We lost a fair bit of flexibility doing it this way. Field validation
needs
to be done the ZConfig way.

How would you prefer to do it?


Validation of the entire config file, and if there are one or more errors
output a readable report at the end with error messages returned by the
validators. The current mechanism just spits out exceptions, which is really
bad for configuration files aimed at end users.


Agreed, this is something that could be added to ZConfig. Or Zope 3 
could catch those exceptions and morph them into useful messages. 
Admittedly, you wouldn't get all the errors though, so ZConfig could do 
with some enhancement here...



If I have a non-required section foo, containing a non-required key bar, if
I try and access config.foo.bar I get an AttributeError. I should get the
default value for bar.


Okay, probably a bug. Have you reported this to the ZConfig author(s)?


My main gripe with the .ini format is the lack of hierarchy, but then I
worry that with XML we'll suffer from an overly complex schema...


Why would you want a schema for the XML?


xml freaks like schemas ;-)

Although I actually probably should have said I worry that we'll end up 
with overly complex and badly structured xml *cough*zcml*cough*...



  - validation handlers would be registered for a particular XML namespace


...yay! we love namespaces *sigh*


  - Config file is loaded into a set of data structures, one for
each XML namespace


That's not a bad idea though...


  - warnings are emitted if there are XML namespaces loaded that don't have
a validator.


Why? If people don't want validators, don't force them...


Of course, .ini would be able to emit more meaningfull error messages:
foo/bar/1 in section [whatever] is required, but not found
blah/whatever in section [whatever] is not a valid url
section [baz] is required but does not exist.


I'm all for just fixing the bugs in ZConfig and I think you'll be happy 
enough. Not sure it's worth the huge upheavels :-S


cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-20 Thread Dieter Maurer
Stuart Bishop wrote at 2006-3-20 10:38 +0700:
Also, there is only one schema.xml so multiple components can't each insert
their own blob of configuration information into the global schema.

Please read

 From: Tres Seaver [EMAIL PROTECTED]
 To: zope3-dev@zope.org
 Subject: [Zope3-dev] Re: httpgz in zope.conf?
 Date: Mon, 20 Mar 2006 09:53:52 -0500

There, you will learn that components don't need to insert anything
into the global schema but can put these things into their local
components.xml.

-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-19 Thread Stuart Bishop
Chris Withers wrote:
 Stuart Bishop wrote:
 the Z3 configuration. This was with Zope 3.0 and integrating our
 config with
 the Z3 config was quite problematic.
 
 What were the problematic bits?

# Disgusting hack to use our extended config file schema rather than the
# Z3 one. TODO: Add command line options or other to Z3 to enable overriding
# this -- StuartBishop 20050406
from zdaemon.zdoptions import ZDOptions
ZDOptions.schemafile = os.path.abspath(os.path.join(
os.path.dirname(__file__), 'lib', 'canonical',
'config', 'schema.xml'))

Also, there is only one schema.xml so multiple components can't each insert
their own blob of configuration information into the global schema.

 We lost a fair bit of flexibility doing it this way. Field validation
 needs
 to be done the ZConfig way.
 
 How would you prefer to do it?

Validation of the entire config file, and if there are one or more errors
output a readable report at the end with error messages returned by the
validators. The current mechanism just spits out exceptions, which is really
bad for configuration files aimed at end users.

 There are issues with non-required fields in
 non-required sections giving us grief. 
 
 Can you describe these a little?

If I have a non-required section foo, containing a non-required key bar, if
I try and access config.foo.bar I get an AttributeError. I should get the
default value for bar. This causes noise in our config files and a
maintenance burden where we have to include non-required sections just so
the default values of keys will be accessible.

 So +1 I guess. Although I personally would prefer using XML, as I
 think it
 will be more readable for complex configurations as it is better able to
 represent heirarchies and provide more flexibility to developers.
 
 My main gripe with the .ini format is the lack of hierarchy, but then I
 worry that with XML we'll suffer from an overly complex schema...

Why would you want a schema for the XML?
  - The config file could just be arbitrary well formed XML.
  - validation handlers would be registered for a particular XML namespace
  - Config file is loaded into a set of data structures, one for
each XML namespace
  - warnings are emitted if there are XML namespaces loaded that don't have
a validator.
  - validators are called, giving them a chance validate and
convert values to native types. They are allowed to raise an error if
they find stuff they don't know about lurking in their namespace.
  - the resulting datastructures are available via a thread global

For sane config data, I don't think the sucky bits of XML need to be worried
about. It is pretty unlikely that you need to include  or  symbols or
wierd Unicode hieroglyphs in your configuration data (but if you do, at
least you already know how to insert it because we all know enough XML now
to do this).

Of course, .ini would be able to emit more meaningfull error messages:
foo/bar/1 in section [whatever] is required, but not found
blah/whatever in section [whatever] is not a valid url
section [baz] is required but does not exist.

-- 
Stuart Bishop [EMAIL PROTECTED]
http://www.stuartbishop.net/




signature.asc
Description: OpenPGP digital signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-14 Thread Chris Withers

Stuart Bishop wrote:

the Z3 configuration. This was with Zope 3.0 and integrating our config with
the Z3 config was quite problematic.


What were the problematic bits?


We lost a fair bit of flexibility doing it this way. Field validation needs
to be done the ZConfig way.


How would you prefer to do it?


There are issues with non-required fields in
non-required sections giving us grief. 


Can you describe these a little?


There seem to be namespace issues,
despite being hierarchical (eg. we have librarianlibrarian_server
instead of just librarianserver, it seems because server is already
used by Z3 so we can't call our section that).


Hmm, that's odd :-S


having are our fault, I would also call that a problem with ZConfig as the
documentation is not detailed enough to match ZConfigs complexity. 


Yep, I'd agree with that. ZConfig is a tool that's very easy to misuse :-/


So +1 I guess. Although I personally would prefer using XML, as I think it
will be more readable for complex configurations as it is better able to
represent heirarchies and provide more flexibility to developers.


My main gripe with the .ini format is the lack of hierarchy, but then I 
worry that with XML we'll suffer from an overly complex schema...


config sucks, we should all just go to the pub...

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-14 Thread Jim Fulton

Chris Withers wrote:

Stuart Bishop wrote:

the Z3 configuration. This was with Zope 3.0 and integrating our 
config with

the Z3 config was quite problematic.



What were the problematic bits?

We lost a fair bit of flexibility doing it this way. Field validation 
needs

to be done the ZConfig way.



How would you prefer to do it?


Budding in:

It would be nice to be able to use Zope schema for
conversion and validation.  ZConfig was developed at
around the same time as Zope schema.  The ZConfig developers
fealt they couldn't wait and reuse the work on zope.schema.
and developed their own schema  system.  It sucks to have
to maintain 2.  It's also a pain for people who might want
to use ZConfig's to have to figure out another.  ZConfig
obviously fits your brain. It doesn't fit mine at all.

JIm

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-14 Thread Andreas Jung



--On 14. März 2006 06:23:33 -0500 Jim Fulton [EMAIL PROTECTED] wrote:


Budding in:

It would be nice to be able to use Zope schema for
conversion and validation.  ZConfig was developed at
around the same time as Zope schema.  The ZConfig developers
fealt they couldn't wait and reuse the work on zope.schema.
and developed their own schema  system.  It sucks to have
to maintain 2.  It's also a pain for people who might want
to use ZConfig's to have to figure out another.  ZConfig
obviously fits your brain. It doesn't fit mine at all.



Could you please explain how zope.schema would deal with hierarchies?
As I mentioned earlier the file format is uninteresting at this point.
Having an easy and flexible framework for defining a configuration schema
should be the goal and I have currently no idea how you would define
some more complex configuration schemas using zope.schema.

-aj


   ---
  -   Andreas JungZOPYX Ltd.  Co KG-
 -   E-mail: [EMAIL PROTECTED]   Web: www.zopyx.com, www.zopyx.de -
  ---


pgpOYKkAlIFTj.pgp
Description: PGP signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-14 Thread Stephan Richter
On Tuesday 14 March 2006 06:29, Andreas Jung wrote:
 Could you please explain how zope.schema would deal with hierarchies?
 As I mentioned earlier the file format is uninteresting at this point.
 Having an easy and flexible framework for defining a configuration schema
 should be the goal and I have currently no idea how you would define
 some more complex configuration schemas using zope.schema.

I have used schemas multiple times to represent hierarchies, most recently in 
the zf.zscp work, where I parse XML files using a schema or a set thereof. 
You simply use the Object field to connect a sub-schema to a schema. I have 
also used hierarchical schema trees in the principal settings code, where you 
hook in dynamically new schemas.

Regards,
Stephan
-- 
Stephan Richter
CBU Physics  Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-14 Thread Andreas Jung



--On 14. März 2006 07:27:54 -0500 Jim Fulton [EMAIL PROTECTED] wrote:



I have used schemas multiple times to represent hierarchies, most
recently in  the zf.zscp work, where I parse XML files using a schema or
a set thereof.  You simply use the Object field to connect a sub-schema
to a schema. I have  also used hierarchical schema trees in the
principal settings code, where you  hook in dynamically new schemas.


Exactly.



Assuming the fact that we could specify ZConfig-style and INI-style 
configuration files using nested zope.schemaswho is now the winner? :-)


-aj

pgpJaak8d7USs.pgp
Description: PGP signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Chris Withers

Jim Fulton wrote:

I don't want to try to make paste deploy or setuptools,
use ZConfig.  There are other tools out there that use
ConfigParser,


Yay! Lowest common denominator programming :-(


I'd like to be able to use configuration files for the
test runner, but I really don't want ZConfig to be a
dependency of the test runner.  I also don't want to
go through all of the gymnasics required to develop a ZConfig
schema just for the test runner.


What are these gymnastics you speak of?

This isn't all that common. 


Hmmm, the number of DeprecationWarnings I seem to see with Zope 
3-related stuff would appear to disagree with that...


...but maybe that's another thread.


Note that in my proposal, I proposed supporting the ZConfig
format, at least for a while.


Isn't there some way we could make this switchable so we don't have to 
make either/or choices?

(I'd still be verymuch in favour of seeing ZConfig as the default.)


I assume this was done because it's too much of a PITA to write ZConfig
schemas.


I think you assume wrong there...


Paste Deploy provides a framework for doing
this.  I'd rather colaborate on something that exists that
have to reinvent it just to keep using ZConfig.


Well, maybe they could use ZConfig? Has anyone asked them?
Either that or make the config switchable for the edge case where you 
want to use paste deploy. But please, don't force a change like this 
through for one particular piece of interoperability that not many of us 
seem to feel as passionately about as you :-S


Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Chris Withers

Jim Fulton wrote:

Would it be possible to write a configuration file that loaded
it's own schemas?  


Not sure what you mean...


For example, suppose I wanted to configure
zope and twisted, could I do something like:

  import zope.app.appsetup
  import zope.app.twisted
  import zope.testrunner


I think it's %import you're looking for.


  site-definition site.zcml


%import site.conf? ;-)


  interrupt-check-interval 200

  server
type HTTP
address 8080
  /server

  zodb
filestorage
  path Data.fs
/filestorage
  /zodb

  accesslog

logfile
  path access.log
/logfile
  /accesslog

  eventlog

logfile
  path z3.log
/logfile

logfile
  path STDOUT
/logfile
  /eventlog

  tests-pattern f?tests$
  tests-path src
  module-pattern 
!^(ZConfig|BTrees|persistent|ThreadedAsync|transaction|ZEO|ZODB|twisted|zdaemon|zope[.]testing|)[.] 


Okay, not sure what the troublesome bit of the above is...


Then free the main program from having
to specify a schema?


Again, not sure what you mean...

cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Jim Fulton

Chris Withers wrote:

Jim Fulton wrote:





I'd like to be able to use configuration files for the
test runner, but I really don't want ZConfig to be a
dependency of the test runner.  I also don't want to
go through all of the gymnasics required to develop a ZConfig
schema just for the test runner.



What are these gymnastics you speak of?


Have you written a ZConfig schema?  Have you tried to read the
documentation on writing one? Have you writtem an application
that uses ZConfig? If you had, I think you'd know what I was
talking about.

This isn't all that common. 



Hmmm, the number of DeprecationWarnings I seem to see with Zope 
3-related stuff would appear to disagree with that...


These aren't due to new features added and discarded.  This is
generally due to refinement based on experience.

...


Note that in my proposal, I proposed supporting the ZConfig
format, at least for a while.



Isn't there some way we could make this switchable so we don't have to 
make either/or choices?

(I'd still be verymuch in favour of seeing ZConfig as the default.)


I don't know what you are asking, since I said in this thread and in
the proposal that we could support the old format.


I assume this was done because it's too much of a PITA to write ZConfig
schemas.



I think you assume wrong there...


Oh? What evidence do you have?  Do you think it was done because
meaningless options are considered a good thing?


Paste Deploy provides a framework for doing
this.  I'd rather colaborate on something that exists that
have to reinvent it just to keep using ZConfig.



Well, maybe they could use ZConfig? Has anyone asked them?


Why don't you do that.  In fact, why don't you lobby to
have it added to the standard library?

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org


___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Chris Withers

Jim Fulton wrote:
Have you written a ZConfig schema? 


Yup, a few, as well as adding a couple of options to zope.conf, which 
totally abuses ZConfig :-/



Have you tried to read the
documentation on writing one? 


Yup :-)


Have you writtem an application
that uses ZConfig? 


Yup, couple...


If you had, I think you'd know what I was
talking about.


Well, admittedly, it's been a while since I wrote the schemas for 
MailingLogger (although that had to be updated for Zope 2.8) and an even 
longer while for Process 
(http://www.simplistix.co.uk/software/applications/process) and X2Y 
(http://www.simplistix.co.uk/software/applications/x2y) but I don't 
remember it being particularly painful.


X2Y, in particular, is quite an extensive app with lots of plugins, each 
with their own schemas, and I just had a look at the schemas again now 
and find them pretty easy to understand.


I must be missing something... is there something particularly complex 
that you're doing?



These aren't due to new features added and discarded.  This is
generally due to refinement based on experience.


potato / potato - tomato / tomato...

Hmmm, that doesn't work so well when it's written down :-/


I don't know what you are asking, since I said in this thread and in
the proposal that we could support the old format.


OK.


I assume this was done because it's too much of a PITA to write ZConfig
schemas.


I think you assume wrong there...


Oh? What evidence do you have?  Do you think it was done because
meaningless options are considered a good thing?


I would assume it's because someone saw an abstraction that wasn't 
really there, as with cache managers in Zope 2.


But, that's just an assumption and likely to be wrong. svn blame may be 
more useful here...



Well, maybe they could use ZConfig? Has anyone asked them?


Why don't you do that.  


I presume 
http://www.webwareforpython.org/archives/list/paste-users.en.html would 
be the right place?



In fact, why don't you lobby to
have it added to the standard library?


Sure, I'll give it a go, but if there's already several in there, I can 
see people being reluctant, even if the ones that are in there are 
inferior :-/


Still, Guido did eventually admit HTMLParser.py, so there's some hope :-)

cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Stuart Bishop
Jim Fulton wrote:

 Have you written a ZConfig schema?  Have you tried to read the
 documentation on writing one? Have you writtem an application
 that uses ZConfig? If you had, I think you'd know what I was
 talking about.

I have. Our chunk of our schema.xml is over 600 lines including comments and
whitespace and defines 22 sections.

We needed a configuration mechanism for Launchpad and related tools. I
decided to go with ZConfig to integrate our configuration with the rest of
the Z3 configuration. This was with Zope 3.0 and integrating our config with
the Z3 config was quite problematic.

We gained a little doing it this way. Having the Z3 configuration with our
own configuration information made is easy to maintain the multiple configs
we need to maintain (several production configs and developer workstation
configs, maintained in the RCS alongside the code). In theory, we could
generate documented configuration templates but we haven't done that -- we
just maintain the commends in 'default' configuration.

We lost a fair bit of flexibility doing it this way. Field validation needs
to be done the ZConfig way. There are issues with non-required fields in
non-required sections giving us grief. There seem to be namespace issues,
despite being hierarchical (eg. we have librarianlibrarian_server
instead of just librarianserver, it seems because server is already
used by Z3 so we can't call our section that). These issues might be ZConfig
in Z3.0 issues, or problems with how we are using it. If the issues we are
having are our fault, I would also call that a problem with ZConfig as the
documentation is not detailed enough to match ZConfigs complexity. Pulling
in configuration information should be light weight, but I find ZConfig puts
too much burden on developers for little gain. Despite being very complex,
it seems to lack features that we need or the features don't work the way we
need them too (eg. we really need to say 'this config file is identical to
that one, except these fields are changed. Extending ZConfig to do this did
not seem fun.).

An alternative would have been a .ini or xml format similar to what you have
already described. I elected to proceed with ZConfig to see how we went and
to integrate our config with Zope3's. In hindsight, I think we would have
had a simpler, more flexible and more easily extensible system if we had
gone with .ini or xml instead.

Migration for us shouldn't be a problem, we access to our configuration is
via a simple API so we can replace the guts. I expect to reimplement our
config machinery when I get a chance to work on low priority tasks (we have
issues with what we have, but it works).

So +1 I guess. Although I personally would prefer using XML, as I think it
will be more readable for complex configurations as it is better able to
represent heirarchies and provide more flexibility to developers.

-- 
Stuart Bishop [EMAIL PROTECTED]
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-13 Thread Dieter Maurer
Jim Fulton wrote at 2006-3-12 15:54 -0500:
Dieter Maurer wrote:
 Jim Fulton wrote at 2006-3-11 18:03 -0500:
 
...
Where is this documented?
 
 
 I do not know. I saw a feature description in the mailing list.
 Fred and Tres (the authors) should be able to tell you whether
 there is a formal documentation and where you can find it.
 
 
Let's pursue this a bit.

Would it be possible to write a configuration file that loaded
it's own schemas?
 
 
 Yes -- with some restrictions: as I described in my previous mail.
 
   The feature essentially works as follows:
 
  You have an abstract section type in your primary
  schema, usually usable in a multisection.
 
Examples: ZServer server, ZODB storage, the general extension
abstract section type
 
  In your module/package, you define your own section type
  implementing the abstract section type in its component.xml.
  You are completely free in its keys and subsections.
 
  In the configuration file, you import your module/package (which
  makes available the definitions in its component.xml) and
  instantiate one or more of the section types defined there.
 
  Your application/module looks for the sections of its type
  and uses them.
 
   The restriction: you cannot have new keys on the top level -- all
   must be nested in a section type defined by your component.xml.
   But, this, I consider an advantage (ZConfig here behave similar
   to a ConfigParser approach).

So the example I gave won't work.  In a schema, I have to have an
abstract type for each thing I might want to add.  But I don't know what
I want to add.  This doesn't sound very extensible.

You can have a *SINGLE* abstract type -- fixed for any extensions
you may imagine and use this as the hook in your component.xml.

Such a single abstract type was added to the Zope schema to
make almost arbitrary extensions possible.

I can't fathom the ZConfig documentation so I don't really know what
an abstract type is.

It is essentially a name, which can be used at various places
in your schema and implemented at other places in different ways.

 I don't know how restrictive it has to be.
Can I define an abstract type that matched anything?

Yes. I even think you currently cannot impose restrictions on
the sectiontypes claiming to implement the abstract type.

Can I define a schema that just defines an abstract type that
matches anything?

I think you must also use the abstract type in a section/multisection
definition. But that would be all.



-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-12 Thread Dieter Maurer
Jim Fulton wrote at 2006-3-11 18:03 -0500:
 ...
Where is this documented?

I do not know. I saw a feature description in the mailing list.
Fred and Tres (the authors) should be able to tell you whether
there is a formal documentation and where you can find it.

Let's pursue this a bit.

Would it be possible to write a configuration file that loaded
it's own schemas?

Yes -- with some restrictions: as I described in my previous mail.

  The feature essentially works as follows:

 You have an abstract section type in your primary
 schema, usually usable in a multisection.

   Examples: ZServer server, ZODB storage, the general extension
   abstract section type

 In your module/package, you define your own section type
 implementing the abstract section type in its component.xml.
 You are completely free in its keys and subsections.

 In the configuration file, you import your module/package (which
 makes available the definitions in its component.xml) and
 instantiate one or more of the section types defined there.

 Your application/module looks for the sections of its type
 and uses them.

  The restriction: you cannot have new keys on the top level -- all
  must be nested in a section type defined by your component.xml.
  But, this, I consider an advantage (ZConfig here behave similar
  to a ConfigParser approach).

-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-12 Thread Jim Fulton

Dieter Maurer wrote:

Jim Fulton wrote at 2006-3-11 18:03 -0500:


...
Where is this documented?



I do not know. I saw a feature description in the mailing list.
Fred and Tres (the authors) should be able to tell you whether
there is a formal documentation and where you can find it.



Let's pursue this a bit.

Would it be possible to write a configuration file that loaded
it's own schemas?



Yes -- with some restrictions: as I described in my previous mail.

  The feature essentially works as follows:

 You have an abstract section type in your primary
 schema, usually usable in a multisection.

   Examples: ZServer server, ZODB storage, the general extension
   abstract section type

 In your module/package, you define your own section type
 implementing the abstract section type in its component.xml.
 You are completely free in its keys and subsections.

 In the configuration file, you import your module/package (which
 makes available the definitions in its component.xml) and
 instantiate one or more of the section types defined there.

 Your application/module looks for the sections of its type
 and uses them.

  The restriction: you cannot have new keys on the top level -- all
  must be nested in a section type defined by your component.xml.
  But, this, I consider an advantage (ZConfig here behave similar
  to a ConfigParser approach).


So the example I gave won't work.  In a schema, I have to have an
abstract type for each thing I might want to add.  But I don't know what
I want to add.  This doesn't sound very extensible.

I can't fathom the ZConfig documentation so I don't really know what
an abstract type is.  I don't know how restrictive it has to be.
Can I define an abstract type that matched anything?
Can I define a schema that just defines an abstract type that
matches anything?

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-11 Thread Jim Fulton

Dieter Maurer wrote:

Fred Drake wrote at 2006-3-6 14:16 -0500:


...
When Tres and I added this, we planned specifically to see how it was


received by the Zope 2 community.

At least, I like it.



...
That said, I don't think Jim's concerns are limited to the Zope
configuration schema, but extend to configurations that do not
necessarily involve the Zope application server at all.



But, you can easily add a hook like this to any schema.


I don't find anything involving ZConfig schemas to be easy.

In any case, I don't expect Paste Deploy to adopt ZConfig anytime soon.
Ditto for setuptools/distutils.

Similarly, I might want to be able to use config files for the Zope test
runner.  I don't want to depend on ZCondig for this.

You haven't responded to my desire to use a single configuration file
for multiple applications.  Perhaps because you don't consider this
to be important, which is understandable.  I do consider this to be
important though.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-11 Thread Benji York

Dieter Maurer wrote:

You can do this [inter-app sharing of config files] with ZConfig --
provided you describe the config with one schema


Jim's after sharing config files between things like Paste, TurboGears, 
Django, etc. as well as Zope-centric projects.  IOW, lots of Python 
projects use ConfigParser-like syntax and we want to inter-operate with 
them.

--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-11 Thread Jim Fulton

Dieter Maurer wrote:

Jim Fulton wrote at 2006-3-11 11:35 -0500:


...
You haven't responded to my desire to use a single configuration file
for multiple applications.  Perhaps because you don't consider this
to be important, which is understandable.



You guessed right.



I do consider this to be
important though.



You can do this with ZConfig -- provided you describe the config with
one schema -- eased by the extension mechanism implemented by
Fred and Tres.


I don't want to try to make paste deploy or setuptools,
use ZConfig.  There are other tools out there that use
ConfigParser,

I'd like to be able to use configuration files for the
test runner, but I really don't want ZConfig to be a
dependency of the test runner.  I also don't want to
go through all of the gymnasics required to develop a ZConfig
schema just for the test runner.


My main driving force to keep ZConfig is the wish for stability.
I have the impression that in Zope3 land quite some things are
introduced and shortly thereafter removed/deprecated again.


This isn't all that common.  But, being able to try things
out and learn from what doesn't work is a lot better than
sticking with something soley for stability.

Note that in my proposal, I proposed supporting the ZConfig
format, at least for a while.


In contrast, I have the tendancy to keep things as long
as they are usable.


Sure. As a developer, I just don't find ZConfig to be very
usable.  I think this ends up limiting what we can do for
end users. A good example is the way that the trace logging
support in Zope 2 reuses the logging configuration schema.
This provides options that don't make sense for the trace logger.
I assume this was done because it's too much of a PITA to write ZConfig
schemas.

I think Zope users would benefit from being able to mix and
Zope with other WSGI applications, servers, and middleware.
Paste Deploy provides a framework for doing
this.  I'd rather colaborate on something that exists that
have to reinvent it just to keep using ZConfig.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-11 Thread Jim Fulton

Dieter Maurer wrote:

Jim Fulton wrote at 2006-3-5 13:56 -0500:


...
Why do you think it's better to have to create a monolithic schema for all
applications bits that want to use the configuration file, rather than letting
individual applications define how to read their own data independently?



You already can do this now with ZConfig -- no need for
a monolithic schema:

  I recently implemented a ZServer based NNTP server.
  There was no monolithic schema required. The new sectiontype
  was described in NNTPServer/component.xml, the configuration
  file simply imported the package and decribed the NNTP-Server.


Formerly, this was possible only for a few abstract types
with corresponding multisections (such as servers, storages, ...).

In modern Zope[2] schemas, there is a general purpose abstract type
precisely for this kind of extensions.


Where is this documented?

Let's pursue this a bit.

Would it be possible to write a configuration file that loaded
it's own schemas?  For example, suppose I wanted to configure
zope and twisted, could I do something like:

  import zope.app.appsetup
  import zope.app.twisted
  import zope.testrunner

  site-definition site.zcml

  interrupt-check-interval 200

  server
type HTTP
address 8080
  /server

  zodb
filestorage
  path Data.fs
/filestorage
  /zodb

  accesslog

logfile
  path access.log
/logfile
  /accesslog

  eventlog

logfile
  path z3.log
/logfile

logfile
  path STDOUT
/logfile
  /eventlog

  tests-pattern f?tests$
  tests-path src
  module-pattern 
!^(ZConfig|BTrees|persistent|ThreadedAsync|transaction|ZEO|ZODB|twisted|zdaemon|zope[.]testing|)[.]


Then free the main program from having
to specify a schema?

If so, how do I make this so?  Being able to do this *soon* would help
me make progress on simpler Zope packaging and simpler server support.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-07 Thread Dieter Maurer
Fred Drake wrote at 2006-3-6 14:16 -0500:
 ...
When Tres and I added this, we planned specifically to see how it was
received by the Zope 2 community.

At least, I like it.

 ...
That said, I don't think Jim's concerns are limited to the Zope
configuration schema, but extend to configurations that do not
necessarily involve the Zope application server at all.

But, you can easily add a hook like this to any schema.

-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-06 Thread Dieter Maurer
Jim Fulton wrote at 2006-3-5 13:56 -0500:
 ...
Why do you think it's better to have to create a monolithic schema for all
applications bits that want to use the configuration file, rather than letting
individual applications define how to read their own data independently?

You already can do this now with ZConfig -- no need for
a monolithic schema:

  I recently implemented a ZServer based NNTP server.
  There was no monolithic schema required. The new sectiontype
  was described in NNTPServer/component.xml, the configuration
  file simply imported the package and decribed the NNTP-Server.


Formerly, this was possible only for a few abstract types
with corresponding multisections (such as servers, storages, ...).

In modern Zope[2] schemas, there is a general purpose abstract type
precisely for this kind of extensions.

-- 
Dieter
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-06 Thread Fred Drake
On 3/6/06, Dieter Maurer [EMAIL PROTECTED] wrote:
 In modern Zope[2] schemas, there is a general purpose abstract type
 precisely for this kind of extensions.

When Tres and I added this, we planned specifically to see how it was
received by the Zope 2 community.  If people liked the way this was
done, it could be added to Zope 3.

That said, I don't think Jim's concerns are limited to the Zope
configuration schema, but extend to configurations that do not
necessarily involve the Zope application server at all.

It may be that a better foundation schema is something to experiment
with, especially for non-Zope applications.

I don't expect to have any time for this in the near future.


  -Fred

--
Fred L. Drake, Jr.fdrake at gmail.com
There is no wealth but life. --John Ruskin
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-06 Thread Chris Withers

Jim Fulton wrote:


At:

  http://dev.zope.org/Zope3/UseConfigParserForHighLevelConfiguration


Sorry, working offline, can't check, hope I haven't missed anything 
crucial :-S



Is a proposal for using ConfigParser, rather than ZConfig for high-level
configuration.

Comments welcome.


This smacks of re-invention for the sake of it. I happen to like ZConfig 
 a lot. The way it's used in Zope 2 is a little unpleasant, admittedly, 
but I think it makes a fine configuration language.


The xml required for the schema is pretty trivial if you're coming from 
zcmhell...


I must be missing some big reasons why you'd want to use an inferior 
config language and rewrite a big chunk of working, battle tested 
software, but I guess that's in the proposal..


Anyway, a big -1 from me, especially given the backwards compatibility 
issues prople like myself, with MailingLogger and a number of 
ZConfig-based, and Dieter, with his NNTP server and doubtless other 
things, will face.


Fred, FWIW, I was really really happy to see that generic section go 
into the schema, I just haven't had a chance to use it yet...


cheers,

Chris

--
Simplistix - Content Management, Zope  Python Consulting
   - http://www.simplistix.co.uk

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Andreas Jung



--On 4. März 2006 21:26:30 -0500 Jim Fulton [EMAIL PROTECTED] wrote:



At:

   http://dev.zope.org/Zope3/UseConfigParserForHighLevelConfiguration

Is a proposal for using ConfigParser, rather than ZConfig for high-level
configuration.

Comments welcome.




-1

The right way would be to refactor ZConfig and decouple it in a reasonable 
way from its dependencies. ZConfig is a very smart module to define 
configurations and to verify configuration files. Replacing ZConfig with a 
very dumb format and moving more complexity (verification, argument 
conversion etc.= into the app layer is the wrong way. Apps should not have 
the need to deal with such low-level issues.


-aj



pgpGNV4s1KHzI.pgp
Description: PGP signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Jim Fulton

Andreas Jung wrote:



--On 4. März 2006 21:26:30 -0500 Jim Fulton [EMAIL PROTECTED] wrote:



At:

   http://dev.zope.org/Zope3/UseConfigParserForHighLevelConfiguration

Is a proposal for using ConfigParser, rather than ZConfig for high-level
configuration.

Comments welcome.




-1

The right way would be to refactor ZConfig and decouple it in a 
reasonable way from its dependencies.


I think this would be a major rewrite.

 ZConfig is a very smart module to
define configurations and to verify configuration files. Replacing 
ZConfig with a very dumb format and moving more complexity 
(verification, argument conversion etc.= into the app layer is the wrong 
way. Apps should not have the need to deal with such low-level issues.


They have to deal with it now, but now it's really hard.  I think that a simpler
approach would allow much simpler configuration support.  To extend ZConfig now,
you have to create XML schema descriptions, and have deep knowledge of how
ZConfig works.

Why do you think it's better to have to create a monolithic schema for all
applications bits that want to use the configuration file, rather than letting
individual applications define how to read their own data independently?

There could still be frameworks to make handling configuration data easier.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Jim Fulton

Stephan Richter wrote:

On Saturday 04 March 2006 21:26, Jim Fulton wrote:


  http://dev.zope.org/Zope3/UseConfigParserForHighLevelConfiguration

Is a proposal for using ConfigParser, rather than ZConfig for high-level
configuration.

Comments welcome.



I am +1. Anything that allows us to decrease our code base and use a standard 
tool is good. We can always have a schema-verifying layer on top of 
ConfigParser. Validating the fields is currently the biggest advantage of 
ZConfig. 


And, now that you mention it, the schema system used by ZConfig is unique
to ZConfig. For historical reasons, it doesn't use zope.schema.  It would be
fairly easy to provide a simple schema system for ConfigParser data based
on zope.schema for verification and conversion.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Andreas Jung



--On 5. März 2006 13:56:38 -0500 Jim Fulton [EMAIL PROTECTED] wrote:



The right way would be to refactor ZConfig and decouple it in a
reasonable way from its dependencies.


I think this would be a major rewrite.


Possibly but I don't consider that to be a strong argument for introducing
a weaker mechanism.


They have to deal with it now, but now it's really hard.  I think that a
simpler
approach would allow much simpler configuration support.  To extend
ZConfig now,
you have to create XML schema descriptions, and have deep knowledge of how
ZConfig works.

Why do you think it's better to have to create a monolithic schema for all
applications bits that want to use the configuration file, rather than
letting
individual applications define how to read their own data independently?


A monolithic schema is of course a problem. I am sure it could be solved by 
refactoring ZConfig.




There could still be frameworks to make handling configuration data
easier.



I agree but I really dislike the idea of flattening a hierarchical 
structure
into a INI-like format. Having /x/y/z as section names looks both funny and 
somewhat unprofessional. The format looks as if would have been invented by 
a first grader. There is no question that ZConfig has the problems you 
described. But I consider such a flat representation as poor and a step back
instead of a step forward (independent of the effort needed to simply and 
refactor ZConfig).


-aj

pgpzRPpYccAEU.pgp
Description: PGP signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Jim Fulton

Andreas Jung wrote:



--On 5. März 2006 13:56:38 -0500 Jim Fulton [EMAIL PROTECTED] wrote:



The right way would be to refactor ZConfig and decouple it in a
reasonable way from its dependencies.



I think this would be a major rewrite.



Possibly but I don't consider that to be a strong argument for introducing
a weaker mechanism.


I don't think that the complexity of ZConfig is justified by the power of
hierarchical sections.


They have to deal with it now, but now it's really hard.  I think that a
simpler
approach would allow much simpler configuration support.  To extend
ZConfig now,
you have to create XML schema descriptions, and have deep knowledge of 
how

ZConfig works.

Why do you think it's better to have to create a monolithic schema for 
all

applications bits that want to use the configuration file, rather than
letting
individual applications define how to read their own data independently?



A monolithic schema is of course a problem. I am sure it could be solved 
by refactoring ZConfig.


Yes.  That's why I tried to propose reimplementing ZConfig on top of the
ZCML engine a few weeks ago.  If that was the only goal, then I think
this would be the way to go.


There could still be frameworks to make handling configuration data
easier.



I agree but I really dislike the idea of flattening a hierarchical 
structure
into a INI-like format. Having /x/y/z as section names looks both funny 
and somewhat unprofessional.


Obvoiously, this is a matter of taste.  Many people are put off by even
the hint of XML in ZConfig.

 The format looks as if would have been

invented by a first grader.


It probably was. ;)

 There is no question that ZConfig has the

problems you described.  But I consider such a flat representation as
poor and a step back
instead of a step forward (independent of the effort needed to simply 
and refactor ZConfig).


I agree, however, I think that there are other benefits of a move to
ConfigParser that far outweigh this disadvantage.

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Andreas Jung



--On 5. März 2006 14:43:48 -0500 Jim Fulton [EMAIL PROTECTED] wrote:


  There is no question that ZConfig has the

problems you described.  But I consider such a flat representation as
poor and a step back
instead of a step forward (independent of the effort needed to simply
and refactor ZConfig).


I agree, however, I think that there are other benefits of a move to
ConfigParser that far outweigh this disadvantage.



Let's try to approach from the other side. We both agree at ZConfig sucks 
at some point. We need something else. The something else should be 
provide

a similar functionality as ZConfig:

 - some support for hierarchies

 - schema-based

 - validation

 - default values

 - local configurations, no global schema

In the first place such a framework would be independent of any input 
format. A ZConfig-style parser or a .INI parser could use such a framework 
to make configuration information available through a unique API. So the 
basic problem of this whole issue is how such a framework would look 
like, how a schema-definition would look like etc. Writing a parser that 
adopts a particular input format to the abstract configuration framework is 
at best an engineering challenge and the decision to use a ZConfig-style or 
an INI-Style configuration file format as default is like a personal 
preference for beer or wine (beer for the mass, wine for the power users 
:-))


-aj



pgpzzcQaftHUO.pgp
Description: PGP signature
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Benji York

Jim Fulton wrote:

  http://dev.zope.org/Zope3/UseConfigParserForHighLevelConfiguration

Is a proposal for using ConfigParser, rather than ZConfig for high-level
configuration.


+1

This is exactly the kind of innovation via reuse I like.
--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com



Re: [Zope3-dev] RFC: Use ConfigParser for High-Level Configuration

2006-03-05 Thread Benji York

Jim Fulton wrote:

It would be fairly easy to provide a simple schema system for
ConfigParser data based on zope.schema for verification and
conversion.


Good idea!
--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com