Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/app/server/ Reimplemented common access log using Twisted's log framework. It uses our
On Thursday 21 April 2005 08:13, Chris Withers wrote: > > Twisted has its own logging framework and the Twisted Web server logs to > > this framework and not the standard Python one. So we have to hook into > > it, which this code exactly does. > > So, just to check I understand correctly: Zope 3 plugs into Twisted's > logging framework such that messages logged to Twisted's logging > framework get piped through to Python's logging package infrastructure? Right, but only for one special case, because twisted does not have the concept of various loggers. It's framework is much more primitive than the zLog-derived logging package. > Chris - personally, I'd love to see twisted's logging stuff die... Right, but there is nothing we can do about that. The process to become a contributor is quiet lengthy (in my opinion), but I am working on it. In many ways the Twisted project does the same mistake as Zope 2, by brewing their own stuff that does not work well with Python. So in some sense Twisted in quiet unpythonic. The problem is that they have not been bitten by it hard yet, simply because of limited exposure. While some developers are very open to suggestions, especially itamar, foom and spiv with which I work on the project, others do not think they are doing anything wrong and keep reinventing the wheel. Reminds me of the old Zope 2 times. I am particularly worried about their testing policies, which are not as strict as ours and as such Twisted is more vulnerable to breakage, which has happened. Unfortunately they brewed together a very non-Python-standard testing framework, which makes it very hard to integrate new testing innovations, such as DocFileSuite. (Note that I understand their need for some customization of their testing framework, since they have to test deferreds.) Other than that, I should repeat again, my experience with Twisted has been very positive so far. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/app/server/ Reimplemented common access log using Twisted's log framework. It uses our
Stephan Richter wrote: On Thursday 21 April 2005 06:38, Chris Withers wrote: Why are you looking to use this rather than Python's logging package? The observer below clearly uses the Python logging package: Oops, sorry, missed that... Twisted has its own logging framework and the Twisted Web server logs to this framework and not the standard Python one. So we have to hook into it, which this code exactly does. So, just to check I understand correctly: Zope 3 plugs into Twisted's logging framework such that messages logged to Twisted's logging framework get piped through to Python's logging package infrastructure? cheers, Chris - personally, I'd love to see twisted's logging stuff die... -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/app/server/ Reimplemented common access log using Twisted's log framework. It uses our
On Thursday 21 April 2005 06:38, Chris Withers wrote:
> Why are you looking to use this rather than Python's logging package?
The observer below clearly uses the Python logging package:
def __init__(self, logger=None):
if logger is None:
logger = logging.getLogger('accesslog')
self.logger = logger
Twisted has its own logging framework and the Twisted Web server logs to this
framework and not the standard Python one. So we have to hook into it, which
this code exactly does.
Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/app/server/ Reimplemented common access log using Twisted's log framework. It uses our
Why are you looking to use this rather than Python's logging package?
Chris
Stephan Richter wrote:
Log message for revision 30066:
Reimplemented common access log using Twisted's log framework. It uses our
setup accesslog. Tests will follow, but BjornT wants to have a look.
Changed:
A Zope3/branches/srichter-twisted-integration/src/zope/app/server/log.py
U Zope3/branches/srichter-twisted-integration/src/zope/app/server/main.py
U Zope3/branches/srichter-twisted-integration/src/zope/app/server/server.py
-=-
Added: Zope3/branches/srichter-twisted-integration/src/zope/app/server/log.py
===
--- Zope3/branches/srichter-twisted-integration/src/zope/app/server/log.py 2005-04-20 19:55:14 UTC (rev 30065)
+++ Zope3/branches/srichter-twisted-integration/src/zope/app/server/log.py 2005-04-20 20:57:53 UTC (rev 30066)
@@ -0,0 +1,121 @@
+##
+#
+# Copyright (c) 2005 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##
+"""Logging Support
+
+Logging Support for the Twisted logging framework. A special observer will
+forward messages to the standard Python Logging Framework.
+
+$Id$
+"""
+__docformat__ = "reStructuredText"
+
+import logging
+import time
+import twisted.python.log
+from twisted import web2
+from zope.interface import implements
+from zope.app.http.httpdate import monthname
+
+
+class CommonAccessLoggingObserver(object):
+"""Outputs accesses in common HTTP log format."""
+
+def __init__(self, logger=None):
+if logger is None:
+logger = logging.getLogger('accesslog')
+self.logger = logger
+
+def computeTimezoneForLog(self, tz):
+if tz > 0:
+neg = 1
+else:
+neg = 0
+tz = -tz
+h, rem = divmod (tz, 3600)
+m, rem = divmod (rem, 60)
+if neg:
+return '-%02d%02d' % (h, m)
+else:
+return '+%02d%02d' % (h, m)
+
+tzForLog = None
+tzForLogAlt = None
+
+def logDateString(self, when):
+logtime = time.localtime(when)
+Y, M, D, h, m, s = logtime[:6]
+
+if not time.daylight:
+tz = self.tzForLog
+if tz is None:
+tz = self.computeTimezoneForLog(time.timezone)
+self.tzForLog = tz
+else:
+tz = self.tzForLogAlt
+if tz is None:
+tz = self.computeTimezoneForLog(time.altzone)
+self.tzForLogAlt = tz
+
+return '%d/%s/%02d:%02d:%02d:%02d %s' % (
+D, monthname[M], Y, h, m, s, tz)
+
+def emit(self, eventDict):
+"""See zope.app.logger.interfaces.IPublisherRequestLogger"""
+if eventDict.get('interface') is not web2.iweb.IRequest:
+return
+
+request = eventDict['request']
+
+firstLine = '%s %s HTTP/%s' %(
+request.method,
+request.uri,
+'.'.join([str(x) for x in request.clientproto]))
+
+self.logger.log(logging.INFO,
+'%s - %s [%s] "%s" %s %d "%s" "%s"' %(
+request.chanRequest.transport.client[0],
+request.response.headers.getRawHeaders(
+'x-zope-principal', ['anonymous'])[-1],
+self.logDateString(response.headers.getHeader('date', 0)),
+firstLine,
+request.response.code,
+request.bytesSent,
+request.headers.getHeader('referer', '-'),
+request.headers.getHeader('user-agent', '-')
+)
+)
+
+def start(self):
+"""Start observing log events."""
+twisted.python.log.addObserver(self.emit)
+
+def stop(self):
+"""Stop observing log events."""
+twisted.python.log.removeObserver(self.emit)
+
+
+
+class CommonFTPActivityLoggingObserver(CommonAccessLoggingObserver):
+"""Outputs hits in common HTTP log format."""
+
+def log(self, request):
+"""See zope.app.logger.interfaces.IPublisherRequestLogger"""
+now = time.time()
+message = ' - %s [%s] "%s %s"' % (task.channel.username,
+ self.log_date_string(now),
+ task.m_name[4:].upper(),
+ task.channel.cwd,
+ )
+
+self.output.logRequest(task
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/publisher/http.py Added back the setting of the principal string; but this time not via this
On Wednesday 20 April 2005 13:54, Chris Withers wrote:
> > + self.response.setHeader('x-zope-principal', message)
>
> Does this header make its way back the the browser?
>
> Should it?
Yep. But James Knight (foom) and I have a better idea. We will extend the WSGI
stuff by a 'logginginfo; dictionary that can be filled by the application.
This will mean that WSGI stuff will creep into our application code, but
that's how it is.
Regards,
Stephan
--
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/src/zope/publisher/http.py Added back the setting of the principal string; but this time not via this
Stephan Richter wrote:
+self.response.setHeader('x-zope-principal', message)
Does this header make its way back the the browser?
Should it?
cheers,
Chris
--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thursday 14 April 2005 14:17, Sidnei da Silva wrote: > On Thu, Apr 14, 2005 at 02:04:06PM -0400, Stephan Richter wrote: > | Any constructive input? Will a temporary file be better? Will it work > | easily cross-platform? How was this solved in Zope 2? > > So, in Zope 2, what we did recently was to introduce a LARGE_FILE_THRESHOLD > ZConfig variable. If the file is larger than that, we use a temporary > file. I don't see a reason for a temporaryfile (using the module > tempfile?) not be cross-platform. Certainly Tim would know if there's > any issue. But this only works if you control both, the server and the application. In WSGI-compliant code, the two are independent and all you are guaranteed to have is a simple input stream. But I think the real problem is within the publisher itself. While I have to keep track of one unperturbed input stream, CGI might create temporary files for part of this input stream. I have no idea on how to avoid this. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thu, Apr 14, 2005 at 02:04:06PM -0400, Stephan Richter wrote: | Any constructive input? Will a temporary file be better? Will it work easily | cross-platform? How was this solved in Zope 2? So, in Zope 2, what we did recently was to introduce a LARGE_FILE_THRESHOLD ZConfig variable. If the file is larger than that, we use a temporary file. I don't see a reason for a temporaryfile (using the module tempfile?) not be cross-platform. Certainly Tim would know if there's any issue. -- Sidnei da Silva <[EMAIL PROTECTED]> http://awkly.org - dreamcatching :: making your dreams come true http://www.enfoldsystems.com http://plone.org/about/team#dreamcatcher Little known fact about Middle Earth: The Hobbits had a very sophisticated computer network! It was a Tolkien Ring... ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thursday 14 April 2005 14:22, Tres Seaver wrote: > > Any constructive input? Will a temporary file be better? Will it work > > easily cross-platform? How was this solved in Zope 2? > > Sidnei is correct that a RAM cache is not appropriate here; a tempfile > (but only one!) is the only sane route. Note that Python's CGI module > *also* creates tempfiles, which can lead to a copy (it does in Zope2): I cannot guarantee that only one temporary file will be created. I have no control over the CGI code. All I know is that we need to retry having the original contents of the file. Also seek() is not an option, since we are now WSGI compliant and WSGI only asks for a stream. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stephan Richter wrote: > On Thursday 14 April 2005 13:43, Sidnei da Silva wrote: > >>On Thu, Apr 14, 2005 at 01:22:53PM -0400, Stephan Richter wrote: >>| On Thursday 14 April 2005 13:00, Florent Guillaume wrote: >>| > > +def __init__(self, stream): >>| > > +self.stream = stream >>| > > +self.cacheStream = cStringIO.StringIO() >>| > >>| > Won't a memory cache be a problem for multi-megabytes POSTs ? >>| >>| Maybe, but we have no choice here. The only other option is to make it a >>| temporary file. >> >>Maybe!?! Please, give it some thought. It has caused lots of pain in >>Zope 2, which just recently got fixed. That means a 'temp hack' can >>live as much as 8 (?) years. > > > Any constructive input? Will a temporary file be better? Will it work easily > cross-platform? How was this solved in Zope 2? Sidnei is correct that a RAM cache is not appropriate here; a tempfile (but only one!) is the only sane route. Note that Python's CGI module *also* creates tempfiles, which can lead to a copy (it does in Zope2): http://cvs.zope.org/Zope/lib/python/ZServer/Attic/HTTPServer.py?hideattic=0&sortby=date Tres. - -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation "Zope Dealers" http://www.zope.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCXrTbGqWXf00rNCgRAinfAJ9NY/OGWNY/2KTOyJ5aLwSA6yDnYwCfXfp+ eeKHGt+EkYwB4/w/bNOqm7c= =I8Qe -END PGP SIGNATURE- ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thursday 14 April 2005 13:43, Sidnei da Silva wrote: > On Thu, Apr 14, 2005 at 01:22:53PM -0400, Stephan Richter wrote: > | On Thursday 14 April 2005 13:00, Florent Guillaume wrote: > | > > + def __init__(self, stream): > | > > + self.stream = stream > | > > + self.cacheStream = cStringIO.StringIO() > | > > | > Won't a memory cache be a problem for multi-megabytes POSTs ? > | > | Maybe, but we have no choice here. The only other option is to make it a > | temporary file. > > Maybe!?! Please, give it some thought. It has caused lots of pain in > Zope 2, which just recently got fixed. That means a 'temp hack' can > live as much as 8 (?) years. Any constructive input? Will a temporary file be better? Will it work easily cross-platform? How was this solved in Zope 2? Also, since you are here, Sidnei, could you please, please fix WebDAV? Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thu, Apr 14, 2005 at 01:22:53PM -0400, Stephan Richter wrote: | On Thursday 14 April 2005 13:00, Florent Guillaume wrote: | > > + def __init__(self, stream): | > > + self.stream = stream | > > + self.cacheStream = cStringIO.StringIO() | > | > Won't a memory cache be a problem for multi-megabytes POSTs ? | | Maybe, but we have no choice here. The only other option is to make it a | temporary file. Maybe!?! Please, give it some thought. It has caused lots of pain in Zope 2, which just recently got fixed. That means a 'temp hack' can live as much as 8 (?) years. -- Sidnei da Silva <[EMAIL PROTECTED]> http://awkly.org - dreamcatching :: making your dreams come true http://www.enfoldsystems.com http://plone.org/about/team#dreamcatcher how are the jails in israel? well, the one I was in was pretty nice ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
On Thursday 14 April 2005 13:00, Florent Guillaume wrote: > > + def __init__(self, stream): > > + self.stream = stream > > + self.cacheStream = cStringIO.StringIO() > > Won't a memory cache be a problem for multi-megabytes POSTs ? Maybe, but we have no choice here. The only other option is to make it a temporary file. Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/branches/srichter-twisted-integration/
Stephan Richter <[EMAIL PROTECTED]> wrote: > Log message for revision 29982: > - Fixed issue 390. Deprecated ``IBaseRequest.body`` and > ``IBaseRequest.bodyFile``. The latter was simply renamed to > ``IBaseRequest.bodyStream``. No code assumes anymore that the input > streams are seekable. [...] > +class HTTPInputStream(object): > +"""Special stream that supports caching the read data. > + > +This is important, so that we can retry requests. > +""" > + > +def __init__(self, stream): > +self.stream = stream > +self.cacheStream = cStringIO.StringIO() Won't a memory cache be a problem for multi-megabytes POSTs ? Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of R&D +33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED] ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
