Re: [Zope] zeo cache-size tuning - number of obj. vs MB
On Wed, 2010-01-06 at 03:57 -0500, Tres Seaver wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Gaute Amundsen wrote:
> > Hi
> >
> > What's the difference between the to cache-size settings here?
> > I've not been able to find a good description to guide me in finding the
> > right values.
> >
> >
> > mount-point /
> > cache-size 1 1.
> >
> > server foo:9080
> > storage main
> > name zeostorage
> > var $INSTANCE/var
> > cache-size 20MB 2.
> >
> >
> >
> > As far as I've been able to determine, the second one 2. refers to a
> > separate cache at the zeo connection level.
>
> The ZEO client cache is a disk-based cache, separate from the RAM-based
> cache used by any ZODB connection. It doesn't use RAM, per se, except
> for an in-memory index of OID -> (layer, offset).
What would happen if set too small or too large?
> > Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram,
> > varnish, apache, and haproxy. Tethering on the edge of too much
> > swapping. Needing to cut down..
>
> Assuming you are running with the default number of "worker threads"
> (4): 1 objects in the cache * 4 active connections per appserver *
> 9 appservers means you are caching 360,000 objects in RAM. If they are
> putting you into swap, then you are averaging something like 10Kb per
> object, which is pretty big.
Sorry, forgot to mention: 1 thread per appserver.
Otherwise there's not much use in haproxy's load balancing we figured.
Meory usage as reported by munin is avg. 160M, max ~220M
and backend loads/stores are about 1000/10 a minute avg.
and iowait at 45% avg today.
committed memory 3.7G avg.
This little oneliner for summing VSZ:
ps auxf | grep zope | awk '{total+=$5} END{print total}'
reports:
1918552 for zope
1489768 for varnish
2270732 for www-data
And for RSS
1375908 zope
177200 varnsih
374624 www-data
in addition to the 9, that includes one appserver with 6 therads/6
obs and one with 1 therad/2 for developers and robots respectively.
> > Would it be better to spend all the memory on 1?
> > Is there a good way to find a balance?
>
> I think I might cut down on the number of appservers on that host, if it
> truly is hitting swap. Or trim the number of threads per appserver down
> from 4 to 2 or 3.
Problem is we run out of them way to often now, and get 503's from
haproxy. I just added another 3 in fact..
Perhaps I'll try with 6 tomorrow..
Thanks :)
Gaute
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zeo cache-size tuning - number of obj. vs MB
Hi What's the difference between the to cache-size settings here? I've not been able to find a good description to guide me in finding the right values. mount-point / cache-size 1 1. server foo:9080 storage main name zeostorage var $INSTANCE/var cache-size 20MB 2. As far as I've been able to determine, the second one 2. refers to a separate cache at the zeo connection level. Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram, varnish, apache, and haproxy. Tethering on the edge of too much swapping. Needing to cut down.. Would it be better to spend all the memory on 1? Is there a good way to find a balance? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZopeProfiler uninstall
On Tue, 2009-12-01 at 20:57 +0100, Sascha Welter wrote:
> (Tue, Dec 01, 2009 at 02:33:04PM +0100) Gaute Amundsen wrote/schrieb/egrapse:
> > We have ZopeProfiler 1.72 installed on a site (Zope 2.7.5-1.fc3) that
> > don't need it anymore.
> > I've removed the folder and deleted the product, but I cant' find a way
> > to remove it form the controlpanel.
> > I notice it's been asked once before, but I cant find any replies.
> >
> > Any hints?
>
> http://www.peterbe.com/uninstall-LeakFinder
> https://mail.zope.org/pipermail/zope-dev/2001-August/012742.html
>
> Same procedure applies to ZopeProfiler too, IIRC.
That one looks dangerous!
> cp = self.Control_Panel
> try: del cp.LeakFinder
> except:
> LOG('Fixup',ERROR,'Exception Occurred:',error=sys.exc_info())
> try:
> del cp._objects # !!!
> except:
> LOG('Fixup',ERROR,'Exception Occurred:',error=sys.exc_info())
I have not consulted the source here, but would not that second try wipe out
everything in cp?
I think I'll go for the open heart surgery :)
Gaute
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZopeProfiler uninstall
On Tue, 2009-12-01 at 11:28 -0500, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Gaute Amundsen wrote: > > We have ZopeProfiler 1.72 installed on a site (Zope 2.7.5-1.fc3) that > > don't need it anymore. > > I've removed the folder and deleted the product, but I cant' find a way > > to remove it form the controlpanel. > > I notice it's been asked once before, but I cant find any replies. > > > > Any hints? > > $ bin/zopectl debug > ... > >>> import transaction > >>> app.Control_Panel.manage_delObjects(['profiler']) # whatever > >>> transaction.commit() > > or something like that. ;) Nothing like doing open heart surgery on > your ZODB to get your pulse pounding. > > > Tres. Hehe, precisely :) Gaute ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZopeProfiler uninstall
On Tue, 2009-12-01 at 18:15 +0100, Dieter Maurer wrote:
> Gaute Amundsen wrote at 2009-12-1 14:33 +0100:
> >We have ZopeProfiler 1.72 installed on a site (Zope 2.7.5-1.fc3) that
> >don't need it anymore.
> >I've removed the folder and deleted the product, but I cant' find a way
> >to remove it form the controlpanel.
> >I notice it's been asked once before, but I cant find any replies.
> >
> >Any hints?
>
> You can delete it in a "bin/zopectl debug" interactive session:
>
> app.Control_Panel._delObject('ZopeProfiler')
> from transaction import commit; commit()
>
> You are only prevented to do it from the ZMI because "Control_Panel"
> does not expose the typical "ObjectManager" API there.
Most excellent :)
I suspected it had to be something like that, but I had no idea where to
begin looking.
Tanks
Gaute
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZopeProfiler uninstall
On Tue, 2009-12-01 at 14:39 +0100, robert rottermann wrote: > Gaute Amundsen schrieb: > > We have ZopeProfiler 1.72 installed on a site (Zope 2.7.5-1.fc3) that > > don't need it anymore. > > I've removed the folder and deleted the product, but I cant' find a way > > to remove it form the controlpanel. > > I notice it's been asked once before, but I cant find any replies. > > > > Any hints? > > > > Regards > > > > Gaute Amundsen > > > > > > > go to the control panel (portal_controlpannel) in the zmi and delete it > robert No such thing as portal_controlpannel here. And no check-boxes or "delete" button in /Controll_Panel either. Please try again ;-) Gaute ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZopeProfiler uninstall
We have ZopeProfiler 1.72 installed on a site (Zope 2.7.5-1.fc3) that don't need it anymore. I've removed the folder and deleted the product, but I cant' find a way to remove it form the controlpanel. I notice it's been asked once before, but I cant find any replies. Any hints? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Finding unused products
On Mon, 2009-11-30 at 10:59 -0500, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Gaute Amundsen wrote: > > On Sun, 2009-11-29 at 16:05 -0500, Tres Seaver wrote: > >> -BEGIN PGP SIGNED MESSAGE- > >> Hash: SHA1 > >> > >> Gaute Amundsen wrote: > >>> On Sun, 2009-11-29 at 12:32 +0100, Jens Vagelpohl wrote: > >>>> On Nov 29, 2009, at 11:59 , Gaute Amundsen wrote: > >>>> > >>>>> Quick question before I reinvent. > >>>>> > >>>>> Is there some established way of finding products which have no > >>>>> instances in zodb, and can be safely removed? > >>>> No there isn't. You may have to invest some time writing a script that > >>>> visits all objects to find out what products they belong to. It may > >>>> not be worth the effort. > >>>> > >>>> jens > >>> Unfortunately it has "not been worth the effort" for a number of years, > >>> but now it has become unavoidable. > >>> Walking the tree it is then. > >> You could probably work directly with the pickles way faster than > >> activating every object in a large site: look at the 'fsdump' script > >> for clues. > >> Tres. > > > > Thanks for the tip, but testing a few tings now, it seems I would > > probably spend way more time on the learning-curve than I'd save in > > parsing. > > > > I think, I'll rather stick my expensive parse results i a table, and > > work from that. > > > > Unless you think otherwise and can point me towards how to get both the > > path, and the metatype out of that pickle. :-) > > The pickle holds the dotted name of the instance class, which should be > a clue. ;) I spotted that playing around with fsdump yes. On second thought I guess my hidden assumption was that context.all_meta_types() would give me the best list of installed products to compare against, and thus I would need the actual meta_type string. On examining the results of the first run of my db-based script, I'm not so sure. It's after all just some strings.. Perhaps I ought to rephrase that as a question :) Can I rely on context.all_meta_types() being complete, or could a product have installed objects that would not be in that list? Is there some other, more reliable way to link an object to it's product? Gaute ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Finding unused products
On Sun, 2009-11-29 at 16:05 -0500, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Gaute Amundsen wrote: > > On Sun, 2009-11-29 at 12:32 +0100, Jens Vagelpohl wrote: > >> On Nov 29, 2009, at 11:59 , Gaute Amundsen wrote: > >> > >>> Quick question before I reinvent. > >>> > >>> Is there some established way of finding products which have no > >>> instances in zodb, and can be safely removed? > >> No there isn't. You may have to invest some time writing a script that > >> visits all objects to find out what products they belong to. It may > >> not be worth the effort. > >> > >> jens > > > > Unfortunately it has "not been worth the effort" for a number of years, > > but now it has become unavoidable. > > Walking the tree it is then. > > You could probably work directly with the pickles way faster than > activating every object in a large site: look at the 'fsdump' script > for clues. > Tres. Thanks for the tip, but testing a few tings now, it seems I would probably spend way more time on the learning-curve than I'd save in parsing. I think, I'll rather stick my expensive parse results i a table, and work from that. Unless you think otherwise and can point me towards how to get both the path, and the metatype out of that pickle. :-) Gaute ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Finding unused products
On Sun, 2009-11-29 at 12:32 +0100, Jens Vagelpohl wrote: > On Nov 29, 2009, at 11:59 , Gaute Amundsen wrote: > > > > > Quick question before I reinvent. > > > > Is there some established way of finding products which have no > > instances in zodb, and can be safely removed? > > No there isn't. You may have to invest some time writing a script that > visits all objects to find out what products they belong to. It may > not be worth the effort. > > jens Unfortunately it has "not been worth the effort" for a number of years, but now it has become unavoidable. Walking the tree it is then. Gaute ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Finding unused products
Quick question before I reinvent. Is there some established way of finding products which have no instances in zodb, and can be safely removed? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Best place to rewrite webdav GET requests?
On Friday 12 September 2008 07:19:29 Sascha Welter wrote: > (Thu, Sep 11, 2008 at 08:49:38AM +0200) Gaute Amundsen wrote/schrieb/egrapse: > > Or would it perhaps be better to route webdav through apache, and try to > > handle it with rewriterules? > > This is how we do it. (Not for NightmareWeaver though.) > Interesting. Then we know it is possible at least :) You wouldn't happen to have that part of httpd.conf handy for a quick cut & paste? :) Regards Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Reinventing a proxy wheel?
Hi. I think I may just have reinvented a kind of cgi-proxy for authentication purposes. Seems to work nicely so far, but oh, so ugly. Thus I would be gratefull for any pointers towards some more enlightened solution. Problem to solve: External php based facility needs to be made available to logged in zope users only, but have no built in mechanism to to handle "everyone must log in to see anything at all" situations. Some users can be synced by way of sql, but many can not. This is a given. My solution so far: An external method that wraps wget. Rearanges the url to connect via 127.0.0.1, and use original host as host header. Use Apache to close access to everyone but 127.0.0.1. Rest of the script basically shuffles headers back and forth from zope to wget, and back. and the script can of course be subject to normal zope access checks. If no major flaws with this approach appears, I will of course replace wget with urllib, and perhaps even productify the thing, but before I do that I have this nagging feeling of reinventing the wheel that I need to address... Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: process_memory reports negative usage
On Friday 01 June 2007 15:48, Izak Burger wrote: > On 6/1/07, Gaute Amundsen wrote: > > I'm using the process_memory plugin for monitoring zope, just as in the > > readme. > > Your the first person to ever contact me about it, so I am very happy > to hear that it was useful to someone else as well :-) It has been extremely useful actually, in discovering that the stability problems we had was caused by a to small a zodb cache, and in finding a new appropriate value for it. That, and the zope cache parameters and Database Activity plugins that I just published :) http://muninexchange.projects.linpro.no/?search&cid=14&pid=108 I am taking the liberty of CCing the zope, and munin lists on this, so perhaps others wil become aware of that usefullness as well :) And thanks for the uppdated version :) Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Scaling problems, or something else?
On Wednesday 16 May 2007 07:52, Gerhard Schmidt wrote: > On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > > So there is no other possible limit in a zope instance than IO or CPU? > > If cpu was the limiting factor I would see the 2 python processes running > > 90% and dozens of httpd's taking up the rest? > > I'm running a zope site with up 70k requests per hour (40k on avarage). > I've noticed that the main reason for performance problem is not enough > memory. By a rough estimate we run about 36K /hour into apache, and perhaps half of that from zope. The munin process_memory plugin I installed yesterday indicates the we only consume about 600M of that, so I guess it looks more and more like this might be hour bottleneck. > We are running 13 Frontend Zope servers and a ZEO Backend. > Woha! Either I am wrong about our load, or you have a completely differnet type of site.. > All our frontendservers have 8gig ram. Zope gets major performance Problems > when it reaches the limit of physical memory. Check your system if the. > So having 2 zope Processes on the same system increases the Problem. > We do actually, have two. From a time we just "hadd to do something"! Consolidation time... > Zope 2.7 doesn't scale very well with the ZEO. The more Frontend server > you get the more Read Conflicts you have. Migration to Zope2.8 reduced this > problem. > > I have build a squid proxy in reverse mode take the Request and spead them > per round robin to the ZopeFrontends. This takes quite a lot load from the > systems as the squid caches most of the static content (images, PDF files > etc). > Hm.. yes, that's the way we have been inclined.. just skip the whole ZEO thing, and go straight for Varnish integration. Lotts of old sinns that have to be paid off for that to work for everything however.. At some point Varnish url rewriting wil be good enough, and then we can cut apache out of it too.. thanks :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Scaling problems, or something else?
On Wednesday 16 May 2007 00:13, Paul Winkler wrote: > On Tue, May 15, 2007 at 12:00:52PM +0200, Gaute Amundsen wrote: > > I set cache-size to 1 last night, up from the default. > > I felt I had to try that before I had a grap in place, so I don't have > > good numbers to estimate tha change, but with about 20 hits a sec on > > apache, there was close to 50 loads the last hour, and just 3000 > > writes. > > Does that look reasonable? > > Not really. Too many loads; you're getting a lot of cache misses and > blowing out the cache a lot. Keep on doubling the cache size until > your whole working set is in RAM all the time. > Will do! > > The built in help seems to indicate I should increase the cache until > > reads approaches zero..? > > Yeah, for some definition of "approaches". I think a more realistic > minimum would be number of threads * number of ZODB writes (since each > write potentially invalidates one cached object per thread). > That would give me 2500*6 = 15000. Looks like a reasonable goal. > It's hard to quantify the point at which you're caching enough, but > you're not even close. > I can see from the config that we tried cache-size 2 once, but probably set it back since something else was probably the problem that time :-( > > > Also, I don't think you've mentioned what sort of app this is. > > > Is it mostly reads or are there lots of writes? > > > "Mostly reads" is a lot easier to optimize :) > > > > Big CMS system with about 70 virtual domains. > I keep mixing up the numbers here I notice, but make that ~180 hosts in apache and ~85 in zope. > Based on your activity graph, you do indeed have a lot more reads than > writes. > > What's the CMS based on? Plone? Silva? Nothing of the sort. I'ts a "innhouse" thing... Could be lots of gremlins burried there I know :-/ > Chris W. had a good point about the catalog. > > Do you have blobby data in the ZODB? (large images or files)? Those > tend to play havoc with zodb cache activity, since one OFS.Image is > stored as an arbitrarily long chain of small persistent objects. So > whereas a Plone Document or a Page Template needs only one entry in > the cache, an Image might need hundreds. > Not _that_ much. There is some from olden times, but we moved all our image stuff over into something based on phpgallery 2 years ago. thanks :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Scaling problems, or something else?
On Tuesday 15 May 2007 15:20, Chris Withers wrote: > Gaute Amundsen wrote: > > On Thursday 10 May 2007 17:57, Paul Winkler wrote: > >> On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > >>> On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > >>>> On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > >>> > >>> So there is no other possible limit in a zope instance than IO or CPU? > >> > >> Well, there's RAM of course. If you run out and start swapping a lot, > >> that's a big problem :) > > > > About 12G in that box, so that's unlikely. > > How big's your Data.fs? If it's bigger than 12GB, then careless code can > easilly chew through all your ram... > As it happens its' very close to 12 G! > >> Very helpful if the trouble is ZODB I/O. Have a look at the activity > >> graph and see if you're getting lots of loads... might mean your ZODB > >> cache is too small and is getting thrashed. > > > > I set cache-size to 1 last night, up from the default. > > If you're using a ZCatalog, you might consider splitting it into it's > own zodb in order to improve the performance of this cache... > Using it in small bits all over the place. That would make it rather hard to mount separately, would it not? > > Big CMS system with about 70 virtual domains. > Make that ~180 hosts in apache and ~85 in zope. > How often does the content change? > Not often at all. I've got a munin plugin for it now, and that indicates the average is less than 2500 stores over 5 min. Rather hard to see actually, against the reads which fluctuates between 30 K and 100 K. (will plublish that plugin, once I add some comments and such.) > cheers, > > Chris ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Scaling problems, or something else?
On Thursday 10 May 2007 17:57, Paul Winkler wrote: > On Thu, May 10, 2007 at 02:10:40AM +0200, Gaute Amundsen wrote: > > On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > > > On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > > > > So there is no other possible limit in a zope instance than IO or CPU? > > Well, there's RAM of course. If you run out and start swapping a lot, > that's a big problem :) > About 12G in that box, so that's unlikely. > btw, if you haven't come across this yet, I recommend reading > Chris McDonough's "Scaling Zope" presentation. It's a couple years old > but still a very good place to start: > http://www.plope.org/misc/szweb/img0.html > Looks good. Thanks :-) > > Something out of Control_Panel/Database/main/manage_activity > > perhaps? > > Very helpful if the trouble is ZODB I/O. Have a look at the activity > graph and see if you're getting lots of loads... might mean your ZODB > cache is too small and is getting thrashed. I set cache-size to 1 last night, up from the default. I felt I had to try that before I had a grap in place, so I don't have good numbers to estimate tha change, but with about 20 hits a sec on apache, there was close to 50 loads the last hour, and just 3000 writes. Does that look reasonable? The built in help seems to indicate I should increase the cache until reads approaches zero..? > Also, I don't think you've mentioned what sort of app this is. > Is it mostly reads or are there lots of writes? > "Mostly reads" is a lot easier to optimize :) > Big CMS system with about 70 virtual domains. > > Is there a way to get that data out without going through port 8080? > > Don't think so. But I've never looked. > Hm. I guess if I at least use a url on the management port, it ought to stay available when everything else bogs down. > > Maybe I wil start logging the responstime directly like that! > > Hm.. good idea :) > > If you need that, the trace log can also tell you exactly what the > response time per request is. > Good for debugging, but for permanent use, I wil use this: http://muninexchange.projects.linpro.no/?search&cid=10&pid=61&phid=81 > > > - DeadlockDebugger may be informative. > > > http://www.zope.org/Members/nuxeo/Products/DeadlockDebugger > > > > Sounds a little drastic on a production server, but it may stil come > > to that.. Ought to test it out on another server I guess. > > Definitely test-drive it on a scratch server, but production is where > you really need it :-) > > re. trace log: > > That looks interesting, except that it can take 15 minutes or more > > to restart zope when load is at the worst. I could try it outside of > > peak ours I guess. > > Yeah, restart during off hours and leave it enabled until you have one > episode of peak load slowdown. Gaute. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Scaling problems, or something else?
On Wednesday 09 May 2007 16:41, Paul Winkler wrote: > On Wed, May 09, 2007 at 12:07:54PM +0200, Gaute Amundsen wrote: > > Just a quick call for ideas on this problem we have... > > > > Setup: > > Zope 2.7.5 (~180 sites) -> apache (-> varnish for 1 high profile site) > > > > Most noticeable symtoms: > > Takes 30 sec. or more to serve a page, or times out. > > Sporadic problem, but allways during general high load. > > Lasts less than 1 hour. > > Restarting zope does not help. > > Lots of apache processes in '..reading..' state > > Apache accesses and volume is down. > > Server load is unchanged, and < 2.0 > > Apache processes is way up (~250 aganinst <40) > > Netstat "established" connections is WAY up (~650 aganist < 50) > > The increase in netstat connections and apache processes indicates > lots of simultaneous traffic, but it's interesting that Apache > accesses is down. Since hits are logged only on completion, it may be > that many of the requests are hung. > That was my reasoning too. > > Is this zope hitting some sort of limit and just letting Apache hang? > > Would setting up ZEO on the same box make a difference, > > ZEO doesn't buy you any performance unless you have multiple Zope > clients reading from it, and a load balancer in front. This will help > IF your application is CPU-bound, which yours is not (I assume by > server load you mean CPU). So there is no other possible limit in a zope instance than IO or CPU? If cpu was the limiting factor I would see the 2 python processes running 90% and dozens of httpd's taking up the rest? Can You think of any good parameters I can get at with a small script that would be good to graph with all the rest to shed som light on this? (we are using munin) Something out of Control_Panel/Database/main/manage_activity perhaps? Is there a way to get that data out without going through port 8080? How about something out of /proc/`cat /home/zope/sites/site1/var/Z2.pid`/XXX? Need to read up on procfs I guess. > ZEO can actually *hurt* if you're IO-bound, because it adds network > overhead to ZODB reads and writes. It's very bad if you have large > Image or File objects (which you probably shouldn't have in the ZODB > anyway). > Good to hear. I ws not particularly relishing the thought of the nescesary load balancing on that single box either :-/ > > or would it be better > > to extend varnish coverage? > > Probably a good idea anyway... but you want to find out what the > problem really is. > > > What would you do to pinpoint the problem? > > I'd first try hitting Zope directly during the problem to see if the > slowdown is there. If so, I'd then try either: > Should be possible with lynx on localhost. Have done that before for other purpose, Should have thought of that. Maybe I wil start logging the responstime directly like that! Hm.. good idea :) > - DeadlockDebugger may be informative. > http://www.zope.org/Members/nuxeo/Products/DeadlockDebugger > Sounds a little drastic on a production server, but it may stil come to that.. Ought to test it out on another server I guess. > - Enable Zope's trace log and use requestprofiler.py to see if there > is a pattern to the requests that trigger the problem. Eg. maybe > all your zope worker threads are waiting on some slow IO task. See > the logger section of zope.conf. That looks interesting, except that it can take 15 minutes or more to restart zope when load is at the worst. I could try it outside of peak ours I guess. Thanks for the innput. Really helped get me unstuck, as you can see :) Regards Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Scaling problems, or something else?
Just a quick call for ideas on this problem we have... Setup: Zope 2.7.5 (~180 sites) -> apache (-> varnish for 1 high profile site) Most noticeable symtoms: Takes 30 sec. or more to serve a page, or times out. Sporadic problem, but allways during general high load. Lasts less than 1 hour. Restarting zope does not help. Lots of apache processes in '..reading..' state Apache accesses and volume is down. Server load is unchanged, and < 2.0 Apache processes is way up (~250 aganinst <40) Netstat "established" connections is WAY up (~650 aganist < 50) Is this zope hitting some sort of limit and just letting Apache hang? Would setting up ZEO on the same box make a difference, or would it be better to extend varnish coverage? What would you do to pinpoint the problem? Any ideas are welcome! Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Making pythonscript changes to formulator fields persistent
On Monday 02 April 2007 18:58, Maciej Wisniowski wrote: > > I was thinking of that... > > How would you do that? Do you know formulator enough to say? > > I don't know your use case good enough. Is this just changing > attributes of formulator fields? Just asking :) > > Put the whole thing in the EM, or something simpler? > > An EM I could just hand off the field object to, to get "persisted"? > > I don't understand what you mean. Field objects already are persistent. > EM gives you no security restrictions in comparision to Script Python. > In hindsight, what I ment, was instead of doing the whole traversal thing in the EM, I could just hand the EM the field, and a dict with the changes to be made, and it would only do the little magic: vals = field.values vals.update(mydict) field.values = vals > In general to simply change field.values['required'] to false I'd write > function to traverse through ZODB to find all FormulatorForm objects > and it's fields. Had that allready. Ended up more or less just pasting that into the EM, then calling the EM from a script and just passing it the context. It just worked, plain and simple. :) > > (I find it a bit cumbersome to develop in EMs, reloading and all, so I > > tend to avoid it.) > > With ExternalMethod you only have to hit save button again to refresh > it. I know, I know but that extra step actually adds 1/3 to the usual cycle of editt->reload. And then it ads another layer of indirection when you come back later and have to figure out how it works. > Simple external method code that you may use to traverse through ZODB: > > def checkFolder(self, fld): > for obj_name, obj in fld.objectItems( 'FormulatorForm' ): > # get form fields here etc > > for fld_name,fld_obj in fld.objectItems( 'Folder' ): > checkFolder( fld_obj ) > for fld_name,fld_obj in fld.objectItems( 'Folder (Ordered)' ): > checkFolder( fld_obj ) > > I'm not sure about 'FormulatorForm', this may be 'Formulator Form' or > something like that. It's fld.objectItems(['Formulator Form']) Sorry to put you to the trouble, I should have been more clear I had that part. Thank you for your help :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Making pythonscript changes to formulator fields persistent
On Monday 02 April 2007 17:51, Maciej Wisniowski wrote: > > What to do? > > > > I guess some developer has worked hard to prevent me from doing just this > > at some point, but why? Do I need to write a product to do it? > > Is anyone at all using this crazy development model where "semi trusted > > users" do stuff in the ZMI??? > > But I digress.. > > > > If anyone knows how to solve this I would be less frustrated for a while > > ;-) > > Use ExternalMethod - easy just like python scripts and it's code is > trusted. I was thinking of that... How would you do that? Do you know formulator enough to say? Put the whole thing in the EM, or something simpler? An EM I could just hand off the field object to, to get "persisted"? (I find it a bit cumbersome to develop in EMs, reloading and all, so I tend to avoid it.) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Making pythonscript changes to formulator fields persistent
Hi. Trying to change various properties of a few hundred formulator fields programmatically. For example assigning to field.values['required'] = False. This seemed to work, but of course was not persisted, and disappeared gradualy. Quite magical to see :) Tried to take a copy of values and "overwrite" values with the changed ones, to trigger the presistence machinery, but that is not allowed. The same goes for setting field._p_changed = True, and accessing field._edit(). I can use field.manage_edit() but then I have to masage all the data from field.values into "request format" to have it accepted, otherwise I get "required" errors, or values gets deleted. What to do? I guess some developer has worked hard to prevent me from doing just this at some point, but why? Do I need to write a product to do it? Is anyone at all using this crazy development model where "semi trusted users" do stuff in the ZMI??? But I digress.. If anyone knows how to solve this I would be less frustrated for a while ;-) Regrads Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How do people work with html-designers?
On Thursday 15 March 2007 10:23, Jaroslav Lukesh wrote: > do google: zpt OR tal lukesh > > I was wrote about this few months ago in this mailinglist. > > Regards JL. Did you try googling that yourself before responding? Not much there.. I searched the mailinglist for various terms as well before posting this, but could find nothing very relevant. Tried "lukesh" on the list now.. You can't be thinking of the "Urgent Help Required" thread. I have not read every last post, but that does no look like a very constructive... A slightly more specific pointer would be appreciated. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] How do people work with html-designers?
I have been meaning to ask this for a while.. How do people set up the zope development process and servers to work well with web-designers who use wysiwyg editors like dreamweaver? I mean, we all probably use webdav, and "external editor", but after that? After all one of the advertised advantages of ZPT, is that you can send the template back to the designer without him destroying the TAL code. (not right away at least) In practice I find that once the original html has been split into "main tempate" and "content template" parts, or the like, it gets rather impractical to involve the designer again, and we end up having the coders do way more css work than they should. I know you can leave the "dummy content" in place, but often the stylesheets, and scripts live in a different folder, and gets aquired by zope for publishing. If I could get this whole process to be more of a two way thing, I could do do stuff, like start a site with a prototype, and have the designer beautyfy that, and work iteratively, instead of allways starting with the graphic design... Perhaps it would be possible to construct a special webdav "editor view" that could "symlink" certain files into any directory, so that the wysiwyg editor finds them, or even "assemble" the templates to a page, and send the edits to the right template, or something like that. Perhaps such a product allready exists? :) Or maybe the fix is in the editor, and you can tell dreamweaver to look for stuff it can't find in the dav-filesystem, at a certain URL instead.. Anyway, I would be glad to hear how other zope-shops, deal with these kinds of issues. Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Two mysql / timestamp related fixes
On Friday 09 March 2007 16:22, Tino Wildenhain wrote:
> Gaute Amundsen schrieb:
> > On Friday 09 March 2007 09:40, Tino Wildenhain wrote:
> >> Gaute Amundsen schrieb:
> >>> On Thursday 08 March 2007 17:27, Tino Wildenhain wrote:
> >>>> Gaute Amundsen schrieb:
> >>>>> Hi.
> >>>
> >>>
> >>>
> >>>>> 1 )
> >>>>> First I got a ValueError 'query' when using a very simple zsql
> >>>>> method. (consisting of only "")
> >>>>
> >>>> Thats a very bad idea. Better make zsql methods
> >>>> for every query
> >>>
> >>> I know the orthodoxy :-/
> >>> I'd rather have a proper DB library, (or model layer, if you will)
> >>> and build my SQL in python.
> >>
> >> So you want to use SQLAlchemy? Feel free to write a product
> >> as wrapper :-)
> >
> > Hm.. have never quite gotten around to writing my first product.
> > Looks like a very nice candidate, but no promises :)
> >
> >>> To build anything complex or generalized in dtml is a PAIN.
> >>
> >> Is it? Can you come up with examples?
> >
> > tabell:string
> > felter:list
> > INSERT INTO SET
> >
> >
> > ,
> > =
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > SELECT LAST_INSERT_ID()
> >
> > and that is not _really_ complex is it?
>
> Well, it looks a bit silly - I mean you make
> a hand-written ORM? This would be the time to
> either write a real ORM as product or try
> some of the available: ZPatterns, Archetypes, ...
>
Hm.. not intentionally :-/
Have heard about them of course, but not yet had a project that could defend
trying them out properly.
I whish there where some way in the zope commulity to get a feeling for what
people are actually using, and what has some momentum... but that is another
thread, I guess.
btw. I found this rather good.
http://blogs.nuxeo.com/sections/blogs/florent_guillaume/2005_08_11_object_relational
But no clear conclusion there either.
> Usual database interacting zope solutions
> have only a handful insert places where you
> need bulk inserts. And even then you can just
> call one ZSQL method (per table) in a loop
> to insert your values from a list.
>
> More, sql_quote does not really work
> for tablenames or column names - it
> may work by accident for MySQL though,
> but not for standards compliant databases.
>
> > There's the whole sqlgroup thing.
>
> Well to be honest I never used it ;)
>
> > dtml is after all depreciated, so I can see no reason to learn it as
> > torougly as ZSQLMethods obviously requires when I can use python instead.
>
> You always can but what do you gain? How does your python script
> code look like for the above problem?
>
Not found a way to work around the sql_delimiter problem yet, so it's not
really done.. but I think you will agree that its is more readable anyway.
def insert(tabell,felter):
sql = "INSERT INTO %s SET " % ( sql_quote(tabell) )
cols = []
for key, val in felter.items():
if same_type(val, ' '):
val = "'%s'" % val
cols.append("%s = %s" % (key, val))
sql += ", ".join(cols)
sql += '; SELECT LAST_INSERT_ID();'
return wrapper( sql=sql )
> >>> And a sea of too specific templates is even worse.
> >>
> >> Well from some level of complexity you could also use
> >> database functions. But if its so complex, why using
> >> MySQL?
> >
> > Don't see quite what you mean here, but it would be just the same
> > problems with PGsql.
>
> Not direct related but if you have complex queries,
> e.g. join over many tables, partial indexes, subqueries, ...
> it may be a bit hard to get reliable performance with MySQL.
> I think it was related to different views at the word "complex
> queries"
>
> >>>> instead of
> >>>> rebuilding your own (obviously not working)
> >>>> templating for SQL.
> >>>
> >>> I's not a template, it's a wrapper to get around the whole mess.
> >>> A hack in fact ;)
> >>>
> >>> If a template this simple "" is "obviously not working"
> >>> then it is ZSQLMethods that have a problem, not me.
> >>
> >> No, the content of your variable sql is wrong :-)
> >> would indeed work
Re: [Zope] How can run multiple query in an Z sql method
On Friday 09 March 2007 10:12, Remil Mathew wrote: > Hi, > How can run multiple query in an Z sql method.. I am using Mysql.. > Also is there any method to return a value from insert query? > > Thanks > Remil could you possibly be thinking of? insert foo into bar SELECT LAST_INSERT_ID() gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Two mysql / timestamp related fixes
On Friday 09 March 2007 09:40, Tino Wildenhain wrote:
> Gaute Amundsen schrieb:
> > On Thursday 08 March 2007 17:27, Tino Wildenhain wrote:
> >> Gaute Amundsen schrieb:
> >>> Hi.
> >
> >
> >
> >>> 1 )
> >>> First I got a ValueError 'query' when using a very simple zsql method.
> >>> (consisting of only "")
> >>
> >> Thats a very bad idea. Better make zsql methods
> >> for every query
> >
> > I know the orthodoxy :-/
> > I'd rather have a proper DB library, (or model layer, if you will)
> > and build my SQL in python.
>
> So you want to use SQLAlchemy? Feel free to write a product
> as wrapper :-)
Hm.. have never quite gotten around to writing my first product.
Looks like a very nice candidate, but no promises :)
> > To build anything complex or generalized in dtml is a PAIN.
>
> Is it? Can you come up with examples?
>
tabell:string
felter:list
INSERT INTO SET
,
=
SELECT LAST_INSERT_ID()
and that is not _really_ complex is it?
There's the whole sqlgroup thing.
dtml is after all depreciated, so I can see no reason to learn it as torougly
as ZSQLMethods obviously requires when I can use python instead.
> > And a sea of too specific templates is even worse.
>
> Well from some level of complexity you could also use
> database functions. But if its so complex, why using
> MySQL?
>
Don't see quite what you mean here, but it would be just the same problems
with PGsql.
> >> instead of
> >> rebuilding your own (obviously not working)
> >> templating for SQL.
> >
> > I's not a template, it's a wrapper to get around the whole mess.
> > A hack in fact ;)
> >
> > If a template this simple "" is "obviously not working"
> > then it is ZSQLMethods that have a problem, not me.
>
> No, the content of your variable sql is wrong :-)
> would indeed work if it contains valid sql
> code.
It did not.
That was the whole point of this mail.
The method was "" and the sql was "select * from foo"
care to spot the invalid code in that?
> > (it's working fine now.)
> >
> >> Quoting is done automatically
> >> when you use
> >
> > I can use sql_quote when I need to, thankyou.
>
> this does not need to work identical - but current
> adaptors indeed do their handmade quoting.
>
> > The only thing that I worry about is transaction support.
> > "insert foo; select last_inserted" works when built inside the template
> > with dtml, but not with the exact same statement passed to this
> > "wrapper". Probably another bug, but I have not taken time to dig
> > properly into it yet.
>
> Yes, MySQL doesnt want ; as sql-delimiter. It rather wants \0 and sql
> methods hack around it for you if you put the template there.
Hm.. interesting, but I looked into it a bit, and I can't find where this
happens.
I see that ZRDB/DA.py does: argdata['sql_delimiter']='\0'
Tried a "query = query.replace(';','\0')" at line 444, but to no effect :)
Suggestions?
I guess this could be a problem with LAST_INSERT_ID, forcing me to use
ZSQLMethods, or dome ugly workaround for inserts, where I need the id
afterwards.
> Transactions are maintained thrughout your session (read: request)
> so there is no need to squeeze it all into one ZSQL call. Every
> subsequent ZSQL call inside your request will be in the same
> database transaction.
>
No plans to squeze it all into one huge SQL statement. Now THAT would be ugly!
but python errors does not trigger rollback it seems. I guess only sql errors
would do that?
regards
Gaute
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Two mysql / timestamp related fixes
On Thursday 08 March 2007 17:53, Paul Winkler wrote: > On Thu, Mar 08, 2007 at 02:07:56PM +0100, Gaute Amundsen wrote: > > this is definitely connected to the timestamp column, because this > > change fixed it: > > > > diff /usr/lib/zope/lib/python/Products/ZMySQLDA/db.py > > /usr/lib/zope/lib/python/Products/ZMySQLDA/db.py~ 173d172 > > < conv[FIELD_TYPE.TIMESTAMP] = DateTime_or_None > > > > I have found no mention of this anywhere? > > Have anyone else had this problem > > Yes, I've observed the same thing, and worked around it with a similar > patch. Good to know. Safety in numbers :) Not very surprising, come to think of it, when: >Latest Release: 2.0.8 >Last Updated: 2001-09-06 Not much use submitting a bugreport either then, I guess? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Two mysql / timestamp related fixes
On Thursday 08 March 2007 17:27, Tino Wildenhain wrote: > Gaute Amundsen schrieb: > > Hi. > > 1 ) > > First I got a ValueError 'query' when using a very simple zsql method. > > (consisting of only "") > > Thats a very bad idea. Better make zsql methods > for every query I know the orthodoxy :-/ I'd rather have a proper DB library, (or model layer, if you will) and build my SQL in python. To build anything complex or generalized in dtml is a PAIN. And a sea of too specific templates is even worse. > instead of > rebuilding your own (obviously not working) > templating for SQL. I's not a template, it's a wrapper to get around the whole mess. A hack in fact ;) If a template this simple "" is "obviously not working" then it is ZSQLMethods that have a problem, not me. (it's working fine now.) > Quoting is done automatically > when you use I can use sql_quote when I need to, thankyou. The only thing that I worry about is transaction support. "insert foo; select last_inserted" works when built inside the template with dtml, but not with the exact same statement passed to this "wrapper". Probably another bug, but I have not taken time to dig properly into it yet. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Two mysql / timestamp related fixes
Hi. Just had som troubles when I added timestamp columns to a table. Commnets? my setup: Mysql: 4.1.20-log Zope Version: (Zope 2.7.5-1.fc3, python 2.3.4, linux2) 1 ) First I got a ValueError 'query' when using a very simple zsql method. (consisting of only "") (I am not totally sure this is related to the timestamp column) I appled the fix to customDefaultReport.dtml mentioned in http://mail.zope.org/pipermail/zope-db/2005-June/004219.html < --- > 25d24 < 26a26 > This does not seem to be exactly the same fix as described in: http://www.zope.org/Collectors/Zope/1812 Should I have this problem in Zope 2.7.5-1.fc3? 2) ValueError: invalid literal for int(): 0- When trying to do a "select *" this is definitely connected to the timestamp column, because this change fixed it: diff /usr/lib/zope/lib/python/Products/ZMySQLDA/db.py /usr/lib/zope/lib/python/Products/ZMySQLDA/db.py~ 173d172 < conv[FIELD_TYPE.TIMESTAMP] = DateTime_or_None I have found no mention of this anywhere? Have anyone else had this problem Comments? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: returning a macro from a pythonscript
On Friday 02 March 2007 17:06, Tres Seaver wrote:
> Gaute Amundsen wrote:
> > On Friday 02 March 2007 11:20, Andreas Jung wrote:
> >> --On 2. März 2007 11:10:57 +0100 Gaute Amundsen <[EMAIL PROTECTED]> wrote:
> >>> I have tried many varieties, but I can't remember ever getting it to
> >>> work...
> >>>
> > foo = restrictedTraverse('/foo/bar/index.html/macros/mymacro')
> > return foo()
> >
> > But of course foo is a list, and I can't see a way I can use any of the
> > contents..
>
> You can't "use it", only inline it into a calling template. E.g.:
>
> ##Script (Python) "random_macro"
> from random import choice # may need module security assertion
> macros = context.some_template.macros
> macro1 = macros['one']
> macro2 = macros['two']
> macro3 = macros['three']
> return choice((macro1, macro2, macro3))
>
> The template would just do:
>
>
>
>
>
> Tres.
I see..
So it is the use-macro statement that takes the datastructure, and turns it
into a tempate/html.
I my case I think I will rather try to use a wrapper approach.
In script: return container.ajax_tpl(mcr=my_macro_from_big_tpl)
In ajax_tpl:
But it's good to know the other option allso.
Thanks.
On Friday 02 March 2007 22:36, David H wrote:
>
> With Zope 2 I use python scripts to fill my "content" area (2-3 column
> css) with macros like so:
> # python script:
>return container.Budget.BudgetMaint.macros['budgetMaint']
>
> It works
>
> All best,
>
> David H
In other words, a script can return a macro, but is has to be into a
use-macro tag.
OK
Thanks.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: returning a macro from a pythonscript
On Friday 02 March 2007 13:35, jerome prudent wrote:
> What do you want to do with your macro?
>
The usual. Have it display some html :)
I return remplates from scripts all the time, and find it very usefull.
return index_html(foo=bar)
On Friday 02 March 2007 13:41, Tonico Strasser wrote:
> Gaute Amundsen schrieb:
> > foo = restrictedTraverse('/foo/bar/index.html/macros/mymacro')
>
> You can use this macro in a template.
>
>
Yes, I know.
I could have a "wrapper template" containing only the "use macro", and return
that instead, but that's a bit ugly, and just what I wanted to avoid.
The purpose of this is to reuse part of a "regular" template in an ajax call.
Gaute.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] returning a macro from a pythonscript
On Friday 02 March 2007 11:20, Andreas Jung wrote:
> --On 2. März 2007 11:10:57 +0100 Gaute Amundsen <[EMAIL PROTECTED]> wrote:
> > I have tried many varieties, but I can't remember ever getting it to
> > work...
> >
> > A few googles shows me nothing either.
>
> returning a macro?
>
> -aj
Something like this:
foo = restrictedTraverse('/foo/bar/index.html/macros/mymacro')
return foo()
But of course foo is a list, and I can't see a way I can use any of the
contents..
G.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] returning a macro from a pythonscript
I have tried many varieties, but I can't remember ever getting it to work... A few googles shows me nothing either. Any quick hints? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get REST friendly urls from sql database
On Wednesday 21 February 2007 16:27, Paul Winkler wrote: > On Wed, Feb 21, 2007 at 12:32:30PM +0100, Gaute Amundsen wrote: > > On Tuesday 20 February 2007 17:39, Paul Winkler wrote: > > > > > You could instead follow Andreas' suggestion and use a through-the-web > Script. It might look something like: > > article_id = traverse_subpath[-1] > data = context.get_the_data_somehow(article_id) > return data > > Your URLs would then look like: > > http://example.com/folder_id/script_id/article_id I had no idea it could be that simple. And certainly not from Andreas riddles ;) That will do nicely for now. Thanks Gaute > -- > > Paul Winkler > http://www.slinkp.com > ___ > Zope maillist - Zope@zope.org > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get REST friendly urls from sql database
On Tuesday 20 February 2007 17:39, Paul Winkler wrote:
> Another option is that the object at /articles could be an instance
> of a class that looks something like:
>
> class MyArticleContainer():
>
> def __before_publishing_traverse__(self, unused, request):
> # Save the rest of the path as 'traverse_subpath'.
> request.set('traverse_subpath',
> reversed(request['TraversalRequestNameStack']))
> # Tell the publisher to stop traversing now.
> request['TraversalRequestNameStack'][:] = []
>
> def __call__(self, *args, **kw):
> subpath = self.REQUEST['traverse_subpath']
> data = get_data_however_you_like(subpath)
> return data
>
>
> For background, read the stuff about traversal hooks at
> http://wiki.zope.org/zope2/ZPublisher
Hm.. that looks more like what I had in mind when I posted, but allso quite a
bit more overhead than rewriterules.
At least for me that has never written a product.
I will have to think about that one..
Would you care to elaborate a little bit on what the tradeoffs would be?
If I wanted my articles in many variants, ie. a large state space, I guess
this way would be good?
Gaute
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get REST friendly urls from sql database
On Tuesday 20 February 2007 16:37, Andreas Jung wrote: > --On 20. Februar 2007 16:30:43 +0100 Gaute Amundsen <[EMAIL PROTECTED]> wrote: > > Hi. > > > > Normally a url to an article in our CMS system would look roughly like > > this: http://www.dom.tld/aritcles/25245 > > > > Where 25245 would be an object of some metatype, say 'article'. > > > > Now what if I wanted to get the data for that page from an sql server > > instead, I would expect to end up with a url like this: > > > > http://www.dom.tld/aritcles/index_html?id=25245 > > > > Now, having recently read the Roy Fielding paper, and expecting to do > > some caching, I would much prefer to have it the first way, even when I > > have to get the data out of a DB. > > > > Anybody have any ideas on how this might be done? > > How about Apache rewrite rules or a script called index_html that fetches > the object by its ID? That's pretty much boring straight-forward stuff :-) > > -aj I was certain that the "index_html" way required the "?" preceding the arguments to work. Easy enoug to test. Will do. Hm.. but rewrite rules would be better I guess. I was actually considering that solution for something else*, just before posting this.. D'oh! (* the task of having the same objects appear under different urls in different states ) Thanks Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] How to get REST friendly urls from sql database
Hi. Normally a url to an article in our CMS system would look roughly like this: http://www.dom.tld/aritcles/25245 Where 25245 would be an object of some metatype, say 'article'. Now what if I wanted to get the data for that page from an sql server instead, I would expect to end up with a url like this: http://www.dom.tld/aritcles/index_html?id=25245 Now, having recently read the Roy Fielding paper, and expecting to do some caching, I would much prefer to have it the first way, even when I have to get the data out of a DB. Anybody have any ideas on how this might be done? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get modules that are called by external methods reloaded?
On Saturday 20 January 2007 12:00, Andreas Jung wrote: > --On 20. Januar 2007 11:55:18 +0100 Jens Vagelpohl <[EMAIL PROTECTED]> > > wrote: > > That database size isn't too large and will not cause 10 minute startup. > > Only the index file is read in during startup, not the whole database - > > unless your index file does not exist or is corrupt. > > ..which might be the result of an unclean shutdown. > > Andreas That sounds plausible. We don't do restarts during bussines ours unless we absolutely _have_to_ of course. The restarts I am talking about was before we got Varnish up and running, when the frontpage of http://www.dagligvarehandelen.com would take up to one minute to load, and apache server status would report loads of connections in the "reading" state. Thanks for your input. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get modules that are called by external methods reloaded?
On Friday 19 January 2007 19:51, Andreas Jung wrote: > --On 19. Januar 2007 10:36:04 +0100 Gaute Amundsen <[EMAIL PROTECTED]> wrote: > > Subject says it sall. > > Only way I have found so far is to restart zope, and on a production > > server, that is not very popular. Takes about 10 minutes on ours.. > > 10 minutes is too long. Even Plone starts on modern hardware in about 30 > seconds. Are you using persistent ZEO client caches are checked against the > ZEO server during the startup phase? > > -aj Hm... We are using the setup right "out of the box" + apache, with Varnish in front of the most heavily loaded sites. Right now (~24:00), it restarted in about aminute actually, but considering that our DB size is 7030.2M and it contains about 70 virtual hosts, does 10 minutes still seem to long when under heavy load? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get modules that are called by external methods reloaded?
On Friday 19 January 2007 19:45, Dieter Maurer wrote: > Gaute Amundsen wrote at 2007-1-19 10:36 +0100: > >Only way I have found so far is to restart zope, and on a production > > server, that is not very popular. Takes about 10 minutes on ours.. > > You can use Python's "reload" builtin to reload them. > > Be warned however, that you must reload also any module that contains > (has imported) functions defined in the reloaded module. Ah.. to late now, but I wil keep that in mind next time :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get modules that are called by external methodsreloaded?
On Friday 19 January 2007 13:42, Jonathan wrote: > - Original Message - > From: "Gaute Amundsen" <[EMAIL PROTECTED]> > To: > Sent: Friday, January 19, 2007 4:36 AM > Subject: [Zope] How to get modules that are called by external > methodsreloaded? > > > Subject says it sall. > > Only way I have found so far is to restart zope, and on a production > > server, > > that is not very popular. Takes about 10 minutes on ours.. > > AFAIK there is no way to reload a module called by an External Method > without restarting Zope. I have encountered this situation many many > times. Ok, then :-( > If at all possible you should set up a development server so that > you can do this more quickly (my restarts take about 10-15 seconds) and not > impact production users. > > If you are really stuck with a single server for production and development > one possible work-around is that you could copy the module source code into > your external method, develop/debug, then move it out into its own file > when has been completed. > > Jonathan It was just during a maneuver like that, that I happened to reload a page, before i had swapped two lib folders... Ok. Thanks. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How to get modules that are called by external methods reloaded?
On Friday 19 January 2007 12:13, Maciej Wisniowski wrote: > > Subject says it sall. > > Only way I have found so far is to restart zope, and on a production > > server, that is not very popular. Takes about 10 minutes on ours.. > > Have you tried to go into Edit form of your external method > and just hit Save button? > This should tell you that External method was updated. Most definitely. That updates everything in the file in the Extensions folder, but noting that this file imports. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] How to get modules that are called by external methods reloaded?
Subject says it sall. Only way I have found so far is to restart zope, and on a production server, that is not very popular. Takes about 10 minutes on ours.. Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: How best to copy and paste in pythonscript?
Thank you, thank you thank you all! Some actuall helpfull advice, and not snotty riddles! I'll try not to feed the troll next time. > I think the keyword argument bit is a red herring. The issue here is > that the methods you are calling are not really designed to be scripted > within the context of a single request: 'manage_copyObjects' sets a > cookie in the *response* object, while 'manage_pasteObjects' looks for > that cookie in the *request* object. This works fine when used as > designed: the cookie from the response gets sent back in the next > request, which can then figure out which object(s) to paste. That is what I was trying to do. That is, manage_copyObjects in one script/ page and manage_pasteObjects in another. I have used both in one script before, with the clipboad object, with no problems. > you might > pass None as clipboard object and the method will try to get the value > from the REQUEST. But in this case you must pass in the REQUEST as keyword > parameter and not as positional argument. I've got to read up on the nuances of keyword vs posissional arguments, I think. I have not gotten around to testing it yet, but it rings true. > But back to the original discussion API vs. implementation. That was not the original discussion, that was your private hobbyhorse. > If you are a frequent reader of this list, you know that AJ's style is to > encourage posters to 1) be complete and precise in the questions they post > and 2) provide just enough infomration for the poster to discover the > answer to his/her post himself. Â This is an excellent pedagogical > strategy but sometimes is frustrating. I know, I know. Now, I asked a simple question, for wich a complete systems description is superfluous, I indicated that I had done a reasonable amount of homework, and I think that a simple answer would be appropriate. Now if he thinks it is his place to educate me with riddles, and about how I should post my questions, it is perhaps only fitting for me to do a bit of educating in return about how to actually read the question, and show some basic human manners. Whith friends like that, zope sure dont need enemies. >No, it's personal laziness to provide the food bite-size :-) It is not I'm afraid. As a programmer you well know that this kind of lazyness uppfront brings you more work afterwards. I can only conclude that your ego gratification is more important to you than actually helping. Now if I have the stamina, to keep educating you back, or if I will just go away somewhere quiet and grep the source instead, that remains to be seen I guess :) G. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How best to copy and paste in pythonscript?
On Monday 28 August 2006 14:06, Andreas Jung wrote: > --On 28. August 2006 14:00:25 +0200 Gaute Amundsen <[EMAIL PROTECTED]> wrote: > > On Monday 28 August 2006 13:16, Andreas Jung wrote: > >> - Docfindertab > >> - lib/python/OFS/interfaces.py (!) > >> - lib/python/OFS/CopySupport.py > >> > >> -aj > > > > Excuse me, my oh so knowledgable friend, but it is not at all obious to > > me, ( or most other readers of this list, I'd wager) what or where > > "Docfindertab" is. > > You might google for Docfindertab and click on the first hit. I think > that's a task you can perform on your own? > I am not in the habit of installing products on our server just because some self important twat, who could not be bothered to write "try the Docfindertab product", mentions a name that might refer to that product. Now you have spent twise as much time typing the reply, as you would have if you had been clear from the start. Not to mention nurturing my growing desire to switch to ROR at the first oportunity, and demonstrating to potential zope users on the list that asking for help wil just get you insulted. Nice. > > On the zope site perhaps, or maybe on _your_ os? > > ? > > > But knowing you a bit by now, me thinks you are infact referring to the > > source.. > > Some API doc that! > > > > Actually, I am starting to think there should be a big notice on the > > frontpage of the http://www.zope.org/Documentation/ section saying: > > > > "don't bother with this, just read the source" > > > > I am sure zopes momentum as a platform would improve greatly, > > and you would not have to bother answering such stupid questions. > > The documentation situation in Zope 2 land is as it is and it won't change > much. And as a Zope developer one should know that APIs are usually > documented in interface files (that's why I mentioned them > *before* the source code file). > > -aj On my installaton there is no interfaces.py for OFS, nor for hardly anything else. But never mind, you can spare me you nuggets of wisdom, they are not worth the bother. G. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How best to copy and paste in pythonscript?
On Monday 28 August 2006 13:16, Andreas Jung wrote: > - Docfindertab > - lib/python/OFS/interfaces.py (!) > - lib/python/OFS/CopySupport.py > > -aj Excuse me, my oh so knowledgable friend, but it is not at all obious to me, ( or most other readers of this list, I'd wager) what or where "Docfindertab" is. On the zope site perhaps, or maybe on _your_ os? But knowing you a bit by now, me thinks you are infact referring to the source.. Some API doc that! Actually, I am starting to think there should be a big notice on the frontpage of the http://www.zope.org/Documentation/ section saying: "don't bother with this, just read the source" I am sure zopes momentum as a platform would improve greatly, and you would not have to bother answering such stupid questions. G. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] How best to copy and paste in pythonscript?
On Monday 28 August 2006 12:21, Andreas Jung wrote: > --On 28. August 2006 12:08:16 +0200 Gaute Amundsen <[EMAIL PROTECTED]> wrote: > > Hi. > > > > I am having some strange problems getting copy and paste to work > > propperly in python script > > > > Basically I do: > > > > parent.manage_copyObjects([currFold.id], REQUEST) > > > > in one script, and: > > > > dest_fold.manage_pasteObjects(REQUEST) > > You must check the APIs carefully. manage_pasteObjects() > takes REQUEST as keyword parameter but not as positional > parameter. So you must pass is it as keyword parameter! > > -aj So you are saying that parent.manage_copyObjects([currFold.id], REQUEST) is ok. and dest_fold.manage_pasteObjects(REQUEST=context.REQUEST) should be ok? When you mention it, where exactly is the APIs? A search of the zope site for "manage_copyObjects" givs me two largely irrelevant hits. Looking at this for example: http://docs.neuroinf.de/api/plone-api/public/OFS.CopySupport.CopyContainer-class.html ( which is one of the first things google throws up ) gives no indication that REQUEST is a keyword parameter. G. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] How best to copy and paste in pythonscript?
Hi. I am having some strange problems getting copy and paste to work propperly in python script Basically I do: parent.manage_copyObjects([currFold.id], REQUEST) in one script, and: dest_fold.manage_pasteObjects(REQUEST) in another I have tried a lot of variations, and read all that I could goolge about it, but stil it won't work. Could somebody please tell me the "canonical" way of dooing this? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] manage_changeProperties has strange side effect on response Content-Type.
On Friday 16 June 2006 20:28, Dieter Maurer wrote:
> Gaute Amundsen wrote at 2006-6-16 13:05 +0200:
> >All my pages normally have a content-type of ISO-8859-1
> >I have not spent much thought on this, it has just worked nicely like
> > that.
> >
> >Today I had a page that called a python script, that even when I made it
> >return an empty string, changed the content-type of my page to utf-8.
> >
> >I finally tracked it down to this difference:
> >
> >item.manage_changeProperties({'title':'Hustype'})gives utf-8.
>
> "manage_changeProperties" has two usage modes: one for programmatic
> use and one for ZMI use. As many other management functions,
> it distinquishes between the two mode by checking whether
> "REQUEST" is "None". When used from the ZMI, the ZPublisher
> passes in "REQUEST" which therefore is not None.
>
> You passed a positional argument to "manage_changeProperties"
> which it interprets as "REQUEST". Therefore, it thinks it were
> used from the ZMI and uses the "management_charset" (or something
> like that) as encoding.
>
> Use "manage_changeProperties(**dict)" instead of
> "manage_changeProperties(dict)" to avoid this problem.
Makes perfect sense when you put it like that, I guess.
But I woluld never have guessed it from reading the APIdoc...
I would have liked to ad a comment about this in the doc, but that function
seems to be out of order for both FF, and Opera.
Thanks anyway.
Gaute
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] manage_changeProperties has strange side effect on response Content-Type.
On Friday 16 June 2006 13:39, Tino Wildenhain wrote:
> Gaute Amundsen schrieb:
> >
> > item.manage_changeProperties({'title':'Hustype'}) gives utf-8.
> > item.manage_changeProperties(title='Hustype') leaves
> > ISO-8859-1 intact.
> >
> > WTF is that?
>
> My guess would be the dict-argument is used in place of
> REQUEST and therefore sets some default encoding in
> request - which it certainly acquires.
>
> But this is all looking into the crystal sphere because
> I not yet have time to actually look at the source -
> otoh it would be of great help if you provide:
>
> 1.) the excat zope version
> 2.) manage_page_charset settings
> 3.) default encodings set in zope.conf etc.
>
> Regards
> Tino Wildenhain
That's making more sense :)
I was suspecting something like that.
I should have learned to ignore Andreas Jung's arrogant misreadings last
time. :(
1)
Zope Version: (unreleased version, 2.7.0-based, python 2.3.3, linux2)
2)
A few quick googles was not able to reveal where I can get hold of the
manage_page_charset settings...
3)
No mention of "char" or "encode" in zope.conf
G.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] manage_changeProperties has strange side effect on response Content-Type.
On Friday 16 June 2006 13:14, Andreas Jung wrote:
> --On 16. Juni 2006 13:05:09 +0200 Gaute Amundsen <[EMAIL PROTECTED]> wrote:
> > item.manage_changeProperties({'title':'Hustype'}) gives utf-8.
>
> This makes less sense. You're passing an ASCII string which has the same
> representation in UTF-8..so why should this be a utf-8 string?
You ask me?
I ask you!
(I think the string is rather irrelevant here, It's the form of the agument..)
> > item.manage_changeProperties(title='Hustype') leaves
> > ISO-8859-1 intact.
>
> see above.
>
> > WTF is that?
>
> Your example is just poor.
It's all there is. Deal with it. (if you like)
> > Now you have been warned :)
>
> Another try?
>
Yes, please do.
G.
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] manage_changeProperties has strange side effect on response Content-Type.
This gave me such a headache today, that I thought I ought to mention it.
All my pages normally have a content-type of ISO-8859-1
I have not spent much thought on this, it has just worked nicely like that.
Today I had a page that called a python script, that even when I made it
return an empty string, changed the content-type of my page to utf-8.
I finally tracked it down to this difference:
item.manage_changeProperties({'title':'Hustype'}) gives utf-8.
item.manage_changeProperties(title='Hustype') leaves ISO-8859-1
intact.
WTF is that?
Now you have been warned :)
Regards
Gaute Amundsen
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Friday 21 April 2006 21:04, Dieter Maurer wrote: > Gaute Amundsen wrote at 2006-4-21 11:01 +0200: > >On Thursday 20 April 2006 20:11, Dieter Maurer wrote: > >> Gaute Amundsen wrote at 2006-4-19 19:03 +0200: > >> >I want to retrieve the form fields in the order that they where > >> > submitted in the http POST. I know this has been brought up before, > >> > and that the answer is "You cant." > >> > >> You might be able to do it. > >> > >>Try "REQUEST['BODY']". For a "POST" request, it should > >>contain the entity content. > >> > >>Of course, you would have to decode it yourself ;-) > > > >What? It can't be that simple! > >I thougt I had been poking all around the corners of REQUEST, looking for > > just somthing like that. > > > >Hm.. no. > > > >Error Type: KeyError > >Error Value: 'BODY' > > > >Are you sure about this? > > I was wrong. > > "BODY" is only defined in the "read_single" case > (see "cgi.FieldStorage" for details) -- this means: > not for "application/x-www-form-urlencoded" nor > for "multipart/" content types (and therefore not > for general "POST" requests as I claimed). > > "FieldStorage" however still preserves the field order > (in "FieldStorage.list"). The order is lost > in "ZPublisher.HTTPRequest.HTTPRequest.processInputs". Hm.. you have had a second look I see :) Now REQUEST.stdin and cgi.parse_qsl solved my problem neatly, and this kind of "under the hood" work stil is a bit of a chalenge for me, so I'm afraid l will not work on a more "proper" solution this time. But thanks for looking into it. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: REQUEST.form variables order
On Friday 21 April 2006 13:36, Tres Seaver wrote: > REQUEST.stdin is the actual stream (either a tempfile or a StringIO). > You will likely need to rewind it before you can use it. > That seems to work YES. with cgi.parse_qsl I'm practically done already. Only some unicode troubles with norwegian charcters to iron out. Thanks :-D Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 20:11, Dieter Maurer wrote: > Gaute Amundsen wrote at 2006-4-19 19:03 +0200: > >I want to retrieve the form fields in the order that they where submitted > > in the http POST. I know this has been brought up before, and that the > > answer is "You cant." > > You might be able to do it. > >Try "REQUEST['BODY']". For a "POST" request, it should >contain the entity content. > >Of course, you would have to decode it yourself ;-) What? It can't be that simple! I thougt I had been poking all around the corners of REQUEST, looking for just somthing like that. Hm.. no. Error Type: KeyError Error Value: 'BODY' Are you sure about this? In that case, any idea why it is missing? We are on Zope 2.7.5-1.fc3, python 2.3.4, behind Apache/2.0.52 Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 17:16, Tino Wildenhain wrote: > > Any hints to get me statred on the zpt version? > > No, you would not do this in ZPT. ZPT is the templating > engine. You write your logic in python. External Method > for example if you dont want to write a complete product. I have the feeling you don't quite understand what Andrew Milton is suggesting. I don't have the time to look into it properly now, but as I understand it, that templating engine can be instantiated independently, and it's methods called with my own data. But thank you for the code and the effort :) I might yet decide to use my own parser to extract the ordering info, but I will try the ZPT engine first. It has the potential to render the form even closer to how the users made the layout, but I must be able to strip out the menus etc. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 15:46, Tino Wildenhain wrote: > Gaute Amundsen schrieb: > ... > > > The order of the form elements that goes into mail headers is ofcourse > > irelevant. I'ts the rest of the form, you know name, adress, street, etc. > > that are the problem. > > It's a purely visual thing, but when you have a form with perhaps 50 > > fields, that the client has carefully grouped and ordered, they can get > > rather pissed if you try to tell them they can only have it in > > semi-random or alpabetic in their mail. > > ... > > > A smiley or two helps, but now I would say you are bordering on arrogant. > > What more do I have to explain to convince you that I understand what the > > problem is? > > We would have saved time if you provided the very usefull information > first :-) > Well, it's amazing how what seems clear to one can be quite opaque to another. I will try to be more explicit next time. > Now you are building some kind of table/list with > form-field-name: form-field-value - am I right? > > how is it supposed to handle checkboxes, radiobuttons > and select fields? > Hm.. I can't recall how I did that. I just made a reiplemetation of how formmail.pl did it. But anything it does, it does looping thru request.form, so I dont think this is relevant. > One possible workaround, if you dont want to touch > ZPublishers form handling would be to run a script > to not only update the forms target (formmail.pl -> zopeform) I use apache "proxy rewrite" for that, no update needed. > but split every form element from > > > into > > > > > which you easily get as list of name/value pairs in > the form variable "body". > > You can even make this transformation any time a user edits > her HTML source - save the users source in a property and > transform this source via some regex or HTML parser > to what you really want here. > > Moderate work and you can even add some sanity checks :-) > > If you can provide some typical samples of the HTML you > are facing I believe you even can get help with the > transformation script. > > Regards > Tino I have considered a number of variations along these lines. Extracting the ordering information and adding a hidden field is also a posibilty. But the potential for messups, and big pain, with a script altering large amounts of user content, is not inconsiderable I would say. I still think my own idea of adding a small proxy to transparently add that "hidden field" is rather more elagant. I expect I will go with Andrew Milton's idea however, since that keeps us inside zope, and seems simpler to implement. I would have prefered to go to the root of the problem, that allways works best in the long run, but it seems I have managed to avoid the effort this time. I still think it is something zope should handle, but for me it is only a sort of medium-term stop-gap measure, so it will do. Thanks for your attention :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 14:01, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | On Thursday 20 April 2006 13:23, Andrew Milton wrote: > | > +---[ Gaute Amundsen ]-- > | > > | > | On Thursday 20 April 2006 12:15, Andrew Milton wrote: > | > | > +---[ Gaute Amundsen ]-- > | > | > | > | > | But, why render the page, when I could just parse it to get the field > | > | order? Fun with regexp, or dom :) > | > > | > Because you can just throw it at the ZPT or DTML engines, convert to > | > text and as a side-effect you get something ready to put into the > | > email? > | > > | > If you want to write your own parser, feel free. > | > | I should have suspected :) > | > | Any hints to get me statred on the zpt version? > > I don't know what you have now, so no not really. > > PageTemplate can be imported from Products.PageTemplates > It as a write method that lets you put the html into it. Thanks. I will get back to the list when I am done, but it may not happen in a few days. Gaute. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 13:23, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | On Thursday 20 April 2006 12:15, Andrew Milton wrote: > | > +---[ Gaute Amundsen ]-- > | But, why render the page, when I could just parse it to get the field > | order? Fun with regexp, or dom :) > > Because you can just throw it at the ZPT or DTML engines, convert to text > and as a side-effect you get something ready to put into the email? > > If you want to write your own parser, feel free. I should have suspected :) Any hints to get me statred on the zpt version? Gaute. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 12:15, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | Hm.. ok. > | I have these 100 forms, all over the place. They live in user editable > | html. To have to track down and alter them all is out of the question. > | Yet the users expect the form fields to appear in their emails in the > | order that they put them on the page. > > Render the page with the fields filled and convert to text and place in > email. Hm.. now that is an idea... I would have to depend on referer I guess, but as long as it degraded gracefully.. But, why render the page, when I could just parse it to get the field order? Fun with regexp, or dom :) It would be a bit of overhead, but so would a php proxy, and the few high-use forms I can rewrite anyway. I think we are onto something here :-D Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Thursday 20 April 2006 12:09, Tino Wildenhain wrote: > Gaute Amundsen schrieb: > > On Wednesday 19 April 2006 21:47, Tino Wildenhain wrote: > >>Gaute Amundsen wrote: > >>>I want to retrieve the form fields in the order that they where > >>> submitted > >> > >>I'd say blaming zope for not reading the docs is somewhat arrogant .-) > > > > I have been reading the docs for thee last 3 years my friend. > > Could you please specify _which_ docs you are refering to here? > > And what do they say that has a bearing on this? > > For example how to construct sane formulars? ;) > Being didactic now eh? ;) Well, yes, formularor is quite nice for example, but this is "someone elses mess", and my job :( > > Hm.. ok. > > I have these 100 forms, all over the place. They live in user editable > > html. > > Bad concept, but there you are. I bet this isnt the only thing you > want to change while you go to an application server (like zope). We have been on zope for years, we just wanted to tighten down the security after one rather nasty attack, and formmail.pl was the only cgi on the server. User editable forms the "right way", now that is another story. I think I have settled on a "separate forms editor" approach, but that is still in the future. > > To have to track down and alter them all is out of the question. > > Yet the users expect the form fields to appear in their emails in the > > order that they put them on the page. > > Well now thats interesting. I see to: cc: subject: and body > elements and their respective form elements. What kind of > other order should be preserved? Its does not matter in which > order mime-headers appear (in fact you dont even see that > in most MUAs) You missunderstand again I'm afraid. I didn't think I was being that brief.. The order of the form elements that goes into mail headers is ofcourse irelevant. I'ts the rest of the form, you know name, adress, street, etc. that are the problem. It's a purely visual thing, but when you have a form with perhaps 50 fields, that the client has carefully grouped and ordered, they can get rather pissed if you try to tell them they can only have it in semi-random or alpabetic in their mail. > > You seem to know formmail.pl, so you know that part :) > > Not the field ordering part. What other elements exist which > need to preserve arbitrary ordering? (Beside the effect > if you have more then one to: oder more then one body > form element (same name) you get a list with ordering > preserved obviously. > > >>>My best Idea so far, is to use the apache proxying we use anyway, to run > >>>all requests to this url through a small perl or php proxy, that ads the > >>>ordering info to the POST. > >>> > >>>Does anyone have any better ideas? > >> > >>Yes. Give the description of your real problem. With more details please! > >>Show example form as it is and where exactly your problem is. > > > > How to retrieve the field order from the http post request. > > No, thats not your problem. Thats the solution you are seeking > to what you think is your problem :-) Maybe it is maybe not. > Please show us an example of the situation you are facing. > I believe there can be a simpler solution to it. > > Regards > Tino Wildenhain A smiley or two helps, but now I would say you are bordering on arrogant. What more do I have to explain to convince you that I understand what the problem is? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: REQUEST.form variables order
On Wednesday 19 April 2006 21:26, Tres Seaver wrote: > Gaute Amundsen wrote: > > I want to retrieve the form fields in the order that they where submitted > > The answer isn't arrogant, and we aren't *preventing* you from gaining > access to that information. The implementation which we use for > extracting the form data from the request is the standard Python > 'cgi.py' module: it doesn't preserve that information, so we can't pass > it on to you. That we accept this state of affairs is a sign that we > don't share your itch, not that we doubt you have it. > > Given that this is open source, you *can* modify ZPublisher such that it > uses a different mechanism to parse the form data, and are free to > submit back such a patch to be included in a future Zope release. > > > Tres. I was waiting for that. "fix it yourself". Yes, I know. Maybe I should have said "patronizing" instedad... Zope uses 'cgi.py', that is interesting. Thank you. The rest is, sorry to say it, redundant. Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] REQUEST.form variables order
On Wednesday 19 April 2006 21:47, Tino Wildenhain wrote: > Gaute Amundsen wrote: > > I want to retrieve the form fields in the order that they where submitted > > I'd say blaming zope for not reading the docs is somewhat arrogant .-) I have been reading the docs for thee last 3 years my friend. Could you please specify _which_ docs you are refering to here? And what do they say that has a bearing on this? > > To painstakingly alter them all to add this sequence ordering > > information, just because zope refuses to do as every other scripting > > system out there that I am aware of, is just out of the queston. > > Sorry, not getting the problem. Last time I saw the formholemail.pl, > it used named form variables. You can easily access form variables > by name in Zope. Hm.. ok. I have these 100 forms, all over the place. They live in user editable html. To have to track down and alter them all is out of the question. Yet the users expect the form fields to appear in their emails in the order that they put them on the page. You seem to know formmail.pl, so you know that part :) > > My best Idea so far, is to use the apache proxying we use anyway, to run > > all requests to this url through a small perl or php proxy, that ads the > > ordering info to the POST. > > > > Does anyone have any better ideas? > > Yes. Give the description of your real problem. With more details please! > Show example form as it is and where exactly your problem is. > How to retrieve the field order from the http post request. I am afraid the problem is as abstract as that. No clever workarounds on the "single form level" will do. > I've yet to see a form problem zope cant handle (provided > the information is actually sent by the browser) > > Regards > Tino Glad to hear it. Tres Seaver told me that zope uses cgi.py Does that mean that I could override dict in that file, or just for FieldStorage, with something like the Ordered Dictionary described at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/107747 and request.form would keep the order? Or better maybe just modify FieldStorage along similar lines? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] REQUEST.form variables order
I want to retrieve the form fields in the order that they where submitted in the http POST. I know this has been brought up before, and that the answer is "You cant." http://mail.zope.org/pipermail/zope/2005-October/162460.html For zope to refuse to expose this information to me is really quite arrogant and un-unix like behavior. After all unix lets you do "rm -rf /"... Now in addition to that, it happens that I have almost a hundred forms scattered around maybe 50 sites that all used to use the formmail.pl script, that now use our own zope based workalike. That is, it can not "work-alike" because zope discards the information I need to make it so! To painstakingly alter them all to add this sequence ordering information, just because zope refuses to do as every other scripting system out there that I am aware of, is just out of the queston. My best Idea so far, is to use the apache proxying we use anyway, to run all requests to this url through a small perl or php proxy, that ads the ordering info to the POST. Does anyone have any better ideas? Get hold of the original http data somehow and parse them myself? Patch zope to use a dictionary type that keeps order for req.post? Patch zope to save the ordering as a property in req.post? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] "roles" argument to validate/ authorize
Can anyone tell me what the argument "roles" to the methods validate and authorize is supposed to be? This is sort of a followup to the thread about exUserFolder that Andrew Milton got stuck answering all on his own (thanks :-), but I have a feeling there is a more general explanation... The reason I ask is that I suspect that a previous developer has done something that has made "roles" contain 'Anonymous user' when it should not, and that this interferes with the propper functioning of login at our sites. Among other things we have our own virtual host system, and several products including document types, so there is scope for a lot of things to interfere... However, while no longer being a newbee at this, I have not yet been able, to find a way to unravel where the "roles" argument originate, and who could have touched it.. Regards. Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] exUserFolder will not redirect to login page
On Saturday 03 December 2005 01:30, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | > Since your index_html and docLogin both seem to require permissions to > | > view, > | > | No, no, and NO again. docLogin noes NOT require permission. > | I can access it without problem. If I cold not this would probably be a > | simple problem, and I would not be posting to the list. > | > | > I would check to make sure that your 'header' and 'footer' items > | > aren't doing something restricted. > | > > | > If you're using DTML, then I'd check that standard_html_header and > | > standard_html_footer. > | > > | > Make sure that in addition to the 'View' permission that also the > | > 'Access Contents information' permissions are set for Anonymous > | > on headers, footers and docLogin (and index_html if required). > | > | I am sorry, but I have been down that path numerous times, and it is all > | in order. > | > | I remove 'view' permission for anonymous from a folder or index.html file > | way below acl_users, and I get the described problem when I try to access > | it. I restore that permission, and everything displays properly. > | If I go to acl_users/docLogin directly, I can log in with cookies, and > | everything works fine. I believe that neatly eliminates the concerns you > | raise here. > > Not really d8) The primary cause for getting a Basic Auth Pop is because > your login form is attempting to do something that an Anonymous user > doesn't have permission to do. Hm.. so if I have /foo/bar/baf/, and baf/ is protected, but I can view /foo/acl_users/docLogin, you think that docLogin still could be trying to touch something restricted? Upgraded to 0_20_1 now, but sill the same behavor. When I create an index.html inside acl_users it displays without complaint as well. I created a new acl_users from scratch, but no difference. > There's another acl_users between > exUserFolder and the content item (so it's getting the wrong user folder, > unlikely in this case), or you have an item in your page, header, or > footer that is duplicated at a lower level that has permissions missing.. You mean that /foo/bar/baf/index.html refers to /image.jpg which is outside /foo/acl_users/ "jurisdiction", and that triggers the Basic Auth Pop, after login succeeded and redirected to /foo/bar/baf/index.html? But if I remove the protection on baf/ and /foo/bar/baf/index.html then does not trigger the B.A. pop, how could that be? I suppose /image.jpg could be restricted for authenticated users, but not for anonymous, but that seems rather farfetched, and would not the error message indicate that? > E.g /a/b/c/docLogin works fine, but, /a/b/c/d/e/docLogin doesn't because > something at /a/b/c/d or lower is locked for Anon users, but, isn't at > /a/b/c In my terms: that image.jpg exists both at / and at /foo/bar/baf/, and that the last one is restricted beyond what I have access to even after having been sucsessfully logged in via /foo/acl_users/? > Does that make sense ? Sort of :) "successful login makes acquisition pick up something that is still restricted, and this triggers a B.A. pop before anything is displayed" Is that about it? Only it is slightly beside the point since my problem is not after login, but before. In other words that given /foo/bar/baf/index.html with baf/ restricted and the nearest UF at /foo/acl_users/, when I try /foo/bar/baf/index.html, I do not get redirected to /foo/acl_users/docLogin, but get the B.A. pop. I suppose something in the redirect part could be triggering the ba pop, sort of "on the way" to /foo/acl_users/docLogin. Ther is one on indication something like this is afoot, and that is that after I enabled debug messages after upgrading, the log reports two times in a row: DEBUG(-200) exUserFolder identify returned None, None with exactly the same timestamp. I will look into this now.. > | I was hoping it would be a product that many would be familiar with, and > | that as such it wold be a good reference point to grapple with the more > | general principles. Noting beats assistance from the author of course, > | but please don't insist on treating this as a newbee question :) > > Maybe if you saw my INBOX you'd understand d8) Well, I can imagine :) These are not easy matters to write clearly and simlpy about, but I hope this is better. Thanks for your patience :) G. -- -- - Gaute Amundsen "Technology today is the campfire [EMAIL PROTECTED] around which we tell our stories.
Re: [Zope] exUserFolder will not redirect to login page
On Friday 02 December 2005 14:46, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | Sorry about that. > | Thought I had checked that, but I must have mixed it up with > | MysqlUserFolder, which setup alerted me to this whole mess. > | > | Will try an upgrade, but since MysqlUserFolder displays the same problem > | I suspect I will be back shortly :-/ > > Given there's two user folders giving you the same response, it's unlikely > to be the same bug in two different user folder implementations :-) My thoughts exactly. It's worth a try anyway at this stage, but I have not had time for that yet. > Since your index_html and docLogin both seem to require permissions to > view, No, no, and NO again. docLogin noes NOT require permission. I can access it without problem. If I cold not this would probably be a simple problem, and I would not be posting to the list. > I would check to make sure that your 'header' and 'footer' items > aren't doing something restricted. > > If you're using DTML, then I'd check that standard_html_header and > standard_html_footer. > > Make sure that in addition to the 'View' permission that also the > 'Access Contents information' permissions are set for Anonymous > on headers, footers and docLogin (and index_html if required). I am sorry, but I have been down that path numerous times, and it is all in order. I remove 'view' permission for anonymous from a folder or index.html file way below acl_users, and I get the described problem when I try to access it. I restore that permission, and everything displays properly. If I go to acl_users/docLogin directly, I can log in with cookies, and everything works fine. I believe that neatly eliminates the concerns you raise here. To be frank I really think my first post described the situation quite precisely. I do not think exUserFolder is faulty in any way, except possibly by our own modification, but that is what I have to work with to try to figure out what is going on. I was hoping it would be a product that many would be familiar with, and that as such it wold be a good reference point to grapple with the more general principles. Noting beats assistance from the author of course, but please don't insist on treating this as a newbee question :) G. -- -- - Gaute Amundsen "Technology today is the campfire [EMAIL PROTECTED] around which we tell our stories. There's this attraction to light and to this kind of power, which is both warm and destructive." Laurie Anderson http://www.div.org ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] exUserFolder will not redirect to login page
On Friday 02 December 2005 10:45, Chris Withers wrote: > Gaute Amundsen wrote: > > As far as I can see that must mean that either something is going wrong > > in exUserFolder, or much more likely, that we are feeding it something > > bogus, or have previously modified it in some non-obvious way. > > My problem is that I have little idea what it is supposed to look like at > > different stages. > > You could always use SimpleUserFolder and CookieCrumbler ;-) > Good idea. Will give it a try. Even if only to try to isolate the problem. G. -- ----- Gaute Amundsen "Technology today is the campfire [EMAIL PROTECTED] around which we tell our stories. There's this attraction to light and to this kind of power, which is both warm and destructive." Laurie Anderson http://www.div.org ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] exUserFolder will not redirect to login page
>| > I don't know what version of exUserFolder you're using, but, try looking >in >| >| exUserFolder-0-10-10 > >On man, this is positively ancient. > >Move to 0.20.x or 0.50.x and try again. > >If you're already setup, I suggest moving to 0.20.x, since 0.50.x involved a >code-reshuffle, and ZODB pickles don't like that so much :-) Sorry about that. Thought I had checked that, but I must have mixed it up with MysqlUserFolder, which setup alerted me to this whole mess. Will try an upgrade, but since MysqlUserFolder displays the same problem I suspect I will be back shortly :-/ G. -- -- ----- Gaute Amundsen "Technology today is the campfire [EMAIL PROTECTED] around which we tell our stories. There's this attraction to light and to this kind of power, which is both warm and destructive." Laurie Anderson http://www.div.org ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] exUserFolder will not redirect to login page
On Friday 02 December 2005 04:31, Andrew Milton wrote: > +---[ Gaute Amundsen ]-- > > | Hi all. > | > | I have this problem with our zope based CMS system. > | No matter what i try I can not make the docLogin page appear when it > | should. If I go directly to it, it works fine, but otherwise the http > | auth poppup appears. > > I don't know what version of exUserFolder you're using, but, try looking in exUserFolder-0-10-10 > doc/UnenlightenedZopistasGuide.txt > > 5.0 TIPS FOR THE UNWARY > > Specifically section 5.1.1; > > 5.1.1 The problem > > When you try to access the folder, instead of getting the form, you > get a popup box, even though you chose Cookie Authentication. Even > when you enter a username and password it doesn't work. I have been over that guide a few times after clues, but unfortunately this description only approximately fits my situations. > 5.1.2 What happened > >You tried to access an area you don't have access to. Zope found the >closest user folder to the object you were trying to access. The user >folder decided you were not authorized and tried to display the login >form. You don't have access to view the login form, so Zope finds the >nearest user folder to the login form, which is the user folder above >the protected directory. It pops up the authentication dialog. If you >put in a valid username and password for this top level, then lower >level then displays the login form. As I mention in the part you quote, I have no problem accessing acl_users/docLogin, or indeed logging in using cookies, when I go directly to this url. The problem is that when I go to a protected url I do not get redirected to docLogin, but controll is passed to the acl_users above, as described, and that acl_users only knows http auth. When I forced cookie_validate to return the redirect at the end, I did get the right docLogin as well, as far as I can see, except I got in that redirect loop because of same forcing. That should eliminate the above explanation pretty conclusively, I would think. (in adition to the fact that acl_users is way outside the protected area) As far as I can see that must mean that either something is going wrong in exUserFolder, or much more likely, that we are feeding it something bogus, or have previously modified it in some non-obvious way. My problem is that I have little idea what it is supposed to look like at different stages. Regards Gaute Amundsen -- -- - Gaute Amundsen "Technology today is the campfire [EMAIL PROTECTED] around which we tell our stories. There's this attraction to light and to this kind of power, which is both warm and destructive." Laurie Anderson http://www.div.org ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] exUserFolder will not redirect to login page
Hi all.
I have this problem with our zope based CMS system.
No matter what i try I can not make the docLogin page appear when it should.
If I go directly to it, it works fine, but otherwise the http auth poppup
appears.
I have started looking into exUserFolder, and putting in zLOG.LOG statements
to try to figure out what is going on.
The best description of what is _supposed_ to be going I have found is this:
http://www.zope.org/Members/vladap/mysqlUserFolder/release-1.0.5/README
Section -> "Zope authentication process"
When i log just above "if user != None:" in validate at approx. line 1040 in
exUserFolder.py I get:
auth: None
roles: None
parent index.html
user: Anonymous User
I get this wether index html allows anonymous or not.
Is this propper behavior?
Where does roles come from? Should not that reflect the roles needed for
access?
If access is allowed all the other requests for images and so on have:
auth: None
roles: ('Manager', 'Anonymous')
parent >
user: Anonymous User
I think our CMS has modified the permission system somewhat to allow access to
some specific folders above acl_users, but I have not found those
changes yet. (and the lead developer is very busy on something else, and don't
remember)
I don't know if this is relevant or not, either..
Anyway, it seems a crucial point must be towards the end of cookie_validate
after "if not self.sessionTracking" at approx. line 930.
If I override here and set roles = ('Manager'), i get to
"raise 'LoginRequired'", and it seems like I should end up at
acl_users/docLogin via docLoginRedirect.dtml, except that I go into a
redirect loop because of the same override.
Normally "if nobody.allowed(parent, roles):" is true, and "ob" is returned,
which is 'Anonymous User', even if the file I am accessing is protected.
Should not this be None in the last case?
And if it returns anonymous, then who decides to ask the next acl_user upp the
chain (who only knows http auth)?
How about some way to trace down where roles come from?
Am I on the right track here?
I had a peek in ZPublisher/BaseRequest.py, but that only made me more
confused..
All confused, and ready to give up now.
Any and all explanations, tips, or good ideas appreciated.
Regards
Gaute Amundsen
-- -
Gaute Amundsen "Technology today is the campfire
[EMAIL PROTECTED] around which we tell our stories.
There's this attraction to light
and to this kind of power, which is
both warm and destructive."
Laurie Anderson
http://www.div.org
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Determining user from session object - who is logged in?
That cleared things up nicely. I will dig into ExuserFolder (which is our standard) a bit more before i decide. Thanks :) Gaute On Thursday 13 October 2005 10:25, Chris McDonough wrote: > On Thu, 2005-10-13 at 00:26 +0200, Gaute Amundsen wrote: > > Well, I know that ther might easily be "login less" sessions, > > but, is it possible to log have "session less" logins? > > Yes. In fact, this is the default for Zope "out of the box". Zope's > default user folder uses HTTP basic authentication, which doesn't > require sessions at all. Ditto for applications that use CookieCrumbler > or user folder implementations that use cookies to store credentials. > > Zope does not create a session for every user by default, just because > not all applications require sessions. It's up to the programmer to > create use sessions. Usually this just means putting a mention of > REQUEST.SESSION in one or more of your scripts or views. > > > The first case would not be a problem. I expect one could easily detect > > the lack of userdata, and skip altering the session, and in that case > > would not an addScript let me solve this without resort to external > > methods, or product code? > > addScript... sure, doing it this way is likely easy for any given > application. It just can't be solved generally for all applications > because there's no easily-overrideable "do this at login time" hook > built in to "stock" Zope (although there are definitely hooks for this > in PAS and probably in exUserFolder too). > > - C ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Determining user from session object - who is logged in?
On Wednesday 12 October 2005 22:58, Chris McDonough wrote: > On Oct 12, 2005, at 4:50 PM, Gaute Amundsen wrote: > > On Wednesday 12 October 2005 20:19, Chris McDonough wrote: > >> On Wed, 2005-10-12 at 19:32 +0200, Gaute Amundsen wrote: > >>> I thought the place to start wold be: /temp_folder/session_data/ > >>> > >>> I have built a small little thing to play with the contents of that. > >>> External method and script below. > >>> > >>> However I have not been able to figure out a way to connect these > >>> sessions with "logged in users". > >> > >> There is no built-in connection between the two. If you want to do > >> this, you need to use a user folder implementations that keeps > >> credentials in the session. > > > > Hm.. ok. > > > > So what strategy would you chose? > > > > 1) Tweak ExuserFolder to put credentials in the session. > > > > 2) Use an addScript on session_data to put credentials in the session. > > > > 3) Use 1 or 2 to put credentials somewhere else? > > > > And thanks :) > > Probably 1, because it's possible for a session to be created without > someone being logged in, and exUserFolder is probably really the only > thing that knows when someone logs in and out. > Well, I know that ther might easily be "login less" sessions, but, is it possible to log have "session less" logins? The first case would not be a problem. I expect one could easily detect the lack of userdata, and skip altering the session, and in that case would not an addScript let me solve this without resort to external methods, or product code? Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Determining user from session object - who is logged in?
On Wednesday 12 October 2005 20:19, Chris McDonough wrote: > On Wed, 2005-10-12 at 19:32 +0200, Gaute Amundsen wrote: > > I thought the place to start wold be: /temp_folder/session_data/ > > > > I have built a small little thing to play with the contents of that. > > External method and script below. > > > > However I have not been able to figure out a way to connect these > > sessions with "logged in users". > > There is no built-in connection between the two. If you want to do > this, you need to use a user folder implementations that keeps > credentials in the session. Hm.. ok. So what strategy would you chose? 1) Tweak ExuserFolder to put credentials in the session. 2) Use an addScript on session_data to put credentials in the session. 3) Use 1 or 2 to put credentials somewhere else? And thanks :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Determining user from session object - who is logged in?
I thought the place to start wold be: /temp_folder/session_data/
I have built a small little thing to play with the contents of that.
External method and script below.
However I have not been able to figure out a way to connect these sessions
with "logged in users".
I suppose I could make some add/remove actions to log the data somewhere, and
then get it from there, but there has to be some more direct way...
Any ideas anyone?
Gaute.
--
def CurrentSessions(self):
root = self.getPhysicalRoot()
session_data = root.restrictedTraverse('/temp_folder/session_data')
return session_data.items()
## Script (Python) "view_sessions"
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind subpath=traverse_subpath
##parameters=
##title=
##
def timeSpan(secs):
# seconds -> hours, minutes og seconds
hours = int(secs/3600)
secs = secs % 3600
minutes = int(secs/60)
secs = int(secs % 60)
return "%02d:%02d:%02d" % (hours, minutes, secs)
print 'View session data'
# flush expired objects...?
session = context.session_data_manager.getSessionData()
session.set('This is','me')
data = container.CurrentSessions()
print str(len(data)) + ' objects'
for i in data:
last = i[1].getLastAccessed()
made = i[1].getCreated()
now = DateTime().timeTime()
print i[1].items()
print 'Made: '+DateTime(made).strftime('%Y-%m-%d %H:%M:%S')
#print 'Last: '+DateTime(last).strftime('%Y-%m-%d %H:%M:%S')
#print 'Now: '+DateTime(now).strftime('%Y-%m-%d %H:%M:%S')
print 'Used: ' + timeSpan(last - made)
print 'Idle: ' + timeSpan(now - last)
print ''
return printed
--
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Webdav and emacs, eldav, nd problem.
Hi. We just upgraded to Zope 2.7.5-1.fc3, python 2.3.4, and now I am having problems with authoring "Script (Python)" (using ssh tunnel: "ssh -L 2001:localhost:8081 [EMAIL PROTECTED]") I am using emacs, using eldav, using nd. I have traced the problem to this command issued by eldav in the process of saving changes to a file. "nd -S -p /localpath/script.py -a Zope http://localhost:2001/path/on/remote/server/script.py"; The problem is replicated on the commandline. What happens is that the python script gets saved as Page Template with Content-Type: text/plain. ( I have tried nd's -T option, but to no avail ) As nothing else have changed, ( afaik ) I must assume there has been some changes made to zopes webdav support. Any ideas anyone? Hm.. I notice we have ExternalEditor (0.5) installed. Could this have any influence? Perhaps I will try using it :) Gaute ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Running out of sockets
On Thursday 12 May 2005 20:45, Dieter Maurer wrote: > Gaute Amundsen wrote at 2005-5-12 00:17 +0200: > > ... > >Restarting zope has fixed it for now, but > > ... > >"lsof -a -U -uzope" and "lsof -uzope | wc -l" > >Shows the number of open sockets steadily growing, > >and apparently none closing.. > > You might be able to determine the second partner for the socket > (with "netstat", I think). This may give you a hint > for what purpose the sockets are used. It turned out the problem was MySQL-python-1.0.0-1 vs. Mysql 4.1 Upgrading MySQL-python to 1.2 solved the problem. Thanks Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] How to disable anonymous sessions
I thought this must be in the FAQ or something, but 20 min googling does not turn up much usefull. Closest I got was this: http://svn.nuxeo.org/trac/pub/ticket/600 At the moment we have 657 objects in /temp_folder/session_data and this must be quite a bit of overhead. We also have over a thousand "INFO(0) ZODB conflict error" a day, that I suspect are related to this. This should be simple, but I can not spot it :-( Anyone? Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Running out of sockets
Coming out of lurk mode here with a really tricky one. We had just moved our server from: Zope Version (unreleased version, python 2.3.3, linux2) Python Version 2.3.3 (#2, Feb 21 2004, 21:17:53) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] To: Zope Version (Zope 2.7.5-1.fc3, python 2.3.4, linux2) Python Version 2.3.4 (#1, Feb 2 2005, 11:44:49) [GCC 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)] The first thing we noticed was: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /xp/pub/topp/invest/. Reason: Error reading from remote server Apache/2.0.52 (Red Hat) Server at www.foo.bar Port 80 ( We are running a homegrown virtual host product very similar to the "virtual host monster" behind apache via rewrites. No ZEO.) This seems to corespond to a lot of: PROBLEM(100) ZServer warning: server accept() threw an exception in the log. Restarting zope has fixed it for now, but "lsof -a -U -uzope" and "lsof -uzope | wc -l" Shows the number of open sockets steadily growing, and apparently none closing.. We have not seen the Proxy Error again yet, but it seems pretty certain sockets will run out at some point. What to do? Where to dig? I have found quite a number of references to similar problems by googling parts of this, but nothing leading me closer to figuring it out. "ulimit -a" shows the limit is 1024 ( at 1004 now everything seems fine except the occasional PROBLEM(100) XPTopic seemingly unrelated, not accept() ) "netstat -axp" gives me loads of lines like this: unix 2 [ ] STREAM CONNECTED 2792969 572/python We have tried various combinations for config options, but without any luck: "force-connection-close on" did not seem to make any difference "max-listen-sockets 500" did not keep "lsof -a -U -uzope | wc -l" from exceeding 500. maximum-number-of-session-objects 1 session-timeout-minutes 90 This should not be related no? I realize this is becoming a very rambling question, I just thought I would include as much info as possible right away :) Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
