Re: [Zope-dev] Re: Unsecure design of ExternalFile
On Fri, 8 Nov 2002, Craeg K Strong wrote: OK How about this for the TODO list for ExternalFile: Hope it isn't too late discussing this issue. I have tested this product and gave up because of security considerations. And now I have to use it for large files. There is another aspect that this discussion so far has not reached, multi-users, regardless of what mechanism is going to be used. Say my Zope system provides virtual hosting for webmasters (or users in my point of view) of different websites. Not all webmasters want their ExternalFile-linked file be freely accessed to the public. So how if a webmaster links a file belonging to another website? I have an idea, but don't know whether it is possible: set uid. If there is a way Zope server can change uid to a predefined one before accessing an externally linked file, each webmaster will have permission to their own home directory plus some shared directories to which all webmasters have permission. Then I can create system accounts for each webmaster, and map them to the Zope users using a product like SystemUserFolder (is there one?) And if also add the 'jail' option (or chroot to the the webmasters home directory), it will be perfect. Back one step, even there is no way to actually change the uid, we can at least check again it before adding an external file. I'm talking about Unix, I think their are eqivalent way on Windows NT. BTW, I think a similar product, ZFS, is facing the same securiy issue. Wei He ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] PathIndex unindex fix
Leonardo Rochael Almeida wrote: If you are to ignore errors, please ignore specific errors that are known to happen. Please don't use a generic except. ...or re-raise only ConflictErrors. Hmmm... this issue is perhaps deeper than it first appeared. Are conflict errors swallowed by the other indexes? Should they be? I wonder if the discussion about swallowing ConflictErrors (on zodb-dev IIRC) was ever resolved? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Single-threaded Zope
Dear all, because of reasons that would be boring to explain,I have the following requirement for my site: I need Zope NOT to run as a single-process multithreaded application (as it does by default) but instead to run as a pool of processes (each one single-threaded); those processes should share the same Data.fs database. Shutting down the threads is trivial (using setNumberOfThreads function) but the tricky part (for me, at least) is to keep the ability to serve many requests concurrently. Since I discarded threads, the only other way is to have many Zope instances running (as independent single-threaded processes) and a sort of dispatcher process (proxy server?) that listens for incoming HTTP connections from the outside world and dispatches it to one of the Zope instances (performing also the reverse operation, of course). The dispatcher should also care to create (and destroy) dynamically the Zope instances, according to the load of traffic (number of requests per second). Is there any way to do so without having to implement the dispatcher (proxy) process from scratch? I have been considering FastCGI as a solution, since it allows a web server (say Apache) to spawn many persistent processes to execute the CGI requests it receives but, to my disappointment, I noticed that the mod_fastcgi directive used in all the howtos on the subject is FastCgiExternalServer, which does not allow for the CGI app to be created by Apache itself. FastCgiExternalServer relies on a 'manual' external starting of the CGI app, which can only be ONE process then! There's another directive (FastCgiConfig) that lets Apache spawn the process(es) that execute the CGI app, the question is: can it be used with Zope instead of FastCgiExternalServer? Since (Fast)CGI apps use pipes to communicate with the web server (when they run on the same machine), the problem becomes how to pass a web request directly to the Zope publishing mechanism over a pipe, bypassing the Zope's HTTP server. How can I do that? Someone out there has ever tried to do so? Also, how does Zope support having multiple instances of it sharing the same database (Data.fs)? One solution could be having many ZEO clients on the same machine, but I wonder if it is possible to create those clients dynamically (to scale according to the load of incoming requests) or rather they can only be installed 'statically' (not at run-time, I mean). Thanks in advance for any help you can give, regards, Carlo. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] consistent naming in ZEO releases
Ok, whoever is in charge of zope.com releases of things like DCOracle2, Zope and ZEO need to decide if the release file names are going to end in .tar.gz or .tgz. The inconsistency between releases is quite annoying. I know that I've been hit by it both from DCOracle2 and ZEO. The actual Zope releases have at least been consistent. Sorry for the rant. --andy. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Single-threaded Zope
The solution to your problem is to use ZEO. This allows any number of Zope application servers to share a single ZODB storage. The app servers and storage server do not need to be on the same machine, just connected via a local network. This is the standard way to make Zope scale for large applications. To solve your use, you would just run each Zope app server single threaded. You could run as many of these single threaded, identical Zope app servers as you like. You would then need some sort of load balancer in front of it. Although I haven't used it myself, there is a python-based software load balancer named Pound that could be used. There are also various other software and hardware load balancers available. In general though running many single threaded Zopes will be more resource intensive then running fewer multi-threaded Zopes. OTOH, running it this way would be potentially beneficial when running on a multi-CPU machine due to the Python global interpreter lock. hth, -Casey On Friday 22 November 2002 08:45 am, Carlo Giomini wrote: Dear all, because of reasons that would be boring to explain,I have the following requirement for my site: I need Zope NOT to run as a single-process multithreaded application (as it does by default) but instead to run as a pool of processes (each one single-threaded); those processes should share the same Data.fs database. Shutting down the threads is trivial (using setNumberOfThreads function) but the tricky part (for me, at least) is to keep the ability to serve many requests concurrently. Since I discarded threads, the only other way is to have many Zope instances running (as independent single-threaded processes) and a sort of dispatcher process (proxy server?) that listens for incoming HTTP connections from the outside world and dispatches it to one of the Zope instances (performing also the reverse operation, of course). The dispatcher should also care to create (and destroy) dynamically the Zope instances, according to the load of traffic (number of requests per second). Is there any way to do so without having to implement the dispatcher (proxy) process from scratch? I have been considering FastCGI as a solution, since it allows a web server (say Apache) to spawn many persistent processes to execute the CGI requests it receives but, to my disappointment, I noticed that the mod_fastcgi directive used in all the howtos on the subject is FastCgiExternalServer, which does not allow for the CGI app to be created by Apache itself. FastCgiExternalServer relies on a 'manual' external starting of the CGI app, which can only be ONE process then! There's another directive (FastCgiConfig) that lets Apache spawn the process(es) that execute the CGI app, the question is: can it be used with Zope instead of FastCgiExternalServer? Since (Fast)CGI apps use pipes to communicate with the web server (when they run on the same machine), the problem becomes how to pass a web request directly to the Zope publishing mechanism over a pipe, bypassing the Zope's HTTP server. How can I do that? Someone out there has ever tried to do so? Also, how does Zope support having multiple instances of it sharing the same database (Data.fs)? One solution could be having many ZEO clients on the same machine, but I wonder if it is possible to create those clients dynamically (to scale according to the load of incoming requests) or rather they can only be installed 'statically' (not at run-time, I mean). Thanks in advance for any help you can give, regards, Carlo. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope] Cache Parameters
Is there a way in Zope 2.5 to see what kind of hit/miss ratio our cache is having? Currently we have 260,000 objects in our database and an ideal cache size of 10,000. And I'm not sure how often it's accessing the database. Thanks, -Brian On Fri, 22 Nov 2002, Toby Dickenson wrote: On Friday 22 November 2002 10:18 am, Chris Withers wrote: Brian R Brinegar wrote: Within the Control_Panel/Database/ Cache Parameters there is a Target max time between accesses what is the Unit for this value? Seconds? Minutes? I haven't found it documented anywhere. Best ask Toby Dickenson on the [EMAIL PROTECTED] list, he is the god of all things cache like ;-) No need to go to a specific list. I am everywhere. ;-) That parameter has been removed in Zope 2.6. Previously its units were in seconds, but the parameter that it controls does not relate to anything easily explained. If you think you need to understand this parameter, then I think you need to upgrade to Zope 2.6 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Thank you for your response. How does one determine if they have a reasonably-sized ZEO cache? In our case we have about 260,000 objects in the database with an ideal cache size of 10,000 objects. I have no idea what our hit/miss ratio is or how to find that out. Any assistance would be great! Thanks again, -Brian Brinegar ECN Purdue University On Thu, 21 Nov 2002, Guido van Rossum wrote: We have diskless ZEO clients (Netboot). There is really no reason for our ZEO clients to write their caches to the disk (RAM Disk). Can we turn this off? To free up RAM for the other caches? That's a new use scenario for me. You can't turn the ZEO cache off. But I think you misunderstand the purpose of the ZEO cache. If you don't have a reasonably-sized ZEO cache, you lose big because you have to go to the server for *every* request. The ZEO cache caches a different kind of data than the other caches, and it caches this data *only* to disk, so it's not the case that it's wasting RAM disk space by writing cached data to disk that's also in memory. --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unsecure design of ExternalFile
Wei He writes: ... I have an idea, but don't know whether it is possible: set uid. If there is a way Zope server can change uid to a predefined one before accessing an externally linked file, each webmaster will have permission to their own home directory plus some shared directories to which all webmasters have permission. Forget about this in a multi-threaded process (with almost independent threads). The uid/gid are process global things. Interesting things happen when threads asynchronously change it. In order to be able to switch users, Zope must run with root privileges -- a huge security risk. Then I can create system accounts for each webmaster, Give each webmaster its own Zope process. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Thank you for your response. How does one determine if they have a reasonably-sized ZEO cache? In our case we have about 260,000 objects in the database with an ideal cache size of 10,000 objects. I have no idea what our hit/miss ratio is or how to find that out. Any assistance would be great! Try this: http://cvs.zope.org/ZODB3/ZEO/README.txt?rev=1.4content-type=text/vnd.viewcvs-markup Simpler would be to check your log file for cache flip messages. If you get these more than a few times per day, you should consider doubling the cache size. --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Brian R Brinegar writes: We have diskless ZEO clients (Netboot). There is really no reason for our ZEO clients to write their caches to the disk (RAM Disk). Can we turn this off? To free up RAM for the other caches? With Zope 2.5 and Zeo 1.x, no cache is written unless you define ZEO_CLIENT. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Dieter Maurer wrote: Brian R Brinegar writes: We have diskless ZEO clients (Netboot). There is really no reason for our ZEO clients to write their caches to the disk (RAM Disk). Can we turn this off? To free up RAM for the other caches? With Zope 2.5 and Zeo 1.x, no cache is written unless you define ZEO_CLIENT. Well, that's not quite correct. If you don't define ZEO_CLIENT, it writes a cache using tempfiles. That usually doesn't matter, but in this case it does matter since there are only RAM disks. One option might be to put the caches on a shared disk. This assumes that shared disk access is faster than accessing the ZEO server. However, since ZEO caches are generally relatively small for today's RAM sizes (20 MB), and there's only one such cache per box, just letting ZEO use the RAM disk seems optimal. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Brian R Brinegar wrote: Okay, Tell me where I'm wrong or if I'm lucky and got it right. There is 1 client cache per thread on a ZEO client and 1 ZEO cache per ZEO client. The ZEO cache is shared between the threads. We have 3 machines 4 threads each that's 3 * 4 + 3 = 15 caches? Is this correct? If so what's cached in each of the caches? When accessing the cache parameters from the Control_Panel which caches am I managing? The control panel manages the ZODB cache, of which you have 4 per box. Your custom_zodb.py manages the size of the ZEO cache. Are you using Zope 2.5 or 2.6? You'll really want 2.6 for this, since it constrains the ZODB cache size much better. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
Okay, Tell me where I'm wrong or if I'm lucky and got it right. There is 1 client cache per thread on a ZEO client and 1 ZEO cache per ZEO client. The ZEO cache is shared between the threads. We have 3 machines 4 threads each that's 3 * 4 + 3 = 15 caches? Is this correct? If so what's cached in each of the caches? When accessing the cache parameters from the Control_Panel which caches am I managing? Thanks again, -Brian ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Caches
On Fri, 22 Nov 2002, Shane Hathaway wrote: Brian R Brinegar wrote: Okay, Tell me where I'm wrong or if I'm lucky and got it right. There is 1 client cache per thread on a ZEO client and 1 ZEO cache per ZEO client. The ZEO cache is shared between the threads. We have 3 machines 4 threads each that's 3 * 4 + 3 = 15 caches? Is this correct? If so what's cached in each of the caches? When accessing the cache parameters from the Control_Panel which caches am I managing? The control panel manages the ZODB cache, of which you have 4 per box. Your custom_zodb.py manages the size of the ZEO cache. Are you using Zope 2.5 or 2.6? You'll really want 2.6 for this, since it constrains the ZODB cache size much better. We're running 2.5 and I don't think it's going to be possible for us to upgrade to 2.6 until March because we have a heavily used production system. Currently we don't specify cache_size in our custom_zodb.py. I guess the default is 20 meg. Our Database (packed) is about 6 Gig. -Brian ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Rapid Alternated Propositions
Hi All, 1/ In ZMI : it could be cool, in general, to select all the checkboxes in one time with a little javascript function or something else ... 2/ if it is possible, why not group by product permissions, else in a general category if permissions are commons FR¢zùYb²Öh¥àÞ¿:)zàÛiÿùb²Û3¢¨®æj)fjåËbú?Î^uëÍ¡Êè²Êh²Û(¬tÌ-éܡا¥j×+-²m§ÿåËlÎ^¢¸?¨¥©ÿ+-wèÿ:)y©ç¢éÜzm§ÿåËlÎ^¢¸?¨¥©ÿ+-wèÿ:)
[Zope-dev] Fw: Rapid Alternated Propositions
Hi All, 1/ In ZMI : it could be cool, in general, to select all the checkboxes in one time with a little javascript function or something else ... 2/ In Security ZMI item : Why not group permissions by product or group in a general category when permissions are commons FR ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Fw: Rapid Alternated Propositions
1) Something like the 'Select all' button strategically placed to the bottom right? If it's what you're looking for don't blame yourself for not finding it. -- Helge - Original Message - From: fr [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, November 23, 2002 12:01 AM Subject: [Zope-dev] Fw: Rapid Alternated Propositions Hi All, 1/ In ZMI : it could be cool, in general, to select all the checkboxes in one time with a little javascript function or something else ... 2/ In Security ZMI item : Why not group permissions by product or group in a general category when permissions are commons ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
