Hello, Nice patch. :) Are you still developing it?
2011/1/16 Hannes Landeholm <[email protected]> > Hi, for anyone that's interested I have now done some stress testing and > read the system call docs for linux more carefully and it appears mmap > doesn't guarantee that the memory region is synchronized with the physical > file on disk (It explicitly tells you to use msync for this), which is a > good thing as there would then probably be no/little disk write overhead. > The only overhead really is the extra disk space. Which might be a good > thing as the cache file acts as a virtual cache/swap for PHP. So you can > let all children for a certain virtual host die, freeing up memory, but the > PHP file cache will remain intact. > > In the stress test I did 3 runs against the index of a copy of a > production wordpress site (wordpress is good as it's crazy bloated and > includes all sort of crap it doesn't need). I used a sample of 500 points > where 5 threads where instructed to request 100 pages each simultaneously. > All tests used the same hardware and a restarted clean apache instance. I'm > assuming a margin of error of ~5 requests per minute. > > No APC clocked in at a throughput of ~140 requests per minute. > Vanilla APC clocked in at a throughput of ~215 requests per minute. > Hacked APC clocked in at a throughput of ~215 requests per minute. > > So it appears my hacked APC works just as well as the vanilla one. However > it must still be considered experimental as it can break in all sorts of > ways. I didn't stress test against multiple virtual hosts for example. Also > note that ZTS was turned off. > > I'm attaching the patch. > > Quick breakdown of the differences between my patch and vanilla APC > (running peruser ofc): > > - Vanilla APC will allocate all shared memory when Apache starts (before > chroot etc) > + Hacked APC will delay shared memory initialization upon the first > request of the child thread (after chroot etc) > > + Hacked APC will reserve a memory region when Apache starts so there will > be no difference in address location. > + Hacked APC only supports exactly one memory segment. > + Hacked APC must be built with --enable-mmap > + Hacked APC is recommended to be built with --enable-apc-pthreadrwlocks > + Hacked APC adds a line "Hacks: Hacked for shared MMAP support" to > phpinfo() to indicate that it has been loaded. > > - Vanilla APC just uses mmap() as a way to allocate shared memory > + Hacked APC utilizes the file backed up feature of mmap, you must set > apc_mmap_file_mask to a place to store the APC cache/swap, and this file > will have the exact size of apc.shm_size. > > - Vanilla APC allocates shared memory once, it never loads previously > memory. > + Hacked APC will always load existing memory from the cache. Every time > you start Apache the start time is recorded. If the start time differs, the > cache will be zeroed, otherwise the cache will be loaded and the already > initialized cache structures in it be used. > > - Vanilla APC has no Environment separation. The cache is initialized once > and shared between all apache childs. > + Hacked APC will load the cache from file specified by > apc_mmap_file_mask. In addition, it will create it if it doesn't exist with > the permissions 0600 and with the uid/gid of the the child process. The > path is also affected by chroot. This is a double layer of security. > > As I said earlier this patch can not only be used for memory separation. > It can also theoretically be used to allow separate unrelated PHP instances > to share the same memory (for example in a PHP FCGI setup). Using PHP + > FCGI is already possible but not compatible with shared memory resulting in > a huge waste of memory. In practice every PHP child needs a 30 MB stack of > their own. So if you have 10 PHP childs on a server, that's 300 MB extra. > Note: In theory this patch should allow this. There could be other problems > I haven't thought of. > > Anyhow, FCGI FTL and Peruser FTW. :) > > Regards, > > Hannes Landeholm > > > > On 16 January 2011 07:36, Hannes Landeholm <[email protected]> wrote: > >> I have just written a hack/patch to APC that changes the behavior so that >> it mmaps the memory *after *request initialization, when the first >> request is handled by the thread. It will then make a mmap without closing >> the file handle so that it can be opened again by threads later on. I have >> tested it (however not stress tested it) and it appears to be working fine. >> There where tons of issues doing this which I wrote workarounds for. The >> biggest nuisance was the fact that the memory contains a bunch of addresses >> and so the mmap had to be mapped to the same memory address requiring >> preallocation and page boundary alignment etc. >> >> One thing I didn't solve however... it appears the file is automatically >> being flushed when MAP_ANONYMOUS is disabled.. so basically the cache is >> constantly written to disk, plus the fact that the file is the size of the >> maximum memory available by APC... 32M by default. This uses up valuable >> resources and there's no way to disable this... And it makes the patch >> kinda pointless as the whole point of APC is to make PHP use *less*memory. >> >> Hannes >> >> >> On 15 January 2011 19:13, Hannes Landeholm <[email protected]> wrote: >> >>> Unfortunately, no. Just checked, and the behavior of xcache is exactly >>> the same as with APC. >>> >>> I set xcache.mmap_path and looking at the strace output of apache I saw >>> that it opened this file in the main process once and long before the >>> peruser processor was chrooted. And since I didn't find the opened file I >>> can only assume that xcache deletes it's mmap file too. >>> >>> Hannes >>> >>> >>> On 14 January 2011 21:23, Leen Besselink <[email protected]> wrote: >>> >>>> On 01/14/2011 08:23 PM, Hannes Landeholm wrote: >>>> > Yes, you could use prefixing to prevent name collision in between >>>> > environments but that doesn't address the security issue. As long as >>>> > environments share memory with APC there are no way to prevent cache >>>> > stealing. The sharing memory part is the root of the problem. >>>> > >>>> >>>> Oops, forgot about that part. :-) >>>> >>>> > I'm looking at using the mmap option >>>> > < >>>> http://php.net/manual/en/apc.configuration.php#ini.apc.mmap-file-mask> >>>> now. >>>> > Ironically, the same feature which could theoretically allow Fast CGI >>>> > processes to share memory, >>>> > < >>>> http://stackoverflow.com/questions/598444/how-to-share-apc-cache-between-several-php-processes-when-running-under-fastcgi >>>> > >>>> > could also theoretically be used to prevent processes from sharing >>>> > memory. Mmap effectively maps a region memory to a physical file >>>> > allowing the file to be used as a handle. This would be perfect for >>>> > chroot'd environments as you could then inherit file system >>>> > permissions to shared memory permissions. The only problem is that APC >>>> > is not coded to take advantage of this. Although it provides an option >>>> > that uses mmap, it's just another way for it to share memory the >>>> > normal way by placing the opened handle in global memory. It also >>>> > immediately deletes the file - preventing it from being used as a >>>> > memory handle. In addition, APC allocates the shared memory in the >>>> > apache module initialization callback before any chrooting has >>>> > actually occoured, so to change APC to make use of mmap for real, >>>> > several changes would need to be made. >>>> > >>>> >>>> Not sure if you prefer APC over xcache, but I think xcache has mmap >>>> support too. Maybe it does it differently ? >>>> >>>> > Hannes >>>> > >>>> > On 14 January 2011 19:36, Leen Besselink <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > On Fri, Jan 14, 2011 at 05:02:52PM +0100, Hannes Landeholm wrote: >>>> > > Hello, >>>> > > >>>> > > FYI, I just tested this on my system and it appears that the >>>> > shared memory >>>> > > can indeed cross the boundary between different environments, >>>> so my >>>> > > assumption that this would be blocked somehow was wrong. As >>>> > Cronfy wrote >>>> > > below, the file caching path collision can be fixed by using a >>>> > directory >>>> > > structure where the first subfolder in the chroot is unique. >>>> > This is infact >>>> > > the case for our system where we have: >>>> > > >>>> > > ../user1/www.site1.com <http://www.site1.com> >>>> > > ../user1/www.example.com <http://www.example.com> >>>> > > ../user2/www.anothersite.com/ <http://www.anothersite.com/> >>>> > > .... >>>> > > >>>> > > An environment chroot corresponds to a user directory so all >>>> > paths are >>>> > > unique. Still, this is an unacceptable security issue as the >>>> > sites now share >>>> > > memory with op codes of potentially sensitive information >>>> > preventing threat >>>> > > containment on intrusion. For example, hacking www.site1.com >>>> > <http://www.site1.com> above and using >>>> > > apc_bin_dump it would be theoretically possible to get the mysql >>>> > passwords >>>> > > of www.anothersite.com <http://www.anothersite.com>. >>>> > > >>>> > > I'm looking into ways of preventing this... >>>> > > >>>> > > Hannes >>>> > > >>>> > >>>> > I think you would need to set an extra value (like php_admin_value >>>> > or environment >>>> > variable) from Apache with the real path or sitename or similair. >>>> > >>>> > And change the code of the opcode-cache-extension to recognise >>>> > that extra value as a >>>> > key/prefix/whatever to prevent them from having the same path >>>> > inside the opcode >>>> > cache. >>>> > >>>> > A simple change in the path in memory could be enough like: >>>> > >>>> > sitename:/path instead of the current /path >>>> > >>>> > It could be useful if the name of the the variable is configurable >>>> > like a >>>> > php.ini-setting. >>>> > >>>> > This might be a bit hackish, maybe there is a better way, I guess >>>> > it also >>>> > depends on the code of the opcode-cache-extension. >>>> > >>>> > > >>>> > > On 11 January 2011 19:55, Cronfy <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > > >>>> > > > Hello, >>>> > > > >>>> > > > >>>> > > > Also say you have two users who have installed WordPress, you >>>> > end up with: >>>> > > >> >>>> > > >> /home/user1/public_html/wp-config.php >>>> > > >> /home/user2/public_html/wp-config.php >>>> > > >> but Eaccelerator (and xcache, APC) only see this: >>>> > > >> /public_html/wp-config.php >>>> > > >> as the file path. Do we know if it is serving the correct >>>> > file when a >>>> > > >> request comes in from user1 or user2 or whether it thinks >>>> > they are the same >>>> > > >> file because of the chroot path? I'm just wondering if anyone >>>> > has come >>>> > > >> across this before with peruser + chroot and knows whether >>>> > these tools are >>>> > > >> working correctly despite the chroot? Or whether there is a >>>> > way to feed the >>>> > > >> full path to these opcode caching tools while using peruser + >>>> > chroot. >>>> > > >> >>>> > > > >>>> > > > Not sure if it will help, but you may try to add site name to >>>> > the path to >>>> > > > make it different: >>>> > > > >>>> > > > /home/user1/wordpress-me.com/public_html/wp-config.php >>>> > <http://wordpress-me.com/public_html/wp-config.php> >>>> > > > /home/user2/wordpress-yourself.com/public_html/wp-config.php >>>> > <http://wordpress-yourself.com/public_html/wp-config.php> >>>> > > > >>>> > > > Anyway, I agree with Hannes: >>>> > > > >>>> > > > >>>> > > > In fact, it would be a security risk of one environment could >>>> > access >>>> > > >> >>>> > > > the cached PHP files of another environment >>>> > > > >>>> > > > And access to shared memory between different users should be >>>> > thoroughly >>>> > > > tested. >>>> > > > >>>> > > > >>>> > > > -- >>>> > > > // cronfy >>>> > > > >>>> > > > _______________________________________________ >>>> > > > Peruser mailing list >>>> > > > [email protected] <mailto:[email protected]> >>>> > > > http://www.telana.com/mailman/listinfo/peruser >>>> > > > >>>> > >>>> > > _______________________________________________ >>>> > > Peruser mailing list >>>> > > [email protected] <mailto:[email protected]> >>>> > > http://www.telana.com/mailman/listinfo/peruser >>>> > >>>> > _______________________________________________ >>>> > Peruser mailing list >>>> > [email protected] <mailto:[email protected]> >>>> > http://www.telana.com/mailman/listinfo/peruser >>>> > >>>> > >>>> >>>> _______________________________________________ >>>> Peruser mailing list >>>> [email protected] >>>> http://www.telana.com/mailman/listinfo/peruser >>>> >>> >>> >> > > _______________________________________________ > Peruser mailing list > [email protected] > http://www.telana.com/mailman/listinfo/peruser > >
_______________________________________________ Peruser mailing list [email protected] http://www.telana.com/mailman/listinfo/peruser
