Re: Install on s390
Steven A. Adams writes: > Has anyone out there installed mod_perl on Linux for s390 (SuSE, RH or > Debian)? I'd be very interested in any success stories or gotchas before > I start on that journey. SLES8 comes with mod_perl-1.27-28.s390.rpm. I've just installed it on my system but I haven't tried it out yet. Regards, --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Linux Technical Consultant IBM EMEA Enterprise Server Group... ...from home, speaking only for myself
Re: Very[OT]:Technical query re: scratchpad lookups for my() vars
Paul writes:
>
> --- Brian Ingerson <[EMAIL PROTECTED]> wrote:
> > Garrett Goebel wrote:
> > >
> > > From: Paul [mailto:[EMAIL PROTECTED]]
> > > >
> > > > Anybody know offhand *why* my() lexicals are supposedly faster?
>
>
>
> > Yes this is OT, but I'll contribute to the problem as well...
> >
> > My coworker Gisle Aas (maybe you've heard of him ;) says that globals
> > and lexicals have identical speed because Perl optimizes out the
> > symbol-table lookup.
> >
> > Trust Gisle.
>
> lol -- now *there's* an answer.
> So for my details I should go to the parse tree docs, and the code, I'm
> thinking.
>
> BTW -- with many thanks to everyone -- my question was "why are they
> faster", but the reason was never the speed -- it was to understand the
> way Perl stores and *accesses* lexicals.
>
> Any input? =o)
If you have a reasonably recent Perl, you can do the following:
% perl -MO=Terse,exec -e '$f = 123'
OP (0x8180688) enter
COP (0x8180628) nextstate
SVOP (0x8175298) const IV (0x80f8770) 123
SVOP (0x817b488) gvsv GV (0x81017b0) *f
BINOP (0x8180600) sassign
LISTOP (0x8180660) leave
% perl -MO=Terse,exec -e 'my $f = 123'
OP (0x81805d0) enter
COP (0x8180598) nextstate
SVOP (0x8104b88) const IV (0x8104c9c) 123
OP (0x817b490) padsv [1]
BINOP (0x81761f0) sassign
LISTOP (0x81752a0) leave
As you can see from the output, for a non-lexical $f, Perl uses an
opcode "gvsv GV *f". The gvsv instruction gets a pointer to the
entire glob (*f) from which it dereferences the SV (scalar) part and
pushes it on the stack. See pp_hot.c:
PP(pp_gvsv)
{
djSP;
EXTEND(SP,1);
if (PL_op->op_private & OPpLVAL_INTRO)
PUSHs(save_scalar(cGVOP_gv));
else
PUSHs(GvSV(cGVOP_gv));
RETURN;
}
For the lexical, Perl has already determined at compile time that
$f is in pad slot number 1 (think stack or register allocation).
padsv is:
PP(pp_padsv)
{
djSP; dTARGET;
XPUSHs(TARG);
...
If you navigate all the macros, you'll find that takes curpad
(a pointer to an array of SV pointers: the current "stack frame"
where "stack" is in the sense of a traditional compiler, not the
(main) Perl stack) and pushes curpad[1] (remember $f was allocated
slot 1 at compile time) onto the (main Perl) stack.
--Malcolm
--
Malcolm Beattie <[EMAIL PROTECTED]>
Unix Systems Programmer
Oxford University Computing Services
Re: [OT] Re: Repost: Anyone using "virtual server" for mod_perl hosts?
Tim Bunce writes: > On Tue, Feb 13, 2001 at 11:28:20AM +0000, Malcolm Beattie wrote: > > > > you'll see that IBM reckons you can get down to $500 per server > > (disk not included) by putting 2500 instances on a brand new fancy > > $1.2 million z900. > > Assuming all the virtual linux servers were fully loaded with tasks > (say apache+mod_perl as an example)... What kind of tradition Intel > platform performance would each virtual linux instance be equivalent to? > > e.g., CPU: ~600MHz PIII? Heck, if IBM would just get a test system on our floor like they've been promising for months, I'd be able to find out. It depends on how the load spreads between the servers. It's much the same problem as determining how many users you can put on a large multi-user system and how much real disk space you need. Say you have 1 users on a machine: it may be that only 500 or 1000 are active at any one time. It depends on the environment. Similarly, you can give people large quotas in many environments (POP servers, some web hosting, some home filestore) because, on average, people only use a small fraction. The problems are similar (but not the same) for running multiple virtual servers on one system. They're similar because you have overallocation and "competing" for shared resources with potential bursty and asymmetric behaviour that the system has to smooth out. They're different because the same rule of thumb numbers don't apply (or at least they apply but only "one level up"). If you have say 100 systems with 1000 users/clients/whatever for each then you'd get the same "hit rate" (in some abstract sense) from 1 user of each virtual server doing 1 hit as from 100 users hitting only one server. In the former case, you've got the system overhead of using memory and scheduling for 100 different kernels; in the second case, 99 of the kernels are sitting idle, paged out, unscheduled and barely affecting the machine at all. All I can do is basic sums on the hardware figures (available in my slides) such as one G6/z900 CPU having roughly 16 times the cache and memory bandwidth of an Intel CPU and needing zero CPU for most of the I/O path which is all offloaded onto SAP/channels/OSA. Until IBM get me that test system, my best guesstimate/hope is that if we were to put 150 virtual servers on a 3-way G5/G6 system with 16 channels and 1 in 10 active at any instant then, if those systems that are active all happen to need maximum CPU at the same time, each is getting about 120MHz-worth of CPU and the equivalent of a fast-wide SCSI bandwidth to disk except that there's almost zero CPU cost for I/O to either disk or network. In general, CPU use (and I/O and memory) will be smoother and scheduled across the entire system so that bursty behaviour for CPU, I/O and memory will all be smoothed out. That's the theory and, at the moment, I've convinced myself that it could theoretically hold in practice too but I've no first-hand evidence (other than other big sites like Telia, the big German ISP and so on going Linux/390). > And what about network i/o? Would the z900 network i/o be a bottleneck > if all the virtual servers were blasting away? Almost certainly not. You can put 24 OSA-Express Gigabit ports (12 cards) into a z900, each taking one of your maximum of 256 channels. See my slides. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: [OT] Re: Repost: Anyone using "virtual server" for mod_perl hosts?
G.W. Haywood writes: > On Mon, 12 Feb 2001, Malcolm Beattie wrote: > > > > you can run *thousands* of separate Linux images on a S/390 > > How much, to the nearest order of magnitude, does a S/390 cost? How long is a piece of string? An S/390 can be anything from about $100 on ebay for an extremely old one which would cost more in power, space and cooling and do less in performance than any reasonable person would want unless they're *really* hooked on history and blinkenlights. At the top end you can pay $5 million or more for a top of the range z900 fully kitted out. More usefully, I'll say that I'm after a system which costs around 1000 GBP per virtual server (that would be $1000 at computing prices of $1 = 1GBP). The question is how large a system I have to get to bring down the per-virtual-server price that far. I'm hoping that 150-200 would do the trick but I'm (a) hoping to pay extremely low academic prices and (b) probably being over-optimistic. If you look at http://www-1.ibm.com/servers/eserver/zseries/linuxconfig/ you'll see that IBM reckons you can get down to $500 per server (disk not included) by putting 2500 instances on a brand new fancy $1.2 million z900. On one hand, I'd guess you may need to pay for some upgrading if they aren't very lightly used servers but on the other hand, no one ever pays list price (I'm reliably informed). On the gripping hand, it's very difficult getting hold of pricing information at all on these things (as mentioned in my last slide, I think) which is one of the big problems. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
[OT] Re: Repost: Anyone using "virtual server" for mod_perl hosts?
Malcolm Beattie writes: > 50 boxes: no problem. 200 boxes: 5 racks or 1U, getting messy. > 1000 boxes: admin nightmare. Plus you don't get much too many > built-in reliability features with a 1U box. Now consider that you can > run *thousands* of separate Linux images on a S/390 box which consists > of just one or two frames (i.e. the size of one or two racks). It'll The slides from my talk "IBM Mainframe Hardware from a Linux Hacker's Viewpoint" are now available at http://users.ox.ac.uk/~mbeattie/newcastle2001/index.html There may well be a few errors in the facts and figures (mainly because most of the reason for the talk was gathering together large numbers of facts and figures from disparate sources). However, the attendees seemed to like it, despite the fact that I had to rush through part of it due to lack of time. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: Repost: Anyone using "virtual server" for mod_perl hosts?
Robert Landrum writes: > The S390 appearently runs some type of software that allows you to > set limits on your partitions, so no matter what, you always have > some percentage of the CPU at your disposal. It's called VM and it's a lot more flexible than that. > This is not the case with the Sun 1. With that machine, you must > explicity set which processors you want partitioned to your virtual > box. With a 16 processor Sun 1, you could set up four, four > processor Sun virtual machines, all sharing the same hard drives and > external adapters (NIC cards and serial ports). Exactly: with E10k (and IBM NUMAQ) you are limited to splitting things up at a "quad" (4 processor) boundary. > Large systems like this are dying, I think you misspelled "beginning to be even more popular" :-) >as they generally require much > more knowledge than simply establishing a server farm of the same > capabilities. It's much easier to higher people to set up 50 boxes > (linux, NT, BSD, Solaris) than it is to find people that can > configure an S390 or Sun 1. 50 boxes: no problem. 200 boxes: 5 racks or 1U, getting messy. 1000 boxes: admin nightmare. Plus you don't get much too many built-in reliability features with a 1U box. Now consider that you can run *thousands* of separate Linux images on a S/390 box which consists of just one or two frames (i.e. the size of one or two racks). It'll need hooking up to a rack or few of disks too. Far less floor space, far less power, far more reliable, far fewer cables and mess, very easy to create a new virtual machine (minutes), pretty much all maintenance and upgrading is concurrent and you can admin the whole lot from one place. Now isn't that worth having to learn a bit about how to admin a VM system? Especially since you wouldn't want some random cheap admin looking after that many boxes and customers anyway. There was a test/benchmark done where more and more Linux virtual machines were added to a system, each running Apache or INN or being a client image pulling data from the server images. The experiment was to see how many images the system could sustain. At 3 images the server images were still providing subsecond response time. The system finally started thrashing at 41400 concurrent virtual machines. I can dig out refs if people want (it was David Boyes who did it, if I recall). In practical terms, you can put thousands of virtual machines on one system: there are big advantages to sharing one large machine. The most recent big name to go Linux/390 is Telia, the Swedish Telco. See http://www.silicon.com/a41413 for an article about it (yeah, I get a quote :-). --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: [SOT] mod_perl on IBM s390
Sean D. Cook writes: > I know that rescently IBM has been successfull in running thousands of > instances of linux on the H30-H70 series machines. I am wondering if > anyone is using one of these beasts to run mod_perl in a production > environment and what kind of millage you are getting. I'm aware of someone who's trying to get mod_perl and WING (my web mail thing) going on Linux/390 but is having problems with httpd segfaulting. I've let him know that mod_perl is very sensitive to version skew between Apache, mod_perl and perl and am waiting to see if compiling from scratch fixes things. We may be getting a S/390 for testing here within the next few months in which case I'll be doing a lot of Apache/mod_perl testing and feeding the results back. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))
Paul J. Lucas writes: > On Fri, 28 Jul 2000, Malcolm Beattie wrote: > > > Assuming the kernel only keeps track of the last fault position in the file, > > it won't recognise that it's being read linearly (twice :-) and may well not > > do the async read ahead and drop behind in page cache that it would do > > otherwise. Once again, you'll lose performance with mmap. > > And reading in the file twice into two separate I/O buffers > is better? I've forgotten exactly what the original situation was so I'm not sure whether it's better or not. With mmap you also have the potential disadvantage that you're taking a page fault into the kernel for every single page (say 4k or 8k) whereas with read(), a single system call will fill up a much larger buffer with data. It's also possible that the kernel can play tricks to get you the data from a read better (on some architectures it could avoid copying through the cache or it may be able to get the data transferred from the source directly into your buffer without passing through the CPU or suchlike). Basically, mmap may well be an advantage from the ease or elegance-of-coding point of view and in some cases it may win out over read() for performance/scalability or whatever but it's not a foregone conclusion and there are kernel/architecture-specific difference which may swing the balance one way or the other. Now we're getting offtopic though. I really just wanted to make the points that saying "mmap is always better than read" is not true and that there are things going on behind the scenes. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> I am looking for a Linux (and Unix Systems Programmer maybe Apache/mod_perl) job/contract Oxford University Computing Services http://users.ox.ac.uk/~mbeattie/cv.html
Re: [OT]: mmap (Was: Templating System (preview Embperl 2.0))
Paul J. Lucas writes: > > This is the quote from "Philip and Alex..." that I was talking about > > earlier. I don't know if its relevant or not: > > > > They replied "NaviServer uses memory-mapped file I/O on single-processor > > machines but automatically configures itself to use operating system read on > > multi-CPU machines so as to reduce shared memory contention." > > mmap(2) is *supposed* to share memory when read-only which is > what's beign done for HTML file reads, so I don't understand > what "contention" there could possibly be. That's where some of the subtlety comes in. The first that springs to mind is that with multiple CPUs of an architecture with a virtually indexed cache, if the mmap maps the file at different virtual addresses (as it very well may without slow global communication between the reading processes) then the data will end up being duplicated in processor cache. That slows things down. Another point is that it can be harder for the kernel to notice when a file is being read linearly when all it sees is page faults through an address space. That means it may not do read-ahead on that file as well. Most decent Unices will probably try to keep track these days (certainly Linux does page-ahead and in fact page-around to try to predict when to read-ahead or drop-behind or whatever). On the other hand, if you have two concurrent processes doing the reading then you'll get page faults at, for example, byte 8k, 16k, 8k(second process starting), 24k, 16k and so on. Assuming the kernel only keeps track of the last fault position in the file, it won't recognise that it's being read linearly (twice :-) and may well not do the async read ahead and drop behind in page cache that it would do otherwise. Once again, you'll lose performance with mmap. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: [input] mod_perl and ISPs in the real world
Shane Nay writes: > Well, as most of us know mod_perl doesn't really lend itself well to the > "hosted" world because of the way it operates. However there is I think a > conceivable solution. This might sound a little crazy (or a lot), but I've > been messing around with vmware, and it's really cool/stable. One thought is > instead of hosting a "site" or a "machine" host a "virtual machine". It might > be more economical to use one of those IBM s/who knows how many machines that > can run many many copies of linux internally for large hosts, but for a smaller > hosting service they might want to give VMWare a shot. I have no personal > connection with VMware, but it is a really cool product and I have been testing > a lot of different things with it and it's VERY impressive. I'm currently writing SILK (Simultaneous Instances of the Linux Kernel) which is a combination of a tweaked Linux kernel hypervisor and a tweaked Linux guest kernel. You can run multiple guest kernels under the hypervisor and do the sort of basic stuff that VM/ESA (the IBM S/390 product) and VMWare, with potential for fancier stuff if things go nicely. I mentioned it on the linux-kernel and linux-390 mailing lists so it's probably in archives somewhere. In a few months IBM will be releasing a "run Linux kernels only" cut down version of VM for free (or nearly so) for S/390 and IBM's targets definitely include ISPs hosting lots of "virtual machines" on a nice reliable, easily-adminned single large system. (Most of you have probably seen various recent Linux/390 articles and the headline figure of over 4 concurrent Linux instances on one S/390, each running an Apache daemon or similar). Cool stuff. --Malcolm -- Malcolm Beattie <[EMAIL PROTECTED]> Unix Systems Programmer Oxford University Computing Services
Re: LARGE PERL footprint
Matt Sergeant writes:
> On Fri, 19 May 2000, David Larkin wrote:
>
> > I require a large array of ints in a real application, just stripped
> > problem down to bear bones for demo.
>
> Is your array sparse by any chance? If not your modperl daemon is going to
> get _much_ larger after you populate that array. If it's sparse, consider
> using a hash - that will give you a slight speed penalty at the benefit of
> not consuming quite so much ram. Other things to consider:
>
> Re-write this critical bit of code in XS (not as hard as it sounds).
> Use a database.
> Use the filesystem.
Simply store them in native form in a big long string, just like C
would do. Pre-extend the string first, if you know how big you want it:
$intsize = 4; # alter if sizeof(int) != 4
$intarray = "\0" x (3 * $intsize);
Update index $i with
substr($intarray, $i * $intsize, $intsize) = pack("I", $newval);
which assumes an unsigned int, or use "i" for signed.
To retrieve index $i use
$val = unpack("I", substr($intarray, $i * $intsize, $intsize));
You can even retrive a bunch of them faster with
@ten_to_thirty = unpack("I20", substr(...));
or set a bunch with
substr(..., $intsize * 10) = pack("I20", @ten_to_thirty);
For that last bit of extra performance, do
sub INTSIZE () { 4 }; # alter if sizeof(int) != 4
--Malcolm
--
Malcolm Beattie <[EMAIL PROTECTED]>
Unix Systems Programmer
Oxford University Computing Services
