I talked with one of the hardware guru's about what I am seeing. On a G6, the linesize (amount of data fetched from memory at a time) is 256 bytes, the L1 cache is 256 KB (1024 lines), and the L2 cache is 4 MB/6 CP (8 MB total). The zSeries has the same linesize of 256 bytes, an L1-I (instructions) of 256 KB, an L1-D (data) of 256 KB and a L2 cache of 8MB/8 CP or a total of 16MB L2 cache.
If you working set size is then greater than 8 MB on a G6 or 16 MB on a zSeries, then you will see the effect of L2 cache being flushed. Since the linesize is 256 bytes, then it would only take 32,768 fetches to fill the cache if your data elements were more than 256 bytes apart, i.e., an array of 32768 256 structures, or one element each of 256 arrays of 8,192 words. Neither case is pathological, when you think about it. Its a rather small table that has 32,768 entries, allowing 256 bytes per entry. In engineering apps, doing calculations over 256 characteristics of 32,768 seperate points is rather modest. In either case, a single traverse of the data will fill the L2 cache. A working set size fo 100MB or 500MB is relatively easy to generate if your data aggregates are large. >From the OS standpoint, how big can your queues and related data be before they start seeing this effect? One OS structure that I can think of that might see this effect is the inode tables for large filesystems. Are they traversed sequentially or randomly? How much of their activity is update (last date used, file expansion, adding or deleing files, etc). Another issue might be the large buffer caches that Linux uses (i.e., all of memory that is not used). High buffer activity would certainly flush L2 cache if you allow too large a buffer cache. However, its your CP cycles. Manage it as you will. ===== Jim Sibley Implementor of Linux on zSeries in the beautiful Silicon Valley "Computer are useless.They can only give answers." Pablo Picasso __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
