I've included some statistics and other info from the system w/ swappiness=0
Yes, we ran for some time before OOM. We have started one at a time, but we did not do any deep investigation (it was our proof of concept). We have set swappiness to 0, to see if that changes things. Also opened a ticket with IBM WAS Support. Going to apply the SP on our proof of concept box, test, and upgrade the remaining servers. I'll let the WAS admin look over those tweaks. I've done some tweaks (mainly TCP) i read from an older best practices guide, but at the linux OS level, im wondering if any system tunables would help aside from those I have set: echo "20000" > /proc/sys/net/core/netdev_max_backlog echo "20000" > /proc/sys/net/core/somaxconn echo "30" > /proc/sys/net/ipv4/tcp_fin_timeout echo "20" > /proc/sys/net/ipv4/tcp_syn_retries echo "20" > /proc/sys/net/ipv4/tcp_synack_retries echo "3" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce --- top --- top - 18:55:23 up 3:59, 5 users, load average: 0.53, 0.63, 0.79 Tasks: 135 total, 1 running, 134 sleeping, 0 stopped, 0 zombie Cpu(s): 27.9%us, 3.3%sy, 0.0%ni, 63.1%id, 2.2%wa, 0.3%hi, 0.7%si, 2.5%st Mem: 10268344k total, 10201956k used, 66388k free, 16684k buffers Swap: 35764956k total, 5074508k used, 30690448k free, 182532k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4148 wasadmin 20 0 1500m 602m 52m S 23 6.0 17:32.73 java 4629 wasadmin 20 0 1494m 595m 52m S 12 5.9 17:22.78 java 6075 wasadmin 20 0 554m 254m 56m S 10 2.5 3:06.60 java 5883 wasadmin 20 0 681m 245m 56m S 6 2.4 2:48.05 java 10371 wasadmin 20 0 551m 288m 56m S 2 2.9 3:00.42 java 11179 wasadmin 20 0 533m 235m 56m S 2 2.4 3:01.71 java 9368 wasadmin 20 0 537m 264m 56m S 1 2.6 2:32.23 java 13317 wasadmin 20 0 546m 217m 56m S 1 2.2 1:50.19 java 6263 wasadmin 20 0 642m 250m 56m S 1 2.5 3:04.74 java 10813 wasadmin 20 0 585m 319m 56m S 1 3.2 2:53.79 java 11946 wasadmin 20 0 582m 279m 56m S 1 2.8 2:46.57 java 12351 wasadmin 20 0 551m 288m 56m S 1 2.9 2:41.00 java 5388 wasadmin 20 0 576m 297m 56m S 0 3.0 2:51.82 java 5556 wasadmin 20 0 615m 293m 56m S 0 2.9 2:55.52 java 5709 wasadmin 20 0 573m 263m 56m S 0 2.6 2:48.57 java 6475 wasadmin 20 0 578m 276m 56m S 0 2.8 2:27.73 java 6981 wasadmin 20 0 582m 275m 56m S 0 2.7 2:24.28 java --- sysinfo follows --- Manufacturer: IBM Type: 2094 Model: 708 S18 Sequence Code: 000000000006CDEE Plant: 02 Model Capacity: 708 00000000 CPUs Total: 20 CPUs Configured: 8 CPUs Standby: 0 CPUs Reserved: 12 Capability: 1456 1920 Adjustment 02-way: 245 249 Adjustment 03-way: 238 241 Adjustment 04-way: 232 235 Adjustment 05-way: 226 229 Adjustment 06-way: 221 222 Adjustment 07-way: 216 216 Adjustment 08-way: 211 210 Adjustment 09-way: 206 205 Adjustment 10-way: 202 202 Adjustment 11-way: 198 198 Adjustment 12-way: 195 195 Adjustment 13-way: 191 193 Adjustment 14-way: 188 190 Adjustment 15-way: 185 188 Adjustment 16-way: 182 185 Adjustment 17-way: 180 183 Adjustment 18-way: 178 181 Adjustment 19-way: 176 179 Adjustment 20-way: 174 178 Secondary Capability: 1456 LPAR Number: 14 LPAR Characteristics: Shared LPAR Name: VM06 LPAR Adjustment: 333 LPAR CPUs Total: 2 LPAR CPUs Configured: 2 LPAR CPUs Standby: 0 LPAR CPUs Reserved: 0 LPAR CPUs Dedicated: 0 LPAR CPUs Shared: 2 VM00 Name: LINXD002 VM00 Control Program: z/VM 5.4.0 VM00 Adjustment: 1000 VM00 CPUs Total: 2 VM00 CPUs Configured: 2 VM00 CPUs Standby: 0 VM00 CPUs Reserved: 0 --- vmstat follows --- nr_free_pages 14436 nr_inactive 615152 nr_active 1892676 nr_anon_pages 1849250 nr_mapped 25365 nr_file_pages 1159871 nr_dirty 61 nr_writeback 0 nr_slab_reclaimable 5808 nr_slab_unreclaimable 12216 nr_page_table_pages 7214 nr_unstable 0 nr_bounce 0 nr_vmscan_write 1220483 nr_writeback_temp 0 pgpgin 2544524 pgpgout 5604952 pswpin 46306 pswpout 1214579 pgalloc_dma 3642085 pgalloc_normal 6246202 pgalloc_movable 0 pgfree 9903077 pgactivate 1531012 pgdeactivate 2168881 pgfault 24064495 pgmajfault 7477 pgrefill_dma 24872660 pgrefill_normal 99930750 pgrefill_movable 0 pgsteal_dma 173635 pgsteal_normal 225696 pgsteal_movable 0 pgscan_kswapd_dma 282422 pgscan_kswapd_normal 588898 pgscan_kswapd_movable 0 pgscan_direct_dma 980259 pgscan_direct_normal 1877844 pgscan_direct_movable 0 pginodesteal 12 slabs_scanned 288256 kswapd_steal 251490 kswapd_inodesteal 1 pageoutrun 2908 allocstall 1763 pgrotated 1143361 htlb_buddy_alloc_success 0 htlb_buddy_alloc_fail 0 --- --- slabinfo follows --- slabinfo - version: 2.1 # name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail> fuse_request 0 0 608 6 1 : tunables 54 27 8 : slabdata 0 0 0 fuse_inode 0 0 768 5 1 : tunables 54 27 8 : slabdata 0 0 0 dm_mpath_io 256 404 16 202 1 : tunables 120 60 8 : slabdata 2 2 0 kcopyd_job 0 0 360 11 1 : tunables 54 27 8 : slabdata 0 0 0 dm_uevent 0 0 2608 3 2 : tunables 24 12 8 : slabdata 0 0 0 dm_clone_bio_info 256 404 16 202 1 : tunables 120 60 8 : slabdata 2 2 0 dm_rq_target_io 256 261 408 9 1 : tunables 54 27 8 : slabdata 29 29 0 dm_target_io 1876 4608 24 144 1 : tunables 120 60 8 : slabdata 32 32 0 dm_io 1876 3360 32 112 1 : tunables 120 60 8 : slabdata 30 30 0 fib6_nodes 5 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0 ip6_dst_cache 4 7 512 7 1 : tunables 54 27 8 : slabdata 1 1 0 ndisc_cache 1 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0 RAWv6 67 68 1024 4 1 : tunables 54 27 8 : slabdata 17 17 0 UDPLITEv6 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0 UDPv6 60 60 1024 4 1 : tunables 54 27 8 : slabdata 15 15 0 tw_sock_TCPv6 165 165 256 15 1 : tunables 120 60 8 : slabdata 11 11 0 request_sock_TCPv6 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0 TCPv6 1276 1276 1792 2 1 : tunables 24 12 8 : slabdata 638 638 0 scsi_sense_cache 19 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0 scsi_cmd_cache 22 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0 zfcp_gpn 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0 zfcp_gid 2 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0 zfcp_sr 64 64 4096 1 1 : tunables 24 12 8 : slabdata 64 64 0 zfcp_fsf 17 24 2048 2 1 : tunables 24 12 8 : slabdata 12 12 0 qeth_hdr 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0 qdio_q 9 10 1536 5 2 : tunables 24 12 8 : slabdata 2 2 0 scsi_tgt_cmd 0 0 80 48 1 : tunables 120 60 8 : slabdata 0 0 0 sgpool-128 2 2 2048 2 1 : tunables 24 12 8 : slabdata 1 1 0 sgpool-64 2 4 1024 4 1 : tunables 54 27 8 : slabdata 1 1 0 sgpool-32 2 8 512 8 1 : tunables 54 27 8 : slabdata 1 1 0 sgpool-16 3 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0 sgpool-8 17 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0 scsi_data_buffer 0 0 24 144 1 : tunables 120 60 8 : slabdata 0 0 0 scsi_io_context 0 0 112 34 1 : tunables 120 60 8 : slabdata 0 0 0 ext3_inode_cache 4298 4310 784 5 1 : tunables 54 27 8 : slabdata 862 862 0 ext3_xattr 0 0 88 44 1 : tunables 120 60 8 : slabdata 0 0 0 journal_handle 12 144 24 144 1 : tunables 120 60 8 : slabdata 1 1 0 journal_head 144 320 96 40 1 : tunables 120 60 8 : slabdata 8 8 0 revoke_table 14 202 16 202 1 : tunables 120 60 8 : slabdata 1 1 0 revoke_record 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0 flow_cache 0 0 96 40 1 : tunables 120 60 8 : slabdata 0 0 0 cfq_io_context 0 0 168 23 1 : tunables 120 60 8 : slabdata 0 0 0 cfq_queue 0 0 136 28 1 : tunables 120 60 8 : slabdata 0 0 0 bsg_cmd 0 0 312 12 1 : tunables 54 27 8 : slabdata 0 0 0 mqueue_inode_cache 1 4 1024 4 1 : tunables 54 27 8 : slabdata 1 1 0 isofs_inode_cache 0 0 616 6 1 : tunables 54 27 8 : slabdata 0 0 0 minix_inode_cache 0 0 632 6 1 : tunables 54 27 8 : slabdata 0 0 0 hugetlbfs_inode_cache 1 7 568 7 1 : tunables 54 27 8 : slabdata 1 1 0 dnotify_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0 dquot 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0 inotify_event_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0 inotify_watch_cache 78 159 72 53 1 : tunables 120 60 8 : slabdata 3 3 0 kioctx 0 0 512 7 1 : tunables 54 27 8 : slabdata 0 0 0 kiocb 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0 fasync_cache 0 0 24 144 1 : tunables 120 60 8 : slabdata 0 0 0 shmem_inode_cache 548 555 776 5 1 : tunables 54 27 8 : slabdata 111 111 0 pid_namespace 0 0 2112 3 2 : tunables 24 12 8 : slabdata 0 0 0 nsproxy 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0 posix_timers_cache 0 0 192 20 1 : tunables 120 60 8 : slabdata 0 0 0 uid_cache 7 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0 UNIX 188 205 768 5 1 : tunables 54 27 8 : slabdata 41 41 0 ip_mrt_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0 UDP-Lite 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0 tcp_bind_bucket 941 1003 64 59 1 : tunables 120 60 8 : slabdata 17 17 0 inet_peer_cache 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0 secpath_cache 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0 xfrm_dst_cache 0 0 512 7 1 : tunables 54 27 8 : slabdata 0 0 0 ip_fib_alias 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0 ip_fib_hash 12 106 72 53 1 : tunables 120 60 8 : slabdata 2 2 0 ip_dst_cache 38 70 512 7 1 : tunables 54 27 8 : slabdata 10 10 0 arp_cache 5 30 256 15 1 : tunables 120 60 8 : slabdata 2 2 0 RAW 65 65 768 5 1 : tunables 54 27 8 : slabdata 13 13 0 UDP 32 32 1024 4 1 : tunables 54 27 8 : slabdata 8 8 0 tw_sock_TCP 35 75 256 15 1 : tunables 120 60 8 : slabdata 5 5 0 request_sock_TCP 23 30 128 30 1 : tunables 120 60 8 : slabdata 1 1 0 TCP 100 100 1792 2 1 : tunables 24 12 8 : slabdata 50 50 0 eventpoll_pwq 969 1007 72 53 1 : tunables 120 60 8 : slabdata 19 19 0 eventpoll_epi 969 990 128 30 1 : tunables 120 60 8 : slabdata 33 33 0 blkdev_integrity 0 0 112 34 1 : tunables 120 60 8 : slabdata 0 0 0 blkdev_queue 23 24 2264 3 2 : tunables 24 12 8 : slabdata 8 8 0 blkdev_requests 89 100 368 10 1 : tunables 54 27 8 : slabdata 10 10 0 blkdev_ioc 58 159 72 53 1 : tunables 120 60 8 : slabdata 3 3 0 biovec-256 370 370 4096 1 1 : tunables 24 12 8 : slabdata 370 370 0 biovec-128 370 370 2048 2 1 : tunables 24 12 8 : slabdata 185 185 0 biovec-64 375 380 1024 4 1 : tunables 54 27 8 : slabdata 95 95 0 biovec-16 372 375 256 15 1 : tunables 120 60 8 : slabdata 25 25 0 biovec-4 378 413 64 59 1 : tunables 120 60 8 : slabdata 7 7 0 biovec-1 495 1010 16 202 1 : tunables 120 60 8 : slabdata 5 5 0 bio_integrity_payload 370 390 128 30 1 : tunables 120 60 8 : slabdata 13 13 0 bio 481 810 128 30 1 : tunables 120 60 8 : slabdata 27 27 0 utrace_attached_engine 0 0 48 77 1 : tunables 120 60 8 : slabdata 0 0 0 utrace 0 0 72 53 1 : tunables 120 60 8 : slabdata 0 0 0 sock_inode_cache 1792 1805 768 5 1 : tunables 54 27 8 : slabdata 361 361 0 skbuff_fclone_cache 90 112 512 7 1 : tunables 54 27 8 : slabdata 16 16 0 skbuff_head_cache 150 180 256 15 1 : tunables 120 60 8 : slabdata 12 12 60 file_lock_cache 45 110 176 22 1 : tunables 120 60 8 : slabdata 5 5 0 task_delay_info 2554 2652 112 34 1 : tunables 120 60 8 : slabdata 78 78 0 taskstats 33 36 328 12 1 : tunables 54 27 8 : slabdata 3 3 0 page_cgroup 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0 proc_inode_cache 487 498 616 6 1 : tunables 54 27 8 : slabdata 83 83 0 sigqueue 13 48 160 24 1 : tunables 120 60 8 : slabdata 1 2 0 radix_tree_node 22137 22148 552 7 1 : tunables 54 27 8 : slabdata 3164 3164 0 bdev_cache 24 30 768 5 1 : tunables 54 27 8 : slabdata 6 6 0 sysfs_dir_cache 4260 4346 72 53 1 : tunables 120 60 8 : slabdata 82 82 0 mnt_cache 28 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0 inode_cache 3373 3395 568 7 1 : tunables 54 27 8 : slabdata 485 485 0 dentry 10710 13794 208 19 1 : tunables 120 60 8 : slabdata 726 726 0 filp 11223 12300 256 15 1 : tunables 120 60 8 : slabdata 820 820 228 names_cache 25 25 4096 1 1 : tunables 24 12 8 : slabdata 25 25 0 key_jar 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0 buffer_head 3964 3996 104 37 1 : tunables 120 60 8 : slabdata 108 108 0 mm_struct 96 96 1024 4 1 : tunables 54 27 8 : slabdata 24 24 0 vm_area_struct 15464 15709 168 23 1 : tunables 120 60 8 : slabdata 683 683 48 fs_cache 143 236 64 59 1 : tunables 120 60 8 : slabdata 4 4 0 files_cache 142 160 1024 4 1 : tunables 54 27 8 : slabdata 39 40 0 signal_cache 198 216 1024 4 1 : tunables 54 27 8 : slabdata 54 54 0 sighand_cache 199 210 2304 3 2 : tunables 24 12 8 : slabdata 70 70 0 task_struct 2527 2527 5864 1 2 : tunables 8 4 0 : slabdata 2527 2527 0 anon_vma 4223 4320 24 144 1 : tunables 120 60 8 : slabdata 30 30 15 pid 2498 2580 128 30 1 : tunables 120 60 8 : slabdata 86 86 0 idr_layer_cache 217 217 544 7 1 : tunables 54 27 8 : slabdata 31 31 0 size-1048576(DMA) 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 size-1048576 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0 size-524288(DMA) 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 size-524288 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0 size-262144(DMA) 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0 size-262144 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0 size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0 size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0 size-65536 12 12 65536 1 16 : tunables 8 4 0 : slabdata 12 12 0 size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0 size-32768 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0 size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0 size-16384 31 32 16384 1 4 : tunables 8 4 0 : slabdata 31 32 0 size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0 size-8192 1 1 8192 1 2 : tunables 8 4 0 : slabdata 1 1 0 size-4096(DMA) 17 17 4096 1 1 : tunables 24 12 8 : slabdata 17 17 0 size-4096 1030 1030 4096 1 1 : tunables 24 12 8 : slabdata 1030 1030 0 size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 8 : slabdata 0 0 0 size-2048 170 170 2048 2 1 : tunables 24 12 8 : slabdata 85 85 0 size-1024(DMA) 19 20 1024 4 1 : tunables 54 27 8 : slabdata 5 5 0 size-1024 893 928 1024 4 1 : tunables 54 27 8 : slabdata 232 232 0 size-512(DMA) 18 32 512 8 1 : tunables 54 27 8 : slabdata 4 4 0 size-512 576 576 512 8 1 : tunables 54 27 8 : slabdata 72 72 0 size-256(DMA) 3 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0 size-256 16444 16455 256 15 1 : tunables 120 60 8 : slabdata 1097 1097 0 size-128(DMA) 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0 size-64(DMA) 23 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0 size-64 3532 6608 64 59 1 : tunables 120 60 8 : slabdata 112 112 0 size-32(DMA) 4 112 32 112 1 : tunables 120 60 8 : slabdata 1 1 0 size-128 942 1110 128 30 1 : tunables 120 60 8 : slabdata 37 37 0 size-32 3261 3360 32 112 1 : tunables 120 60 8 : slabdata 30 30 0 kmem_cache 153 155 768 5 1 : tunables 54 27 8 : slabdata 31 31 0 On Mon, Jul 26, 2010 at 6:17 PM, Marcy Cortes <[email protected]> wrote: > Do you run at all before you run out of memory? Have you tried just starting > one of the apps at a time and seeing what each seems to do to memory usage? > Could it be that one of them has a leak? > > I think all the tweaks are really app dependent. You'll have to measure with > both a good VM performance monitor and a good java monitor. > Our largest app finds gencon garbage collection much more efficient (uses > more than 10% less CPU for them). Another found that using Async i/o in WAS > exhibited native memory leak symptoms... But other than that, we don't have > specific tweeks that we do for every app. > > > Marcy > "This message may contain confidential and/or privileged information. If you > are not the addressee or authorized to receive this for the addressee, you > must not use, copy, disclose, or take any action based on this message or any > information herein. If you have received this message in error, please advise > the sender immediately by reply e-mail and delete this message. Thank you for > your cooperation." > > > -----Original Message----- > From: Linux on 390 Port [mailto:[email protected]] On Behalf Of Daniel > Tate > Sent: Monday, July 26, 2010 3:57 PM > To: [email protected] > Subject: Re: [LINUX-390] OOM Condition on SLES11 running WAS - Tuning > problems? > > We set swappiness to 0 and started all servers. we will see if that helps > and I will look into the sp level upgrade. Does anyone have any good tweeks > for z websphere by chance? > > On Jul 26, 2010 4:59 PM, "Marcy Cortes" <[email protected]> > wrote: >> Thanks for that clarification! >> >> >> Marcy >> >> -----Original Message----- >> From: Linux on 390 Port [mailto:[email protected]] On Behalf Of Mark > Post >> Sent: Monday, July 26, 2010 2:21 PM >> To: [email protected] >> Subject: Re: [LINUX-390] OOM Condition on SLES11 running WAS - Tuning > problems? >> >>>>> On 7/26/2010 at 05:05 PM, Marcy Cortes <[email protected]> > wrote: >>> I was going to suggest a dump and a ticket to Novell, but it looks like > you >>> aren't SP1, and so are unsupported. >> >> SLES11 GA is fully supported until 6 months after SP1 went GA. Even after > that, NTS supports the product line throughout its life. We just can't get > Level 3 to look at a problem after 6 months, which means bug fixes won't be > created for a customer unless it's against SP1. >> >> >> Mark Post >> >> ---------------------------------------------------------------------- >> For LINUX-390 subscribe / signoff / archive access instructions, >> send email to [email protected] with the message: INFO LINUX-390 or > visit >> http://www.marist.edu/htbin/wlvindex?LINUX-390 >> ---------------------------------------------------------------------- >> For more information on Linux on System z, visit >> http://wiki.linuxvm.org/ >> >> ---------------------------------------------------------------------- >> For LINUX-390 subscribe / signoff / archive access instructions, >> send email to [email protected] with the message: INFO LINUX-390 or > visit >> http://www.marist.edu/htbin/wlvindex?LINUX-390 >> ---------------------------------------------------------------------- >> For more information on Linux on System z, visit >> http://wiki.linuxvm.org/ > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For more information on Linux on System z, visit http://wiki.linuxvm.org/
