If you think there are memory leaks and you are on the Solaris platform, you can use these tools to find out if there are memory leaks: - mdb - libumem
See here for more information: http://blogs.oracle.com/pnayak/entry/finding_memory_leaks_within_solaris If you don't want to run the debugger against your production server/process, you can generate a core file and evaluate it the same way on a separate machine. Do note that while generating the core the process will freeze. The speed at which the core file is written and the size of the core file will determine the amount of time required to generate the core file. I suggest using libumem for all the Remedy process anyway though, since the application is heavily multi-threaded. This will significantly increase the memory footprint, but it comes with benefits. There are several memory allocators in Solaris and the default is less than suitable for multi-threaded applications. - malloc - mtmalloc - ptmalloc - Hoard - libumem See here for a brief comparison of the various memory management libraries: http://dsc.sun.com/solaris/articles/multiproc/multiproc.html libumem will help with the performance of the application in addition to giving you the tools to find out if there are memory leaks and where the leaks are coming from, at the cost of increasing the memory usage of the process. Axton Grams On Fri, Dec 30, 2011 at 8:45 PM, L G Robinson <[email protected]> wrote: > ** Hi Patrick, > > Thanks for the link, but the Mid-tier isn't even running on the same > hardware. The AR Server software running on the Solaris box is: > > AR Server 7.6.03 Build 001 201008170035 > > Here is additional memory information throughout the day, 5 minute > intervals. Memory usage continues to increase for no apparent reason. Is > this normal? > > S UID PID PPID C PRI NI RSS SZ WCHAN STIME TTY > TIME CMD > S root 21007 21006 0 40 20 464456 473136 ? 05:00:39 ? > 9:30 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 468552 477232 ? 05:00:39 ? > 9:41 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 468552 477232 ? 05:00:39 ? > 9:45 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 472648 481328 ? 05:00:39 ? > 9:52 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 476968 485648 ? 05:00:39 ? > 10:04 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 1 40 20 481064 489744 ? 05:00:39 ? > 10:16 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 481064 489744 ? 05:00:39 ? > 10:23 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 485160 493840 ? 05:00:39 ? > 10:31 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 489256 497936 ? 05:00:39 ? > 10:35 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 489256 497936 ? 05:00:39 ? > 10:41 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 493352 502032 ? 05:00:39 ? > 10:50 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 497448 506128 ? 05:00:39 ? > 10:56 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 497448 506128 ? 05:00:39 ? > 11:05 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 501544 510224 ? 05:00:39 ? > 11:11 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 505640 514320 ? 05:00:39 ? > 11:18 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 505640 514320 ? 05:00:39 ? > 11:27 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 509736 518416 ? 05:00:39 ? > 11:36 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 513832 522512 ? 05:00:39 ? > 11:43 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 513832 522512 ? 05:00:39 ? > 11:51 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 517928 526608 ? 05:00:39 ? > 11:58 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 522024 530704 ? 05:00:39 ? > 12:04 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 522040 530720 ? 05:00:39 ? > 12:14 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 526136 534816 ? 05:00:39 ? > 12:20 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 530232 538912 ? 05:00:39 ? > 12:26 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 530232 538912 ? 05:00:39 ? > 12:36 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 534328 543008 ? 05:00:39 ? > 12:49 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 50 20 538424 547104 ? 05:00:39 ? > 12:57 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 538424 547104 ? 05:00:39 ? > 13:12 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 542520 551200 ? 05:00:39 ? > 13:18 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 558904 567584 ? 05:00:39 ? > 13:48 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 558904 567584 ? 05:00:39 ? > 14:02 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 48 20 558904 567584 ? 05:00:39 ? > 14:16 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 558904 567584 ? 05:00:39 ? > 14:24 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 558904 567584 ? 05:00:39 ? > 14:40 ./arserverd -s ars00srv -i /local/b > O root 21007 21006 0 49 20 558904 567584 05:00:39 ? > 14:48 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 563000 571680 ? 05:00:39 ? > 14:59 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 571328 580008 ? 05:00:39 ? > 15:13 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 571376 580056 ? 05:00:39 ? > 15:21 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 571432 580112 ? 05:00:39 ? > 15:35 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 575528 584208 ? 05:00:39 ? > 15:44 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 579624 588304 ? 05:00:39 ? > 15:57 ./arserverd -s ars00srv -i /local/b > O root 21007 21006 0 99 20 583736 592416 05:00:39 ? > 16:31 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 583736 592416 ? 05:00:39 ? > 17:19 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 587832 596512 ? 05:00:39 ? > 17:30 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 591928 600608 ? 05:00:39 ? > 17:36 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 591928 600608 ? 05:00:39 ? > 17:50 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 596024 604704 ? 05:00:39 ? > 18:00 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 600128 608808 ? 05:00:39 ? > 18:14 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 600128 608808 ? 05:00:39 ? > 18:25 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 604224 612904 ? 05:00:39 ? > 18:31 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 608320 617000 ? 05:00:39 ? > 18:40 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 608320 617000 ? 05:00:39 ? > 18:51 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 612416 621096 ? 05:00:39 ? > 18:59 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 612416 621096 ? 05:00:39 ? > 19:04 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 616512 625192 ? 05:00:39 ? > 19:15 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 620608 629288 ? 05:00:39 ? > 19:23 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 620608 629288 ? 05:00:39 ? > 19:29 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 624704 633384 ? 05:00:39 ? > 19:40 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 628800 637480 ? 05:00:39 ? > 19:44 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 628800 637480 ? 05:00:39 ? > 19:50 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 632896 641576 ? 05:00:39 ? > 20:01 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 632896 641576 ? 05:00:39 ? > 20:07 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 636992 645672 ? 05:00:39 ? > 20:15 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 641088 649768 ? 05:00:39 ? > 20:26 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 641088 649768 ? 05:00:39 ? > 20:31 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 645184 653864 ? 05:00:39 ? > 20:38 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 649280 657960 ? 05:00:39 ? > 20:48 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 649280 657960 ? 05:00:39 ? > 20:58 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 653376 662056 ? 05:00:39 ? > 21:04 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 49 20 657472 666152 ? 05:00:39 ? > 21:17 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 657472 666152 ? 05:00:39 ? > 21:23 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 665664 674344 ? 05:00:39 ? > 21:33 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 665664 674344 ? 05:00:39 ? > 21:44 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 665664 674344 ? 05:00:39 ? > 21:52 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 673856 682536 ? 05:00:39 ? > 22:05 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 673856 682536 ? 05:00:39 ? > 22:17 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 673856 682536 ? 05:00:39 ? > 22:24 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 677952 686632 ? 05:00:39 ? > 22:29 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 682048 690728 ? 05:00:39 ? > 22:46 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 682048 690728 ? 05:00:39 ? > 22:53 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 686144 694824 ? 05:00:39 ? > 22:59 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 690240 698920 ? 05:00:39 ? > 23:10 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 690240 698920 ? 05:00:39 ? > 23:16 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 694336 703016 ? 05:00:39 ? > 23:27 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 698432 707112 ? 05:00:39 ? > 23:39 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 698432 707112 ? 05:00:39 ? > 23:47 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 702528 711208 ? 05:00:39 ? > 24:30 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 706624 715304 ? 05:00:39 ? > 24:43 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 710720 719400 ? 05:00:39 ? > 24:50 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 710720 719400 ? 05:00:39 ? > 24:57 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 714816 723496 ? 05:00:39 ? > 25:11 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 714816 723496 ? 05:00:39 ? > 25:18 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 723008 731688 ? 05:00:39 ? > 25:28 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 723008 731688 ? 05:00:39 ? > 25:41 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 723008 731688 ? 05:00:39 ? > 25:49 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 727104 735784 ? 05:00:39 ? > 25:56 ./arserverd -s ars00srv -i /local/b > O root 21007 21006 0 40 20 731200 739880 05:00:39 ? > 26:08 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 731200 739880 ? 05:00:39 ? > 26:16 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 735296 743976 ? 05:00:39 ? > 26:22 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 739392 748072 ? 05:00:39 ? > 26:35 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 739392 748072 ? 05:00:39 ? > 26:55 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 743488 752168 ? 05:00:39 ? > 27:04 ./arserverd -s ars00srv -i /local/b > S root 21007 21006 0 40 20 747584 756264 ? 05:00:39 ? > 27:19 ./arserverd -s ars00srv -i /local/b > > > Thanks for any further insights. > Larry > > On Fri, Dec 30, 2011 at 2:51 PM, patrick zandi <[email protected]>wrote: > >> ** https://communities.bmc.com/communities/docs/DOC-18162 >> >> What Patch you on??? >> >> >> On Fri, Dec 30, 2011 at 1:35 PM, L G Robinson <[email protected]> wrote: >> >>> ** Hi Phil, >>> >>> >>> I have taken a few snapshots of the arserverd during the day. Looks like >>> the memory usage is slowly creeping up. The slowdown is not too bad yet, >>> but noticeably slower than immediately following the restart: >>> >>> S UID PID PPID C PRI NI RSS SZ WCHAN STIME TTY >>> TIME CMD >>> S root 21007 21006 0 40 20 415296 423976 ? 05:00:39 ? >>> 5:35 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 419392 428072 ? 05:00:39 ? >>> 6:33 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 439880 448560 ? 05:00:39 ? >>> 8:03 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 464456 473136 ? 05:00:39 ? >>> 9:30 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 468552 477232 ? 05:00:39 ? >>> 9:41 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 468552 477232 ? 05:00:39 ? >>> 9:45 ./arserverd -s ars00srv -i /local/b >>> S root 21007 21006 0 40 20 472648 481328 ? 05:00:39 ? >>> 9:52 ./arserverd -s ars00srv -i /local/b >>> >>> Is this normal? Should the memory usage be increasing like this? The >>> above is over a three hour period. >>> >>> Thanks. >>> Larry >>> >> _attend WWRUG12 www.wwrug.com ARSlist: "Where the Answers Are"_ _______________________________________________________________________________ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org attend wwrug12 www.wwrug12.com ARSList: "Where the Answers Are"
