Re: jBASE 4.1.5.17 - does anyone face "out of memory" problems after SSELECT?

[Jim Idle]

Pawel (privately) wrote: Welcome back, I suggested MALLOCOPTION=disclaim, because we have noticed that on 1 of test machines memory situation was like that: 01:02 AM: ~ jBASE Free in total 350 MB, jBASE Used in total 5 GB 03:20 AM: ~ jBASE Free in total 8 GB, jBASE Used in total 7 GB (more "Free" than "Used", this is not my mistake) As I say, it is possible that that option would allow the system to reuse the pages, but the real problem is the allocation pattern. You need to use watson and gather the same information. Above output suprised me seriously. Shown numbers are not exact - I tried to recall them and they are quite precise. Situation at 3:20 AM was exactly like that (server has 24 GB of physical mem + 10 GB of swap). We have also noticed that memory allocation growth (jump) is on SSELECTs. So I am guessing that freed memory should formulate continious blocks. Well, it will be as many blocks as were required to be re-allocated, but when freed were not big enough to satisfy further growth. That is why I claimed all the time that "Free" memory is not given back to other processes and this is our problem. It's not quite how it works, but it is a problem, that watson should address by allowing the processes to reuse the freed blocks (because it can amalgamate them better upon free()) I understood that this may be problem of memory fragementation, but I think that yorktown allocator never gives memory back unless (probably) MALLOCOPTION=disclaim is set. Well, the disclaim is only going to let things shrink when it can. Given the allocation pattern, my feeling is that it would not make much difference, but it might. That why I asked about memory page size. I am thinking about this whole stuff and can not imagine such a big fragmentation (notice that at 3:20 AM are processes which for example allocated and used 300 MB, but immediately went back to 100 MB used and 200 MB free). Different allocation patterns allow different reuse, and of course if a new process starts up then it starts from scratch. Speculation isn't going to help you basically, you have to test it. We have checked also wheter some processes try to allocate more than 2 GB and there are such COB agents, but hopefully not many. We need to do more analysis carefully. Testing of Watson is scheduled to weekend - you can imagine now how people from test team are protesting against such changes :) Well, they shouldn't be - they obviously don't understand the process. That's what your test system is for and I don't know the details of why you cannot just switch that on in your test machine now, but I am sure it is politics rather than technical points. I forgot to tell, but QA area was all the time working on MALLOCTYPE=buckets. We did this setting, because people from CSHD gave us such advice. We wonder if it could somehow accelerate our problems or not. Recently we are facing out of memory problems on test servers and we did not (!) increase memory/swap size. Well, you are using different policies on your test and live machines now, so the QA doesn't mean anything :-). You need your QA and live machines running the same parameters exactly, other than when you are testing new parameters for going live with. Jim --~--~---------~--~----~------------~-------~--~----~ Please read the posting guidelines at: http://groups.google.com/group/jBASE/web/Posting%20Guidelines IMPORTANT: Type T24: at the start of the subject line for questions specific to Globus/T24 To post, send email to jBASE@googlegroups.com To unsubscribe, send email to jbase-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/jBASE?hl=en -~----------~----~----~----~------~----~------~--~---