Re: [Interest] memory fragmentation?

2012-08-23 Thread Thiago Macieira
On quarta-feira, 22 de agosto de 2012 17.09.21, Alex Malyushytskyy wrote: For example: assume you need 1 GB for a single array, and all other data can't grow beyond total 500 MB. First thing you can do - reserve 1 GB the earliest possible, and let other allocation happen normally.

Re: [Interest] memory fragmentation?

2012-08-23 Thread Konstantin Tokarev
22.08.2012, 21:27, Jason H scorp...@yahoo.com: Now, here's my crazy idea. If we have small objects - say Qt's interface classes, and large objects, say Qt's private classes, then could we do some d-ptr trickery where Qt can reallocate and copy/move the memory around and reassign a d-ptr?

Re: [Interest] memory fragmentation?

2012-08-23 Thread Thiago Macieira
On quinta-feira, 23 de agosto de 2012 13.10.27, Konstantin Tokarev wrote: I think this idea is more applicable to smart pointers (like QSharedPointer and QSharedDataPointer). GC would be able to move their data without breaking user code (unless somebody stores value of raw pointer somewhere

Re: [Interest] memory fragmentation?

2012-08-23 Thread Konstantin Tokarev
23.08.2012, 13:19, Thiago Macieira thiago.macie...@intel.com: On quinta-feira, 23 de agosto de 2012 13.10.27, Konstantin Tokarev wrote:  I think this idea is more applicable to smart pointers (like QSharedPointer  and QSharedDataPointer). GC would be able to move their data without  

Re: [Interest] memory fragmentation?

2012-08-23 Thread Thiago Macieira
On quinta-feira, 23 de agosto de 2012 11.18.09, André Somers wrote: Op 23-8-2012 3:54, Tony Rietwyk schreef: Hi Jason, If there are 5 public objects whose d-pointers reference the same private object address, how are you going to update them when you relocate the private object? I

Re: [Interest] memory fragmentation?

2012-08-23 Thread Adriano Rezende
On 08/22/2012 07:27 PM, ext Jason H wrote: C++ on .NET functions as it does now, however the compiler introduces the operator of ^ as a type modifier like * (pointer) ^ are handles to managed objects, as * are addresses of objects. The runtime then handles dereferencing the handles for you.

Re: [Interest] memory fragmentation?

2012-08-23 Thread Alex Malyushytskyy
On Linux, you wouldn't allocate 1 GB of memory with sbrk. You'd use an anonymous mmap, which reserves address space but provides no memory backing. The allocation is done on faulting the page. That's how glibc's malloc() serves allocations above a certain adaptive threshold. Thiago, Many thanks

Re: [Interest] memory fragmentation?

2012-08-22 Thread Lukas Geyer
Am 22.08.2012 05:45, schrieb Graeme Gill: Till Oliver Knoll wrote: Folks, I gave up checking for NULL pointers (C, malloc) or bad_alloc exceptions (new, C++) a long time ago. I remember a discussion several years ago (here on Qt interest?) about desktop memory managers actually never

Re: [Interest] memory fragmentation?

2012-08-22 Thread Graeme Gill
Lukas Geyer wrote: The address space is expanded immediately but physical memory pages are assigned at the moment the memory is accessed; either never, always or at kernels discretion, depending on the implemented overcommit strategy [1]. Right, but malloc will return NULL if the address

Re: [Interest] memory fragmentation?

2012-08-22 Thread Harri Pasanen
On 08/21/2012 01:27 AM, Alex Malyushytskyy wrote: For example one of our application on Windows ( this is the case when Windows is more flexible than Linux) - reserves the largest amount of continues memory we could afford for data (leaving enough for your widgets ) using VirtualAlloc with

Re: [Interest] memory fragmentation?

2012-08-22 Thread Lukas Geyer
Am 22.08.2012 09:44, schrieb Graeme Gill: Lukas Geyer wrote: The address space is expanded immediately but physical memory pages are assigned at the moment the memory is accessed; either never, always or at kernels discretion, depending on the implemented overcommit strategy [1]. Right, but

Re: [Interest] memory fragmentation?

2012-08-22 Thread Marc Mutz
On Wednesday August 22 2012, Konrad Rosenbaum wrote: For category 1 (real time, small memory FP) I can only suggest: separate the processes. Have one that does the critical stuff in a deterministic manner with pre-allocated memory and another process for the display - if there is a memory

Re: [Interest] memory fragmentation?

2012-08-22 Thread Jason H
the cache. From: Graeme Gill grae...@argyllcms.com To: interest@qt-project.org Sent: Tuesday, August 21, 2012 11:45 PM Subject: Re: [Interest] memory fragmentation? Till Oliver Knoll wrote: Folks, I gave up checking for NULL pointers (C, malloc) or bad_alloc

Re: [Interest] memory fragmentation?

2012-08-22 Thread Graeme Gill
Jason H wrote: This seems like it would be slow, especially when you started using haddrist swap space, unless you have that turned off. I think that it would be better to just check against a limit yourself as to avoid allocating items that would be swapped.* Not at all, because the aim

Re: [Interest] memory fragmentation?

2012-08-22 Thread Jason H
result in better (more contiguous) memory allocations. From: Konrad Rosenbaum kon...@silmor.de To: interest@qt-project.org Sent: Wednesday, August 22, 2012 4:50 AM Subject: Re: [Interest] memory fragmentation? Hi, On Tuesday 21 August 2012 12:01:49 Bo

Re: [Interest] memory fragmentation?

2012-08-22 Thread BRM
From: Jason H scorp...@yahoo.com C++ on .NET functions as it does now, however the compiler introduces the operator of ^ as a type modifier like * (pointer) ^ are handles to managed objects, as * are addresses of objects. The runtime then handles dereferencing the handles for you. jsut like

Re: [Interest] memory fragmentation?

2012-08-22 Thread Alex Malyushytskyy
Harri Pasanen ha...@mpaja.com wrote: I wonder why you say Linux is less flexible? Couldn't you just write a custom allocator directly using sbrk()? Also, 32 bit linux process leaves more application space free, so you can easily reach 2Gb process size, while I recall windows XP having issues

Re: [Interest] memory fragmentation?

2012-08-22 Thread Constantin Makshin
. From: Konrad Rosenbaum kon...@silmor.de To: interest@qt-project.org Sent: Wednesday, August 22, 2012 4:50 AM Subject: Re: [Interest] memory fragmentation? Hi, On Tuesday 21 August 2012 12:01:49 Bo Thorsen wrote: Memory fragmentation is defined as the problem where you allocate

Re: [Interest] memory fragmentation?

2012-08-22 Thread Jason H
that. From: Constantin Makshin cmaks...@gmail.com To: Qt Interest interest@qt-project.org Sent: Wednesday, August 22, 2012 8:37 PM Subject: Re: [Interest] memory fragmentation? I doubt that implementing moveable private objects/pointers would be [really] useful. Firstly, Qt can't control

Re: [Interest] memory fragmentation?

2012-08-22 Thread Tony Rietwyk
AM To: Constantin Makshin; Qt Interest Subject: Re: [Interest] memory fragmentation? I'll take that as a compliment. I fully expected someone to raise the issues you did. And I will counter them with the following comments. Yes, Qt would be less predictable, but more predictable than .NET's

Re: [Interest] memory fragmentation?

2012-08-21 Thread Thiago Macieira
On segunda-feira, 20 de agosto de 2012 17.31.27, John Weeks wrote: On 20-Aug-2012, at 4:20 PM, Thiago Macieira wrote: In any case, if the application needs memory and can't get it, what do you suggest it do? You put up an alert telling the user that the operation can't be completed due to

Re: [Interest] memory fragmentation?

2012-08-21 Thread Rainer Wiesenfarth
Am 21.08.2012 09:52, schrieb Thiago Macieira: On segunda-feira, 20 de agosto de 2012 17.31.27, John Weeks wrote: On 20-Aug-2012, at 4:20 PM, Thiago Macieira wrote: In any case, if the application needs memory and can't get it, what do you suggest it do? You put up an alert telling the user

Re: [Interest] memory fragmentation?

2012-08-21 Thread Bo Thorsen
Hi Jason, I'm going back to the original question, the thread seems to have wandered off :) Den 20-08-2012 17:51, Jason H skrev: What can be done to combat this in C++/Qt? Initially I thought there might be some kind of QObject d-ptr magic where the large allocations can be in the private

Re: [Interest] memory fragmentation?

2012-08-21 Thread Konstantin Tokarev
Hi all, My question is slightly off-topic, but are there any dynamic analysis tools for measuring memory fragmentation? -- Regards, Konstantin ___ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest

Re: [Interest] memory fragmentation?

2012-08-21 Thread Thiago Macieira
On terça-feira, 21 de agosto de 2012 11.53.45, Rainer Wiesenfarth wrote: How? Putting up an alert usually requires allocating memory somewhere, be it in your application or in the display server. Look at the various solutions used as new handlers. For example, you could pre-allocate some

Re: [Interest] memory fragmentation?

2012-08-21 Thread Adriano Rezende
On 08/20/2012 05:51 PM, ext Jason H wrote: What can be done to combat this in C++/Qt? Initially I thought there might be some kind of QObject d-ptr magic where the large allocations can be in the private class and re-allocate and copy the private class, without affecting the pointers that

Re: [Interest] memory fragmentation?

2012-08-21 Thread Bob Hood
On 8/20/2012 3:39 PM, Scott Aron Bloom wrote: Ok.. Call me taught.. but I can tell you, last time I tried, it didn't compile in 64 bit mode. I only use VS2008, and I have routinely compiled 4.6, 4.7 and 4.8 lines of Qt in both 32- and 64-bit modes for my project. You're just not trying hard

Re: [Interest] memory fragmentation?

2012-08-21 Thread Jason H
By returning out of memory, unrolling the stack, then at the top level displaying the error. From: Thiago Macieira thiago.macie...@intel.com To: interest@qt-project.org Sent: Tuesday, August 21, 2012 3:52 AM Subject: Re: [Interest] memory fragmentation

Re: [Interest] memory fragmentation?

2012-08-21 Thread Jason H
? From: Tony Rietwyk t...@rightsoft.com.au To: interest@qt-project.org Sent: Monday, August 20, 2012 9:52 PM Subject: Re: [Interest] memory fragmentation? ... In the end I chose Qt 4.8.1  QXmlStreamWriter(QString *).   The QByteArray version seemed the logical choice - but since the API only

Re: [Interest] memory fragmentation?

2012-08-21 Thread Atlant Schmidt
. Atlant From: interest-bounces+aschmidt=dekaresearch@qt-project.org [mailto:interest-bounces+aschmidt=dekaresearch@qt-project.org] On Behalf Of Jason H Sent: Tuesday, August 21, 2012 10:17 AM To: Tony Rietwyk; interest@qt-project.org Subject: Re: [Interest] memory fragmentation

Re: [Interest] memory fragmentation?

2012-08-21 Thread pritam.ghang...@gmail.com
On Wed, Aug 22, 2012 at 9:15 AM, Graeme Gill grae...@argyllcms.com wrote: Till Oliver Knoll wrote: Folks, I gave up checking for NULL pointers (C, malloc) or bad_alloc exceptions (new, C++) a long time ago. I remember a discussion several years ago (here on Qt interest?) about desktop

Re: [Interest] memory fragmentation?

2012-08-20 Thread Thiago Macieira
On segunda-feira, 20 de agosto de 2012 07.56.36, Jason H wrote: In comparing technologies (Qt vs .NET) one of the biggest architectural differences is the garbage collection model. Qt uses the standard C++ new/free, where .NET uses a garbage collector. Interestingly the GC not only manages

Re: [Interest] memory fragmentation?

2012-08-20 Thread Jason H
thiago.macie...@intel.com To: interest@qt-project.org Sent: Monday, August 20, 2012 11:23 AM Subject: Re: [Interest] memory fragmentation? On segunda-feira, 20 de agosto de 2012 07.56.36, Jason H wrote: In comparing technologies (Qt vs .NET) one of the biggest architectural differences

Re: [Interest] memory fragmentation?

2012-08-20 Thread Atlant Schmidt
=dekaresearch@qt-project.org] On Behalf Of Jason H Sent: Monday, August 20, 2012 11:51 AM To: Thiago Macieira; interest@qt-project.org Subject: Re: [Interest] memory fragmentation? What can be done to combat this in C++/Qt? Initially I thought there might be some kind of QObject d-ptr magic where

Re: [Interest] memory fragmentation?

2012-08-20 Thread Thiago Macieira
On segunda-feira, 20 de agosto de 2012 08.51.02, Jason H wrote: What can be done to combat this in C++/Qt? In Qt, very little. We're not going to change the programming paradigm. In C++, a bit more. You can write code with special allocators. You can also change the implementation of malloc(),

Re: [Interest] memory fragmentation?

2012-08-20 Thread Ben Swerts
Message- From: interest-bounces+benswerts=telenet...@qt-project.org [mailto:interest- bounces+benswerts=telenet...@qt-project.org] On Behalf Of Scott Aron Bloom Sent: Monday, August 20, 2012 19:07 To: Thiago Macieira; interest@qt-project.org Subject: Re: [Interest] memory fragmentation

Re: [Interest] memory fragmentation?

2012-08-20 Thread Thiago Macieira
On segunda-feira, 20 de agosto de 2012 10.07.20, Scott Aron Bloom wrote: Really? Sorry but where can I find a win64-msvc2008 mkspecs file?? If it exists, Im stumped... It's called win32-msvc2008. win32 doesn't stand for the number of bits, it stands for the Win32 API. That is not changed in

Re: [Interest] memory fragmentation?

2012-08-20 Thread d3fault
Interesting thread. I like the bit about allocating everything you could possibly need at application startup. I tend to do the same thing when dealing with multiple buffers (never crossed my mind to do it with widgets), but always wonder if I'm prematurely eja optimizing. Onto my question: Qt

Re: [Interest] memory fragmentation?

2012-08-20 Thread Yuvraaj Kelkar
Of Thiago Macieira Sent: Monday, August 20, 2012 1:30 PM To: interest@qt-project.org Subject: Re: [Interest] memory fragmentation? On segunda-feira, 20 de agosto de 2012 10.07.20, Scott Aron Bloom wrote: Really? Sorry but where can I find a win64-msvc2008 mkspecs file?? If it exists, Im

Re: [Interest] memory fragmentation?

2012-08-20 Thread Thiago Macieira
On segunda-feira, 20 de agosto de 2012 14.39.08, Scott Aron Bloom wrote: Ok.. Call me taught.. but I can tell you, last time I tried, it didn't compile in 64 bit mode. Last time I tried, it did. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source

Re: [Interest] memory fragmentation?

2012-08-20 Thread Alex Malyushytskyy
Nonsense. Any application using a gigabyte or more of memory should HAVE SWITCHED to 64-bit a couple of years ago. It is not nonsense. You provide to users version which works on their system (whatever they have). And even though application may need close to 2GB of data (or more) to work

Re: [Interest] memory fragmentation?

2012-08-20 Thread John Weeks
On 20-Aug-2012, at 4:20 PM, Thiago Macieira wrote: In any case, if the application needs memory and can't get it, what do you suggest it do? You put up an alert telling the user that the operation can't be completed due to insufficient memory, and then field the tech support call that