Hi,
it also may be that your GPU runs out of memory and starts to kick out
textures, then those has to be stored in RAM and uploaded again when needed.
This can be monitored with the OpenGL Driver Monitor app.
Btw: 15 sprites/Billboards shouldn't be a problem at all. Usually you have to
care about how many pixels you are rendering and therefor 15 times the complete
screen could be a lot of pixels (totally depending on how your result looks
like and how often you rendering a single pixel again and again). This is
important when going for speed. In your case it seems you are doing off-screen
rendering into a movie file which hasn't be in real time.
What kind of hardware you are using?
Achim Breidenbach
Boinx Software
On 27.03.2012, at 15:01, Mathieu BEN wrote:
> Dear Achim,
>
> many thanks for your answer.
>
> 1. Unfortunately we yet have Clear patches in our compositions...
> Note that the 'ghost' images appeared when our QC composition became more and
> more complex. Do you think there could be a limit on the number of levels of
> overlayed 'sprites' or 'billboards' in a composition, which could be related
> to the 'number of texture units' given by the 'OpenGL Info' patch of QC (8 in
> our case) ?
>
> 2. I know Core Image kernels are specifically designed to renderer on GPU.
> This should be the case when using CI programmatically. But is it actually
> the case when using a Core Image patch in QC ? Indeed, I noticed a dramatic
> rendering frame rate drop when using Core Image patches to do the image
> compositing in Quartz Composer, in addition to a large increase in RAM usage.
> Furthermore, we did not change anything (at all) in our Cocoa application
> between test 1. (image compositing with overlayed sprites or billboards) and
> test 2. (image compositing using a Core Image patch), we only modified our
> Quartz Composer patches. And the memory leak happenned... So it seems that
> the leak is indeed in QC, not in our application (which we tested against
> memory leaks with Instruments)
>
> Best,
>
> Mathieu
>
> Le 27/03/12 12:38, Achim Breidenbach a écrit :
>>
>> Hello Mathieu and Boris,
>>
>> 1) Sometimes I get those artifacts when not using a "Clear" patch even if
>> any of the other Billboards and Sprites are rendering full screen. When you
>> don't clear the buffer you are rendering in first, it could be that there is
>> some texture garbage with z-buffer garbage in it. This happens when the
>> graphics card rearranged its memory blocks. The "Clear" (don't forget to set
>> it up to be rendered first ;-) ) will erase the destination buffer and make
>> sure that in every render cycle it looks the same.
>>
>> 2) Of cause the CoreImage kernels are rendered in the GPU. Only in some rare
>> cases QuartzComposer decides to render something on the CPU (e.g. some core
>> image code can't be processed by the used GPU). Then it falls back to
>> software rendering which causes the frame rate dropping dramatically. I am
>> sure you would notice this too. If you are getting the memory issues within
>> your application you can use Instruments to monitor the memory usage
>> and drill down to the very object which allocated the memory blocks.
>>
>> best,
>>
>> Achim Breidenbach
>> Boinx Software
>>
>>
>> On 27.03.2012, at 11:35, Mathieu BEN wrote:
>>
>>> Hello Achim,
>>>
>>> I'm joining the discussion as I've been working on this issue with Boris.
>>>
>>> Actually we've been facing 2 problems with Quartz Composer:
>>>
>>> 1. The first one was something like 'ghost' images appearing in a
>>> composition frame where images from a QuickTime source movie were rendered.
>>> The rest of the composition is made of still images representing banners,
>>> logos, illustration images, etc.. . The QC composition is quite complex and
>>> the total number of rendering patches (Sprites or Billboards) reaches 15
>>> and even more (more than 15 levels of overlayed images).
>>> We use the QC composition in a Cocoa application with a QCRenderer
>>> initialized 'off-screen', and we write images (NSImage objects) from the
>>> source movie (via a QTMovie object) on a published image input of the
>>> composition. Then we call the snapshotImage: method of the QCRenderer to
>>> get ouput images from the composition and put them in an ouput QTMovie.
>>> As explained, we sometimes (not always) have 'ghost' images appearing in
>>> the output movie, in the frame where the source video is rendered. This
>>> ghost images randomly and sparsely occur (3 or 4 times per minutes of
>>> generated video), and they correspond to other images used in the
>>> composition (banners, logos,..), always appearing in their original side,
>>> color, orientation, etc., so they may correspond to input image buffers,
>>> substituing an output buffer... . Furthermore, these ghost images usually
>>> appear in groups of 3 or 4, each separated from the following one with a
>>> single 'expected' source movie image.
>>> We had no memory leak issue with this setup, only 'ghost' images
>>>
>>>
>>> 2. The second problem was the one explained by Boris: to by-pass the first
>>> problem of 'ghost' images, we tried to reduce the number of rendering
>>> modules (sprites and/or billboards) in the composition and using Core Image
>>> filters embeded in the composition to do the image compositing (image
>>> overlay). We tried 2 things for that : 1) simply using a
>>> CISourceOverCompositing filter in the 'edit filter function' of the Core
>>> Image Quartz Composer patch. 2) coding the following Core Image kernel in
>>> the CI QC patch :
>>>
>>> kernel vec4 coreImageKernel(sampler bckImage, sampler topImage)
>>> {
>>> vec4 bckPix, topPix;
>>> topPix = sample(topImage, samplerCoord(topImage));
>>> bckPix = sample(bckImage, samplerCoord(bckImage));
>>>
>>> return topPix + (1.0 - topPix.a) * bckPix;
>>> }
>>>
>>>
>>> Both of these solutions did the overlay work correctly but both lead to a
>>> big memory leak, in particular with HD source movie images, rendered in HD
>>> ouput images: in this case the 24 GBytes of our MacPro was eaten in a few
>>> tens of seconds, while our Cocoa application doesn't use more than
>>> 1,5GBytes of memory (according to the activity monitor of the Mac).
>>>
>>> My hypothesis is that Quartz Composer actually does the image compositing
>>> in GPU when using the rendering patches like sprites and billboards, but
>>> that Core Image filters embeded in a QC composition does not, thus working
>>> in RAM and causing the observed memory leak.
>>>
>>> I have no idea now how to do our compositing work with these to issue, wich
>>> really look like bugs in Quartz Composer...
>>> Any suggestion would be appreciated.
>>>
>>> Best,
>>>
>>> Mathieu
>>>
>>>
>>>>> De : Achim Breidenbach <ac...@boinx.com>
>>>>> Objet : Rép : Quartz Composer memory leak
>>>>> Date : 27 mars 2012 09:28:04 HAEC
>>>>> À : Boris Rousseau <boris.rouss...@powedia.com>
>>>>> Cc : quartzcomposer-dev@lists.apple.com
>>>>>
>>>>> Hello Boris,
>>>>>
>>>>> in my experience QuartzComposer does all the graphics processing in the
>>>>> GPU not in the CPU. If your RAM goes up, this is usually caused by the
>>>>> inputs you are using. E.g. a H264 video has a big memory footprint. Can
>>>>> you describe in more detail, what your composition does and what sources
>>>>> are you using? What does "suddenly" mean to you (after certain seconds,
>>>>> randomly)?
>>>>>
>>>>> best,
>>>>>
>>>>> Achim Breidenbach
>>>>> Boinx Software
>>>>>
>>>>> On 27.03.2012, at 08:42, Boris Rousseau wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I am facing a memory leak while using Quartz Composer to process and
>>>>>> generate videos.
>>>>>> Quartz Composer starts processing the video in HD starting from 700Mb
>>>>>> and suddenly raises over the RAM limit until the Mac Pro swaps.
>>>>>> Apparently, the issue is probably link to Core Image processing while
>>>>>> converting or rescaling images to HD but I am not sure about it.
>>>>>>
>>>>>> Did anyone already have this problem, please ?
>>>>>> How do I debug Quartz Composer memory leak, please ?
>>>>>>
>>>>>> Thanks for your help.
>>>>>>
>>>>>> --
>>>>>> Boris Rousseau
>>>>>> <Pièce jointe Mail.png>
>>>>>>
>>>>>> 12A Avenue des Peupliers
>>>>>> 35510 Cesson-Sévigné, France
>>>>>> fixe: +(33) 2 30 96 12 83
>>>>>> mobile: +(33) 6 19 12 65 54
>>>>>> www.powedia.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Do not post admin requests to the list. They will be ignored.
>>>>>> Quartzcomposer-dev mailing list (Quartzcomposer-dev@lists.apple.com)
>>>>>> Help/Unsubscribe/Update your Subscription:
>>>>>> https://lists.apple.com/mailman/options/quartzcomposer-dev/achim%40boinx.com
>>>>>>
>>>>>> This email sent to ac...@boinx.com
>>>>>
>>>>
>>>
>>
>
> --
> Mathieu BEN
> <logo-O-Bleu1-Fond-Blanc-petit.png>
> 12A Avenue des Peupliers
> 35510 Cesson-Sévigné, France
> +(33) 2 30 96 12 84
> www.powedia.com
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Quartzcomposer-dev mailing list (Quartzcomposer-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/quartzcomposer-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
