[ looping the list back into the discussion, as this is generally useful info... ]
Glad to see this was resolved! Unfortunately, I couldn't find this "don't modify input buffers" requirement in the xDM docs(!), though it is a requirement. I see it's been added to the xDM "1.0 beta" interface docs (and will be in the "1.0 final" docs as well) - see the "Postcondition" of IVIDDEC1_process() here, for example: https://www-a.ti.com/downloads/sds_support/targetcontent/XDAIS/xdais_5_1 0/xdais_5_10/docs/html/struct_i_v_i_d_d_e_c1___fxns.html#af6e71c633dc644 f468bece76b6ea696 I'll try to add this to the 0.9 xDM interface docs in the next xDAIS release so it's clearly stated. Note that, for similar reasons, you can't pass data _to_ the codec in the outBufs... because those buffers aren't invalidated prior to calling the codec's process() - and the codec may get stale data. ------------------------------ As for _why_ this is required - it's a contract between the codecs and the framework, so each side has clear responsibilities. In your case, the codec modified the input buffer and the framework _didn't_ ensure there is coherency of that input buffer between the cached contents (containing your modifications) and the physical memory (containing the original buffer). After the process() call returned to the ARM-side app, at some point later the DSP-side cache controller needed to bring something else into cache, and evicted the cached contents (your inBuf modifications) into physical memory. This corrupted the physical memory that's now owned by the ARM side. This may happen _much_ later, so the corruption of that memory could happen at any time (very bad). Chris > -----Original Message----- > From: X. Zhou [mailto:[EMAIL PROTECTED] > Sent: Tuesday, November 28, 2006 3:39 AM > To: Ring, Chris; [EMAIL PROTECTED] > Subject: RE: cache coherency problem > > > > hi, Chris and santhoshk, sorry for my carelessness. > > At the moment of sending the last mail, I forget one thing: > there is a > branch in my codec where CPU will write the input buffer, ( since this > branch is added some days ago, I almost forget it! ) > > Unit this afternoon, I recalled it on a sudden. And now I > have modified > this branch to remove the write access for input buffer, it seems that > thehe cache coherency problem doesn't appear any more. > > Yes, Chris told the truth : the codec should not write into an input > buffer !!!!!!!!! > > Chris, you are great! Thank you very very much! > And santhoshk, thank you too for your warm-heart. > > But I still have a doubt, why the XDAIS-compliant codecs should not > write input buffers, in the "FrequentlyAskedQuestionsCache.pdf" doc, > the below rules is refered that: > > ----- C6000 algorithms must not issue any CPU read/writes to > buffers in > external memory that are involved in DMA transfers. This also > applies to > the input buffers passed to the algorithm through its algorithm > interface. > > According to my understanding, this rules only applies to the input > buffers which are involved in DMA transfers. Since DMA transfer will > not keep cosistency between cache and external memory. > > > But in my case, at DSP side, the input buffer will be > accessed(read/written) only by CPU, and never be accessed by DMA !!! > > Why CPU write access is still prohibited in this case? > > Wait for your response! > > Thank again for you both! :) > > > Best Regards, > Zhou > > > > > -----Original Message----- > From: X. Zhou > > Sent: Tuesday, November 28, 2006 10:24 AM > To: 'Ring, Chris'; '[EMAIL PROTECTED]' > Cc: '[email protected]' > Subject: RE: cache coherency problem > > > -------------------------------------------------------------- > ---------- > -------------------------------------------------------------- > ---------- > ---------- > [ If the codec is xDAIS-xDM compliant, do we need to the cache > ivalidation/flush with the process() call? ] > -------------------------------------------------------------- > ---------- > -------------------------------------------------------------- > ---------- > ---------- > > Yes, I agree with Chris. According to the feedback in my DSP > trace file, > it is indicated that the skeleton of codec egine will do the cache > management for DSP before and after each processing call, so > the codec, > if it is xDAIS-xDM compliant, need not do cache invalidation/flush > manually. > > Besides, I think, the ARM application should not do any cache > invalidation/flush operation since all the input/ouput buffers are > non-cached. > > -------------------------------------------------------------- > ---------- > -------------------------------------------------------------- > ---------- > ---------- > [ Zhou, I'm interested in helping you resolve this. Did my last mail > help? Which codec are you using (is it one of TI's or one of your > own?)? Could it be either 1) buffer mis-alignment (i.e.not on 128-bit > boundary) or 2) the codec writing into an input buffer? > 3)Can either of > you reproduce the issue in one of the simple copy-codecs? ] > -------------------------------------------------------------- > ---------- > -------------------------------------------------------------- > ---------- > ---------- > > Chris, thank you for your help. The codec is a H.264 video > decoder of my > own. There are only one input buffer for my decoder ------ bistream > buffer, and one output buffer -----reconstructed image buffer. > > (1) I have checked my input/ouput buffer physical address, > thery are all > 128-bit boundary. > > But for each procssing call, the size of input buffer is not always > 128-bit aligned. Since I use this input buffer to transfer bitstream > between DSP and ARM. As you know, the bistream length is a variable > element. But I don't think it is the cause of this problem, since no > other buffers will share the same cache line with this buffer ------- > the bistream input buffer is pre-allocated with enough space ( 8000000 > bytes). > (2) I don't think my decoder will write the bitstream buffer. For each > processing call, my decoder read bistream from the bitstream > buffer and > if decoding successlly, write reconstrued image to reconstructed image > buffer and some feedback info to outArgs, else, only wirte feedkback > info to outArgs. > (3) I havn't test whether the issue will be reproduced if I > use a simple > copy-codecs, but I have ever done a experiment -- disable the main > operations of my processing call, i.e.: a almost-null processing call, > the issue still exists. > (4) Until now, I have done lot of experiments. Per some hints, I found > that if only I disable video capture and OSD device in my ARM > application, the system seems ok. now I doubt that whether > video capture > device and OSD device will do any cache management. Anyway, if any > progress, I'll inform you both. > > > P.s.: More detail about the control flow of my applcation: > 1) start video capture device > 2) start video display device > 3) start OSD device > 4) capture a video frame from camera > 5) encode one frame via a XDM-compliant encoder, and save the > bitstream to file (optional, can be skipped) > 7) read bitstream from another file > 8) decode one frame via a XDM-compliant decoder, and output the > reconstructed image to video display buffer > 9) jump to (1), looping... > > > Thanks you both very much! > > B.R, > Zhou > > > -----Original Message----- > From: Ring, Chris [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, November 28, 2006 12:28 AM > To: Santhosh Kumar B N > Cc: X. Zhou > Subject: RE: cache coherency problem > > > By design, the Codec Engine framework manages cache across the ARM-DSP > boundaries, for the 8 VISA classes (the 8 xDM classes). The codec > should _not_ have to manage cache. > > If you're not using the Codec Engine, your framework will > similarly have > to manage cache across the ARM-DSP boundary. > > Chris > > > > -----Original Message----- > > From: Santhosh Kumar B N [mailto:[EMAIL PROTECTED] > > Sent: Monday, November 27, 2006 7:01 AM > > To: Ring, Chris > > Cc: X. Zhou > > Subject: RE: cache coherency problem > > > > > Chris, > > > > > Thanks for including me in the descussion. > > > > > I have got one doubt, > > > > > If the codec is xDAIS-xDM compliant, do we need to the cache > > ivalidation/flush with the process() call? > > > > > > > > Regards, > > ~Santhosh > > > > > > > > > > > On Mon, 2006-11-27 at 08:02 -0600, Ring, Chris wrote: > > > Ok, Santhosh. > > > > > > > Zhou, I'm interested in helping you resolve this. Did my > last mail > > > help? Which codec are you using (is it one of TI's or one of your > > > > own?)? Could it be either 1) buffer mis-alignment (i.e. > > not on 128-bit > > > boundary) or 2) the codec writing into an input buffer? > > > > > > > Can either of you reproduce the issue in one of the simple > > copy-codecs? > > > > > > > Chris > > > > > > > > -----Original Message----- > > > > From: Santhosh Kumar B N [mailto:[EMAIL PROTECTED] > > > > Sent: Sunday, November 26, 2006 11:07 PM > > > > To: X. Zhou > > > > Cc: Ring, Chris > > > > Subject: Re: cache coherency problem > > > > > > > > > Hi Zhou/Chris, > > > > > > > > > I am also looking at this issue, If you guys have any further > > > > > discussion on this topic please copy to me also. > > > > > > > > > With warm regards, > > > > Santhosh > > > > > > > > > > > > > > > > > > > On Fri, 2006-11-24 at 13:27 +0800, X. Zhou wrote: > > > > > Hi, > > > > > This nasty problem got me mad!! > > > > > > > > > > > I am now using DVEM6446 board and DVSDK enviroment to > > develop a ARM > > > > > client + DSP video decoder application. > > > > > > > > > > > Now i found that sometimes the bitstream buffer which > > is transferred > > > > > by ARM to DSP exists dirty data case. > > > > > > > > > > > Detail information about this case were given here: > > > > > > > > > > > (1) the bitstream buffer was pre-allocated at ARM side, via > > > > > Memory_contigAlloc() function; > > > > > > > > > > > [ in my opinion, Memory_contigAlloc() should provide > > > > buffers which > > > > > were not only aligned, > > > > > > but also non-cached and physically contiguous. > > > > > > > > > > > Isn't it right?? > > > > > > ] > > > > > > > > > > > > (2) each time when i call the VIDDEC_process() interface at > > > > ARM side, > > > > > i pass the bitstream buffer > > > > > > pointer via the "XDM_BufDesc inBufs" parameter. e.g.: > > > > > > > > > > > streamBuf = Memory_contigAlloc(800000, -1); > > > > > > > > > > > ................ > > > > > > > > > > > actualStreamSize = fread(streamBuf, 1, 800000, fp); > > > > > > > > > > > ................ > > > > > > > > > > > inBufs.numBufs = 1; > > > > > inBufAddr[0] = streamBuf; > > > > > inBufSize[0] = actualStreamSize; > > > > > inBufs.bufs[0] = inBufAddr; > > > > > inBufs.bufSizes[0] = inBufSize; > > > > > > > > > > > printf("ARM : 0x%x 0x%x 0x%x 0x%x 0x%x 0x%x\n", > > > > > *((unsigned char *)inBufAddr[0]+0), > > > > > *((unsigned char *)inBufAddr[0]+1), > > > > > *((unsigned char *)inBufAddr[0]+2), > > > > > *((unsigned char *)inBufAddr[0]+3), > > > > > *((unsigned char *)inBufAddr[0]+4), > > > > > *((unsigned char *)inBufAddr[0]+5)); > > > > > > > > > > > /* Call the process function to decode the nalu > buffers */ > > > > > status = VIDDEC_process(hdecode, > > > > > &inBufs, > > > > > &outBufs, > > > > > (IVIDDEC_InArgs > *)(&decoder_inargs), > > > > > (IVIDDEC_OutArgs > > *)(&decoder_outargs)); > > > > > > > > > > > ................ > > > > > > > > > > > (3) each time when the dsp-side IVIDDEC_process() function > > > > is called, > > > > > i invaliate the cache related to > > > > > > bitstream buffer. e.g: > > > > > > > > > > > XDAS_Int32 IVIDDEC_process(IVIDDEC_Handle h, > > > > XDM_BufDesc *inBufs, > > > > > XDM_BufDesc *outBufs, > > > > > > IVIDDEC_InArgs *inArgs, > > > > IVIDDEC_OutArgs > > > > > *outArgs) > > > > > { > > > > > int i; > > > > > for ( i = 0; i < inBufs->numBufs; i ++ ) > > > > > { > > > > > GT_6trace(curDecTrace, GT_ENTER, "DSP : before > > > > invalid: 0x > > > > > %x 0x%x 0x%x 0x%x 0x%x 0x%x\n", > > > > > *((unsigned char *)inBufs->bufs[i]+0), > > > > > *((unsigned char *)inBufs->bufs[i]+1), > > > > > *((unsigned char *)inBufs->bufs[i]+2), > > > > > *((unsigned char *)inBufs->bufs[i]+3), > > > > > *((unsigned char *)inBufs->bufs[i]+4), > > > > > *((unsigned char *)inBufs->bufs[i]+5)); > > > > > > > > > > > BCACHE_inv(inBufs->bufs[i], inBufs->bufSizes[i], > > > > > TRUE); //invaildate cache > > > > > > > > > > > > GT_6trace(curDecTrace, GT_ENTER, "DSP : after > > > > invalid: 0x% > > > > > x 0x%x 0x%x 0x%x 0x%x 0x%x\n", > > > > > *((unsigned char *)inBufs->bufs[i]+0), > > > > > *((unsigned char *)inBufs->bufs[i]+1), > > > > > *((unsigned char *)inBufs->bufs[i]+2), > > > > > *((unsigned char *)inBufs->bufs[i]+3), > > > > > *((unsigned char *)inBufs->bufs[i]+4), > > > > > *((unsigned char *)inBufs->bufs[i]+5)); > > > > > } > > > > > > > > > > > ............. > > > > > > > > > > > decode_one_frame(inBufs, .....); > > > > > > > > > > > ............ > > > > > } > > > > > > > > > > > > > > > > > > While, the experiment results show that sometimes the > > data readden > > > > > by DSP are inconsistent with > > > > > > the datat written by ARM, and if inconstistent case > > > > exists, it seems > > > > > that the data readden by dsp > > > > > > in this time is the same data with the data written > > by ARM last > > > > > time, > > > > > > > > > > > How to fix this problem? > > > > > > > > > > > Is the cache on ARM side unflushed (if > > > > Memory_contigAlloc() provides > > > > > buffers with cachalility) ? > > > > > or is the cache on DSP side ninvalidated unsucessfully (if > > > > > BCACHE_inv() is a void function )? > > > > > > > > > > > I got mad! Help me, please?!!! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This message (including any attachments) is for the named > > > > > addressee(s)'s use only. It may contain sensitive, > confidential, > > > > > > private proprietary or legally > > privileged > > > > > information intended for a > > > > > specific individual and purpose, and is protected by > > law. If you are > > > > > not the intended recipient, > > > > > please immediately delete it and all copies of it from > > your system, > > > > > destroy any hard copies of it > > > > > and notify the sender. Any use, disclosure, copying, or > > distribution > > > > > of this message and/or any > > > > > attachments is strictly prohibited. > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Davinci-linux-open-source mailing list > > > > > [email protected] > > > > > > > > > > > > > > http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source > > > > > > > > > > > > > > > > > > > > > > > > > > This message (including any attachments) is for the named > addressee(s)'s use only. It may contain > sensitive, confidential, private proprietary or legally > privileged information intended for a > specific individual and purpose, and is protected by law. If > you are not the intended recipient, > please immediately delete it and all copies of it from your > system, destroy any hard copies of it > and notify the sender. Any use, disclosure, copying, or > distribution of this message and/or any > attachments is strictly prohibited. > > > _______________________________________________ Davinci-linux-open-source mailing list [email protected] http://linux.davincidsp.com/mailman/listinfo/davinci-linux-open-source
