>-----Original Message----- >From: Ahmed Mansour [mailto:ahmed.mans...@nxp.com] >Sent: 14 February 2018 22:25 >To: Verma, Shally <shally.ve...@cavium.com>; Trahe, Fiona ><fiona.tr...@intel.com>; dev@dpdk.org >Cc: Athreya, Narayana Prasad <narayanaprasad.athr...@cavium.com>; Gupta, >Ashish <ashish.gu...@cavium.com>; Sahu, Sunila ><sunila.s...@cavium.com>; De Lara Guarch, Pablo ><pablo.de.lara.gua...@intel.com>; Challa, Mahipal ><mahipal.cha...@cavium.com>; Jain, Deepak K <deepak.k.j...@intel.com>; Hemant >Agrawal <hemant.agra...@nxp.com>; Roy >Pledge <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com> >Subject: Re: [RFC v2] doc compression API for DPDK > >On 2/14/2018 12:41 AM, Verma, Shally wrote: >> Hi Ahmed >> >>> -----Original Message----- >>> From: Ahmed Mansour [mailto:ahmed.mans...@nxp.com] >>> Sent: 02 February 2018 02:20 >>> To: Trahe, Fiona <fiona.tr...@intel.com>; Verma, Shally >>> <shally.ve...@cavium.com>; dev@dpdk.org >>> Cc: Athreya, Narayana Prasad <narayanaprasad.athr...@cavium.com>; Gupta, >>> Ashish <ashish.gu...@cavium.com>; Sahu, Sunila >>> <sunila.s...@cavium.com>; De Lara Guarch, Pablo >>> <pablo.de.lara.gua...@intel.com>; Challa, Mahipal >>> <mahipal.cha...@cavium.com>; Jain, Deepak K <deepak.k.j...@intel.com>; >>> Hemant Agrawal <hemant.agra...@nxp.com>; Roy >>> Pledge <roy.ple...@nxp.com>; Youri Querry <youri.querr...@nxp.com> >>> Subject: Re: [RFC v2] doc compression API for DPDK >>> >>>>>> [Fiona] I propose if BFINAL bit is detected before end of input >>>>>> the decompression should stop. In this case consumed will be < >>>>>> src.length. >>>>>> produced will be < dst buffer size. Do we need an extra STATUS response? >>>>>> STATUS_BFINAL_DETECTED ? >>>>> [Shally] @fiona, I assume you mean here decompressor stop after >>>>> processing Final block right? >>>> [Fiona] Yes. >>>> >>>> And if yes, >>>>> and if it can process that final block successfully/unsuccessfully, then >>>>> status could simply be >>>>> SUCCESS/FAILED. >>>>> I don't see need of specific return code for this use case. Just to >>>>> share, in past, we have practically run into >>>>> such cases with boost lib, and decompressor has simply worked this way. >>>> [Fiona] I'm ok with this. >>>> >>>>>> Only thing I don't like this is it can impact on performance, as normally >>>>>> we can just look for STATUS == SUCCESS. Anything else should be an >>>>>> exception. >>>>>> Now the application would have to check for SUCCESS || BFINAL_DETECTED >>>>>> every time. >>>>>> Do you have a suggestion on how we should handle this? >>>>>> >>> [Ahmed] This makes sense. So in all cases the PMD should assume that it >>> should stop as soon as a BFINAL is observed. >>> >>> A question. What happens ins stateful vs stateless modes when >>> decompressing an op that encompasses multiple BFINALs. I assume the >>> caller in that case will use the consumed=x bytes to find out how far in >>> to the input is the end of the first stream and start from the next >>> byte. Is this correct? >> [Shally] As per my understanding, each op can be tied up to only one stream >> as we have only one stream pointer per op and one >stream can have only one BFINAL (as stream is one complete compressed data) >but looks like you're suggesting a case where one op >can carry multiple independent streams? and thus multiple BFINAL?! , such as, >below here is op pointing to more than one streams >> >> -------------------------------------------- >> op --> |stream1|stream2| |stream3| >> -------------------------------------------- >> >> Could you confirm if I understand your question correct? >[Ahmed] Correct. We found that in some storage applications the user >does not know where exactly the BFINAL is. They rely on zlib software >today. zlib.net software halts at the first BFINAL. Users put multiple >streams in one op and rely on zlib to stop and inform them of the end >location of the first stream.
[Shally] Then this is practically case possible on decompressor and decompressor doesn't regard flush flag. So in that case, I expect PMD to internally reset themselves (say in case of zlib going through cycle of deflateEnd and deflateInit or deflateReset) and return with status = SUCCESS with updated produced and consumed. Now in such case, if previous stream also has some footer followed by start of next stream, then I am not sure how PMD / lib can support that case. Have you had practically run of such use-case on zlib? If yes, how then such application handle it in your experience? I can imagine for such input zlib would return with Z_FLUSH_END after 1st BFINAL is processed to the user. Then application doing deflateReset() or Init-End() cycle before starting with next. But if it starts with input that doesn't have valid zlib header, then likely it will throw an error. >> >> Thanks >> Shally >>
