Hi Fiona,

> Hi Shally, Ahmed, and anyone else interested in compressdev,
> I mentioned last week that we've been exploring using something other than 
> mbufs to pass src/dst buffers to compressdev PMDs.
> Reasons:
>  - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for 
> compression
>     can be greater and it would add cycles to have to break up into smaller 
> segments.
>  - data may originate in mbufs, but is more likely, particularly for storage 
> use-cases,  to
>     originate in other data structures.
>  - There's a 2 cache-line overhead for every segment in a chain, most of this 
> data
>     is network-related, not needed by compressdev
> So moving to a custom structure would minimise memory overhead, remove 
> restriction on 64k-1 size and give more flexibility if
> compressdev ever needs any comp-specific meta-data.
> We've come up with a compressdev-specific structure using the struct iovec 
> from sys/uio.h, which is commonly used by storage
> applications. This would replace the src and dest mbufs in the  op.
> I'll not include the code here - Pablo will push that to github shortly and 
> we'd appreciate review comments there.
> https://github.com/pablodelara/dpdk-draft-compressdev
> Just posting on the mailing list to give a heads-up and ensure this reaches a 
> wider audience than may see it on github.
> Note : We also considered having no data structures in the op, instead the 
> application
> would supply a callback which the PMD would use to retrieve meta-data (virt 
> address, iova, length)
> for each next segment as needed. While this is quite flexible and allow the 
> application
> to keep its data in its native structures, it's likely to cost more cycles.

As I said in different thread it will not only slowdown things, but will make 
it difficult
(if possible at all) for compressdev PMDs to support DPDK MP model.

> So we're not proposing this at the moment, but hope to benchmark it later 
> while the API is still experimental.
> General feedback on direction is welcome here on the mailing list.
> For feedback on the details of implementation we would appreciate comments on 
> github.
> Regards,
> Fiona.

Reply via email to