Good write up, comments inline. Make sure you read to the bottom of my email as I do switch gears toward the bottom :-)
On Aug 31, 2014, at 2:10 AM, Gilad Ben Yossef <[email protected]> wrote: > Hi, > > Regarding the discussion about user meta data which was brought on during the > conf, call, I thought I would try my hand at a short concise summary: > > - It is often useful to carry application specific meta data which is tied to > a packet. > > - This application specific meta data is of application specific size > > - The application specific meta data life cycle is that of packet buffer > allocation/release. > > - Because application meta data size is not known before hand, some platform > will require a memory allocation for its storage space > > - On such platforms, it is useful to pre-allocate a per packet meta data > memory and store a handle or pointer to it in each packet buffer at packet > buffer creation. This has been referred to as packet buffer persistent meta > data. > > - Such packet buffer persistent meta data life cycle is from that of packet > buffer *pool* allocation/release. > > - As far as we can tell, storing the handle of packet meta data is the only > use for packet buffer persistent meta data. > > > > The current suggestion to handle this is, if I understand it correctly: > > - Have an API for requesting packet buffer persistent meta data at pool > allocation time. > > - This API will be optional – so some platform may not implement it. > > - For those that will implement it, the packet buffer meta data (persistent > or otherwise) will be of a variable platform specific size but no less than 8 > bytes. > > > > I find the current suggestion very sub optimal since it does not take into > consideration the needs of ODP application developers. Consider the following: > > I am an application creator. I require some meta data of size X and I chose > ODP specifically to cover as many platforms as possible. What are my options? > > The platform I use may support meta data allocation of user meta data of the > size I need. > > If not, the platform I use may still support allocation of persistent meta > data so I can allocate a buffer for per packet user meta data and store that > in the persistent meta data. > > If not, I need to implement my meta data in the form of a meta data "header" > I add to packet data. > > Now, as application developer I either need to implement all 3 options OR > implement fewer and be sub-optimal on some platforms or not be able to run on > some.I believe this is bad for the application writer and the ODP eco system. > > Also note, that since the application writer ALWAYS has the option to > allocate a buffer for its meta data and stuffing a pointer/handle to said > buffer as a "meta header" in the packet, so does a platform ALWAYS an options > to implement packet buffer meta data (just do what the application writer > would have done… > > Therefore, I wish to suggest a different approach: > > 1. Application meta data will only be available to > ODP_BUFFER_TYPE_PACKET buffers. I agree only a specific type of buffer can have meta data. > > 2. No persistent packet meta data will be supported in the API. I disagree. If we are going to have persistent meta-data we need at least 8 bytes of persistent meta data. Fixing the persistent meta data size to 8 bytes is reasonable for all of the platforms IMO. Let say the developer needs meta data, the platform will provide 8 bytes and it is persistent. The developer can use this pointer or 8 bytes to point to non-persistent meta data or persistent meta data or both. Now the platform must only provide 8 bytes of persistent data space. If the platform is able to provide say 64 bytes total of meta data then it can document how the developer can access that data. Lets say the platform can provide 64 bytes of meta-data, it could just place a pointer to the meta-data in the 8 byte persistent meta-data (which could be the first 8 bytes of the 64 bytes) and the developer can reference it via the pointer. Meta-data +—————————————+ | Persistent 8 bytes | —> points to persistent data. +—————————————+ | | <— Non-Persistent data could be non-contiguous | platform specific | | size non-persistent | | could be zero | | | +—————————————+ <— Cache line size or multiple of a cache line seems reasonable. If the non-persistent meta-data is non-contiguous then a pointer can be placed in the persistent data or a new API can help access the extra data based on the persistent data, which is platform specific. > > 3. Application developer wishing to make use of packet meta data will > specify how much user meta data they require at pool creation time, with a > size limitation equal to the size limitation of the biggest buffer pool > supported by the platform and a platform MAY perform needed initialization > and allocation at this point. > > 4. Access to packet meta data will be done via an API, and a platform > MAY perform allocation and initialization at first access time. > > 5. A generic implementation for allocating packet meta data buffers > from an internal buffer pool and storing a pointer to said buffer as a "meta > header" in the packet will provided as a helper function and any platform > that does not provide a custom implementation MUST use this generic helper > implementation as a default one. > > 6. A platform that is able to provide a better option to support this > API, SHOULD do so. > > This helps ODP application writes because they now have a single way to ask > for meta data and they trust the platform implementers to do the most > efficient thing for their platform – as it should be. Also, any improvement > in a platform implementation will help ALL ODP programs on that platform. > > What do you think? If we have persistent meta data then we can also have non-persistent meta data by using the persistent data to reference the non-persistent data. If we require all platforms to provide meta data and it must be persistent, plus that persistent meta-data size is fixed at 8 bytes. Now the platform only needs to provide one type of meta-data. The persistent meta data can point to or reference non-persistent meta data. The platform can provide a simple allocation routine to provide the non-persistent meta data with any size restrictions it needs. A ODP supplied default allocation routine will just allocate memory from the heap with some max size restriction (say 1K or page size). —————————————— Now I am going to switch tracks here and suggest we do not require persistent meta data for any platform. It appears to me that persistent meta data is causing the biggest problem in defining meta data. We all agree we need some meta data and persistent meta data is gumming up the works. If we state a platform must provide meta data and the size of the meta data is platform specific, but no less then 8 bytes (or possible a cache line is the right size) and we do not care about the max size as it is platform specific other then a multiple of a cache line. I would suggest we do place a reasonable max size. It could be the page size of the system 2K, 4K or 8K. A 4K size seems very reasonable to me as a 2K system can just allocate two pages. Do we have a lot of 8K page systems? Trying to keep the meta data to a cache line size or page size as accessing the same cache line from two CPUs is going to be a problem. I know I dropped the persistent meta data as I believe I was the one that suggested it :-) but we need to move forward and I am hoping dropping the persistent meta data will help use do that. > > Gilad Ben-Yossef > Software Architect > EZchip Technologies Ltd. > 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel > Tel: +972-4-959-6666 ext. 576, Fax: +972-8-681-1483 > Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388 > Email: [email protected], Web: http://www.ezchip.com > > > "Ethernet always wins." > > — Andy Bechtolsheim > > > > _______________________________________________ > lng-odp mailing list > [email protected] > http://lists.linaro.org/mailman/listinfo/lng-odp Keith Wiles, Principal Technologist with CTO office, Wind River mobile 972-213-5533 _______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
