The conclusion from last week's call was the user meta data is required and persistence is optional on a per-platform basis.
I think we've become a bit too focused on implementation questions rather than taking an application-centric view. The consensus was that it's better to ask an implementation do to something (within reason) that an application would otherwise have to do itself because that means that applications can be more broadly portable and implementations can potentially do things more efficiently since they know their platforms best. So in the case of user meta data we don't care *how *an implementation chooses to implement this feature. The user API is odp_buffer_udata() that returns a pointer to the user meta data associated with the buffer (as well as the size of that area). There's no requirement (or assumption) that this area is contiguous with any other data associated with the buffer. If the implementation can store the application-required number of bytes as part of the main buffer structures then great. If it can't, then the implementation is expected to allocate a block of memory and store a pointer to it (that it will return in response to an odp_buffer_udata() call) and store that address in manner that can be efficiently derived from the supplied odp_buffer_t. How it does that is transparent to the application and entirely under the implementation's control. The document offers several suggestions for how this can be done, but implementations are under no obligation to choose any of these since they know what works best for their platform. That's all that's being said here. Where not saying what we require the HW to do, only the semantics that the implementation must provide. The implementation is some combination of HW and SW that will vary from platform to platform and presumably will itself evolve over time. It is understood that the performance of certain APIs will vary from platform to platform, or based on the specific features of the APIs used. These should be documented so that programmers can take that into consideration in designing their applications and selecting platforms to best match their needs. The goal of ODP is not to require complete uniformity in performance (that's clearly impossible) but rather to provide a portability baseline for applications and a functional target for implementations that permits both to add value in their respective areas. On Sun, Aug 31, 2014 at 10:47 AM, Wiles, Roger Keith < [email protected]> wrote: > Good write up, comments inline. > > Make sure you read to the bottom of my email as I do switch gears toward > the bottom :-) > > On Aug 31, 2014, at 2:10 AM, Gilad Ben Yossef <[email protected]> wrote: > > > Hi, > > > > Regarding the discussion about user meta data which was brought on > during the conf, call, I thought I would try my hand at a short concise > summary: > > > > - It is often useful to carry application specific meta data which is > tied to a packet. > > > > - This application specific meta data is of application specific size > > > > - The application specific meta data life cycle is that of packet buffer > allocation/release. > > > > - Because application meta data size is not known before hand, some > platform will require a memory allocation for its storage space > > > > - On such platforms, it is useful to pre-allocate a per packet meta data > memory and store a handle or pointer to it in each packet buffer at packet > buffer creation. This has been referred to as packet buffer persistent meta > data. > > > > - Such packet buffer persistent meta data life cycle is from that of > packet buffer *pool* allocation/release. > > > > - As far as we can tell, storing the handle of packet meta data is the > only use for packet buffer persistent meta data. > > > > > > > > The current suggestion to handle this is, if I understand it correctly: > > > > - Have an API for requesting packet buffer persistent meta data at pool > allocation time. > > > > - This API will be optional – so some platform may not implement it. > > > > - For those that will implement it, the packet buffer meta data > (persistent or otherwise) will be of a variable platform specific size but > no less than 8 bytes. > > > > > > > > I find the current suggestion very sub optimal since it does not take > into consideration the needs of ODP application developers. Consider the > following: > > > > I am an application creator. I require some meta data of size X and I > chose ODP specifically to cover as many platforms as possible. What are my > options? > > > > The platform I use may support meta data allocation of user meta data of > the size I need. > > > > If not, the platform I use may still support allocation of persistent > meta data so I can allocate a buffer for per packet user meta data and > store that in the persistent meta data. > > > > If not, I need to implement my meta data in the form of a meta data > "header" I add to packet data. > > > > Now, as application developer I either need to implement all 3 options > OR implement fewer and be sub-optimal on some platforms or not be able to > run on some.I believe this is bad for the application writer and the ODP > eco system. > > > > Also note, that since the application writer ALWAYS has the option to > allocate a buffer for its meta data and stuffing a pointer/handle to said > buffer as a "meta header" in the packet, so does a platform ALWAYS an > options to implement packet buffer meta data (just do what the application > writer would have done… > > > > Therefore, I wish to suggest a different approach: > > > > 1. Application meta data will only be available to > ODP_BUFFER_TYPE_PACKET buffers. > > I agree only a specific type of buffer can have meta data. > > > > 2. No persistent packet meta data will be supported in the API. > > I disagree. If we are going to have persistent meta-data we need at least > 8 bytes of persistent meta data. Fixing the persistent meta data size to 8 > bytes is reasonable for all of the platforms IMO. > > Let say the developer needs meta data, the platform will provide 8 bytes > and it is persistent. The developer can use this pointer or 8 bytes to > point to non-persistent meta data or persistent meta data or both. Now the > platform must only provide 8 bytes of persistent data space. > > If the platform is able to provide say 64 bytes total of meta data then it > can document how the developer can access that data. Lets say the platform > can provide 64 bytes of meta-data, it could just place a pointer to the > meta-data in the 8 byte persistent meta-data (which could be the first 8 > bytes of the 64 bytes) and the developer can reference it via the pointer. > > Meta-data > +—————————————+ > | Persistent 8 bytes | —> points to persistent data. > +—————————————+ > | | <— Non-Persistent data could > be non-contiguous > | platform specific | > | size non-persistent | > | could be zero | > | | > +—————————————+ <— Cache line size or multiple of a cache line seems > reasonable. > > If the non-persistent meta-data is non-contiguous then a pointer can be > placed in the persistent data or a new API can help access the extra data > based on the persistent data, which is platform specific. > > > > 3. Application developer wishing to make use of packet meta data > will specify how much user meta data they require at pool creation time, > with a size limitation equal to the size limitation of the biggest buffer > pool supported by the platform and a platform MAY perform needed > initialization and allocation at this point. > > > > > 4. Access to packet meta data will be done via an API, and a > platform MAY perform allocation and initialization at first access time. > > > > 5. A generic implementation for allocating packet meta data > buffers from an internal buffer pool and storing a pointer to said buffer > as a "meta header" in the packet will provided as a helper function and any > platform that does not provide a custom implementation MUST use this > generic helper implementation as a default one. > > > > 6. A platform that is able to provide a better option to support > this API, SHOULD do so. > > > > This helps ODP application writes because they now have a single way to > ask for meta data and they trust the platform implementers to do the most > efficient thing for their platform – as it should be. Also, any improvement > in a platform implementation will help ALL ODP programs on that platform. > > > > What do you think? > > If we have persistent meta data then we can also have non-persistent meta > data by using the persistent data to reference the non-persistent data. If > we require all platforms to provide meta data and it must be persistent, > plus that persistent meta-data size is fixed at 8 bytes. Now the platform > only needs to provide one type of meta-data. > > The persistent meta data can point to or reference non-persistent meta > data. The platform can provide a simple allocation routine to provide the > non-persistent meta data with any size restrictions it needs. A ODP > supplied default allocation routine will just allocate memory from the heap > with some max size restriction (say 1K or page size). > > —————————————— > Now I am going to switch tracks here and suggest we do not require > persistent meta data for any platform. It appears to me that persistent > meta data is causing the biggest problem in defining meta data. We all > agree we need some meta data and persistent meta data is gumming up the > works. > > If we state a platform must provide meta data and the size of the meta > data is platform specific, but no less then 8 bytes (or possible a cache > line is the right size) and we do not care about the max size as it is > platform specific other then a multiple of a cache line. > > I would suggest we do place a reasonable max size. It could be the page > size of the system 2K, 4K or 8K. A 4K size seems very reasonable to me as a > 2K system can just allocate two pages. Do we have a lot of 8K page systems? > Trying to keep the meta data to a cache line size or page size as accessing > the same cache line from two CPUs is going to be a problem. > > I know I dropped the persistent meta data as I believe I was the one that > suggested it :-) but we need to move forward and I am hoping dropping the > persistent meta data will help use do that. > > > > Gilad Ben-Yossef > > Software Architect > > EZchip Technologies Ltd. > > 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel > > Tel: +972-4-959-6666 ext. 576, Fax: +972-8-681-1483 > > Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388 > > Email: [email protected], Web: http://www.ezchip.com > > > > > > "Ethernet always wins." > > > > — Andy Bechtolsheim > > > > > > > > _______________________________________________ > > lng-odp mailing list > > [email protected] > > http://lists.linaro.org/mailman/listinfo/lng-odp > > Keith Wiles, Principal Technologist with CTO office, Wind River mobile > 972-213-5533 > > > _______________________________________________ > lng-odp mailing list > [email protected] > http://lists.linaro.org/mailman/listinfo/lng-odp >
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
