Hi Lincoln, Glad to see this thread rebooted. Here are some comments on the metadata search API proposal. I would be glad to see feedback from others too.
Data Types The proposal suggests each attribute has one of 3 types: string, numeric, date. How do we deduce the type for user metadata attributes i.e. object_meta_<name> attributes etc.? This is important for range searches since types such as dates are not sorted correctly when represented as strings. This seems to warrant a configuration file which contains schema information. In addition, using attribute naming conventions could help set the default behavior e.g. object_meta_start_date could refer to a date by default since it has the suffix _date. Also, additional types which seem important are a floating point type and a type for representing geo-spatial coordinates and/or general spatial information. Ideally the set of types which can be handled should be extensible. System Metadata Regarding system metadata, many attributes are part of this proposal which are not currently part of Swift. There should be a minimal set of attributes - probably those attributes which are part of Swift - which should be available for indexing by all vendor implementations. Other than that, different vendors would have different requirements for generating new system metadata for indexing, and it probably shouldn't be a mandatory part of the API. Also, in general, the user should be able to control which user and system metadata attributes are indexed in order to control the amount of resources invested in metadata indexing/search. Let me know what you think. Best regards Paula Paula Ta-Shma, Ph.D. Storage Research IBM Research - Haifa Phone: +972.3.7689402 Email: [email protected] From: "Thomas, Lincoln (HP Storage R&D)" <[email protected]> To: "[email protected]" <[email protected]>, Date: 11/01/2014 04:09 AM Subject: [openstack-dev] [Swift] Metadata Search API Rebooting this thread now that I've reorg'd the Wiki page. The proposed REST API spec for searching system and custom metadata in Swift, across accounts, containers, and objects, is now posted at: https://wiki.openstack.org/wiki/MetadataSearchAPI I've also made the first modification to the API since the Icehouse design summit where I introduced this project: adding "metadata:" per conversation with Paula Ta-Shma's team at IBM Storage Research. The home page for this project remains at: https://wiki.openstack.org/wiki/MetadataSearch See that home page for further details, and the history of this email thread. Feel free to edit the Wiki as described on the home page! As Brian Cline (SoftLayer) said so eloquently in this thread: > Today, about the best one can do is iterate through everything and inspect metadata > along the way - obviously an infinitely expensive (and hilariously insane) > operation. > If there are any others who have implemented search in Swift, please speak up and > help shape this. We both want to get community consensus on a standard search API, > then get a pluggable reference implementation into Swift. > This is all work-in-progress stuff, but we'd welcome any feedback, concerns, > literal jumps for joy, etc. in this thread, both on the API and on a reference > architecture. Thanks in advance, Lincoln Thomas (IRC lincolnt) System/Software Engineer, HP Storage R&D Portland, OR, USA, +1 (503) 757-6274 _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
