drin commented on issue #34451: URL: https://github.com/apache/arrow/issues/34451#issuecomment-1456970474
In some work I'm doing I plan on putting metadata keys that are system-specific. It seems that part of the requirement here would be a way of "namespacing" metadata attributes for cooperative system design. I would also think that, at minimum, the following namespaces would always potentially co-exist: 1. acero (or arrow, generally) 2. substrait 3. "the system" (top-level system managing arrow data) 4. _some_ application (whatever is interacting with the data stored in an arrow object) With the following possible caveats: * (3) and (4) could be the same (maybe some large application such as duckdb that doesn't share the data with any other system). * (2) and (4) could be the same (maybe an application funnels all intent through a substrait producer). * If acero is never actively used, then conflicts with its namespace are moot (though prone to refactoring if acero is every adopted in the future) Not sure if there has been any other proposal of metadata management in arrow that should be leveraged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
