goldmedal commented on issue #11268: URL: https://github.com/apache/datafusion/issues/11268#issuecomment-2211228822
> I think following the model for `named_struct` makes the most sense here > > From my reading of https://docs.rs/arrow/latest/arrow/array/struct.MapArray.html the idea is that each element of a `MapArray` is itself a map with some arbitrary number of key/value pairs > > I think we could use https://docs.rs/arrow/latest/arrow/array/builder/struct.MapBuilder.html to create the actual array/value > > ## Syntax Proposal > Maybe we could use a function like `make_map` to follow the naming convention of `make_array`: https://datafusion.apache.org/user-guide/sql/scalar_functions.html#make-array > > ```sql > -- create {'foo': 1, 'bar': 2} > select make_map('foo', 1, 'bar', 2); > ``` > I have finished [a draft function](https://github.com/goldmedal/datafusion/blob/583b14a3492ce822a008b792b57f32104561dc37/datafusion/functions/src/core/map.rs#L10) that followed my original design yesterday. I follow how arrow-rs to [create a map from strings](https://github.com/apache/arrow-rs/blob/bed37466af718be3deff00f1b21c7b38d37eb8a5/arrow-array/src/array/map_array.rs#L308). It could be used like DuckDB syntax that accepts two lists to construct a map. (In my design, it could accept many key-value array pairs) ```sql SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30]); ``` It may not fit the syntax you propose. I will follow the `named_struct` model to implement another one. The reason I didn't use `MapBuilder` is that I can't find a smart way to get the corresponding ArrayBuilder for the arguments ( MapBuilder requires two initial builders for the key and value ). If we have some existing solutions for this issue, I think using `MapBuilder` is better. > ## Implementation suggestion > I recommend doing this in 2 PRs: > > 1. Implement test the function (e.g. `make_map`) > 2. Implement the rewrite from the map literal to `make_map` > I agree with you. I will implement the function in this issue and have a follow-up one for the map literal. > ## Notes / caveats > I am not sure how complete the MapArray support in arrow-rs is, so it wouldn't surprise me if we hit various unimplemented errors when we tried to do basic operations like compare maps or sort them. I don't think that is a reason not to proceed here, but I wanted to point it out Thanks for reminding this problem. I think if I occurred this kind of problem, I might implement the required functions on the DataFusion side first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
