Flyangz commented on code in PR #1903:
URL: https://github.com/apache/auron/pull/1903#discussion_r2715807490
##########
native-engine/datafusion-ext-functions/src/lib.rs:
##########
@@ -39,51 +39,57 @@ pub fn create_auron_ext_function(
name: &str,
spark_partition_id: usize,
) -> Result<ScalarFunctionImplementation> {
+ macro_rules! cache {
+ ($func:path) => {{
+ static CELL: OnceLock<ScalarFunctionImplementation> =
OnceLock::new();
+ CELL.get_or_init(|| Arc::new($func)).clone()
+ }};
+ }
Review Comment:
In rust, statics inside functions or blocks are not global singletons
sharing the same name; they are local singletons unique to that specific scope
instantiation. In this case, it is to that specific matching arm.
Just running this test in `AuronQuerySuite`
```
test("my cache test") {
withTable("my_cache_table") {
sql("""
|create table my_cache_table using parquet as
|select col1, col2 from values ('a,A', '{"a":"1", "b":"2"}'),
('b,B', '{"a":"3", "b":"4"}'), ('c,C', '{"a":"5", "b":"6"}')
|""".stripMargin)
sql("""
|select split(col1, ',')[0],
| split(col1, ',')[1],
| get_json_object(col2, '$.a'),
| get_json_object(col2, '$.b')
|from my_cache_table
|""".stripMargin).show()
}
}
```
we can see the following correct answer.
```
+---------------------+---------------------+--------------------------+--------------------------+
|split(col1, ,, -1)[0]|split(col1, ,, -1)[1]|get_json_object(col2,
$.a)|get_json_object(col2, $.b)|
+---------------------+---------------------+--------------------------+--------------------------+
| a| A| 1|
2|
| b| B| 3|
4|
| c| C| 5|
6|
+---------------------+---------------------+--------------------------+--------------------------+
```
It can handle different ext function `StringSplit`, `GetParsedJsonObject`
and `ParseJson`.
```
ProjectExec [
(spark_ext_function_Spark_StringSplit(#2@0, ,)).[1] AS #16,
(spark_ext_function_Spark_StringSplit(#2@0, ,)).[2] AS #17,
spark_ext_function_Spark_GetParsedJsonObject(spark_ext_function_Spark_ParseJson(#3@1),
$.a) AS #18,
spark_ext_function_Spark_GetParsedJsonObject(spark_ext_function_Spark_ParseJson(#3@1),
$.b) AS #19
], schema=[#16:Utf8;N, #17:Utf8;N, #18:Utf8;N, #19:Utf8;N]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
