This is an automated email from the ASF dual-hosted git repository.
liuneng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-gluten.git
The following commit(s) were added to refs/heads/main by this push:
new 2be6c86be [GLUTEN-6262][CH]Json input format ignore key case #6263
2be6c86be is described below
commit 2be6c86bef0a5de834ec4a88276787cfcd453aed
Author: kevinyhzou <[email protected]>
AuthorDate: Mon Jul 8 15:04:23 2024 +0800
[GLUTEN-6262][CH]Json input format ignore key case #6263
What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: #6262)
this pr is depend on clickhouse pr: ClickHouse/ClickHouse#61750
How was this patch tested?
test by ut
---
backends-clickhouse/src/test/resources/text-data/json-settings/data.txt | 1 +
cpp-ch/local-engine/Common/CHUtil.cpp | 1 +
2 files changed, 2 insertions(+)
diff --git
a/backends-clickhouse/src/test/resources/text-data/json-settings/data.txt
b/backends-clickhouse/src/test/resources/text-data/json-settings/data.txt
index 0541ce346..230b46ec3 100644
--- a/backends-clickhouse/src/test/resources/text-data/json-settings/data.txt
+++ b/backends-clickhouse/src/test/resources/text-data/json-settings/data.txt
@@ -1,4 +1,5 @@
{"a":1,"b":2,"c":3}
+{"a":"a5", "B":"b6", "c":7}
{"a":"4"}
{"t":{"ta":"cc","tb":100,"tc":1.234}}
{"t":{"ta":"cc","tb":100,"td":"ignore"}}
diff --git a/cpp-ch/local-engine/Common/CHUtil.cpp
b/cpp-ch/local-engine/Common/CHUtil.cpp
index 770fbbc59..800039f1d 100644
--- a/cpp-ch/local-engine/Common/CHUtil.cpp
+++ b/cpp-ch/local-engine/Common/CHUtil.cpp
@@ -747,6 +747,7 @@ void
BackendInitializerUtil::initSettings(std::map<std::string, std::string> & b
settings.set("input_format_parquet_import_nested", true);
settings.set("input_format_json_read_numbers_as_strings", true);
settings.set("input_format_json_read_bools_as_numbers", false);
+ settings.set("input_format_json_ignore_key_case", true);
settings.set("input_format_csv_trim_whitespaces", false);
settings.set("input_format_csv_allow_cr_end_of_line", true);
settings.set("output_format_orc_string_as_string", true);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
