mihailom-db commented on code in PR #47364:
URL: https://github.com/apache/spark/pull/47364#discussion_r1699535248
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##########
@@ -734,6 +865,10 @@ public CollationIdentifier identifier() {
public static final String PROVIDER_ICU = "icu";
public static final List<String> SUPPORTED_PROVIDERS =
List.of(PROVIDER_SPARK, PROVIDER_ICU);
+ private static final String COLLATION_CATALOG = "SYSTEM";
+ private static final String COLLATION_SCHEMA = "BUILTIN";
Review Comment:
I do not think this information should be stored here, could you take a look
at `ShowTablesCommand`. What we do there is get the catalog and then work on
from there. We could actually do something similar here, as UDF collations in
future will be defined with catalog and schema fields, and will be stored
somewhere. This is a future problem, but for now, let's try to keep code as
close as what we will need in future.
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##########
@@ -704,6 +789,52 @@ private String collationName() {
}
return builder.toString();
}
+
+ @Override
+ protected Meta buildMeta() {
+ return new Meta(
+ COLLATION_CATALOG,
+ COLLATION_SCHEMA,
+ collationName(),
+ ICULocaleMap.get(locale).getLanguage(),
+ ICULocaleMap.get(locale).getCountry(),
Review Comment:
Could we possible change here to getDisplayCountry() and
getDisplayLanguage(). The idea is to later be able to look for locales not only
on name, but also on country name i.e. Greece, France...
##########
common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java:
##########
@@ -734,6 +865,10 @@ public CollationIdentifier identifier() {
public static final String PROVIDER_ICU = "icu";
public static final List<String> SUPPORTED_PROVIDERS =
List.of(PROVIDER_SPARK, PROVIDER_ICU);
+ private static final String COLLATION_CATALOG = "SYSTEM";
+ private static final String COLLATION_SCHEMA = "BUILTIN";
Review Comment:
Ideally we would love to have a buffer on session.catalog level called
collations and then store any user defined collations in there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
