PHILO-HE commented on code in PR #8583:
URL: https://github.com/apache/incubator-gluten/pull/8583#discussion_r1924782665
##########
gluten-ut/spark35/src/test/scala/org/apache/spark/sql/GlutenDateFunctionsSuite.scala:
##########
@@ -29,6 +29,34 @@ class GlutenDateFunctionsSuite extends DateFunctionsSuite
with GlutenSQLTestsTra
private def secs(millis: Long): Long =
TimeUnit.MILLISECONDS.toSeconds(millis)
+ test("unix_timestamp/to_unix_timestamp") {
Review Comment:
Seems there is no need to add a new test. We can just remove the below
commented code to allow test legacy policy.
https://github.com/apache/incubator-gluten/blob/main/gluten-ut/spark35/src/test/scala/org/apache/spark/sql/GlutenDateFunctionsSuite.scala#L118
https://github.com/apache/incubator-gluten/blob/main/gluten-ut/spark35/src/test/scala/org/apache/spark/sql/GlutenDateFunctionsSuite.scala#L237
Ditto for spark32/33/34.
##########
cpp/velox/compute/WholeStageResultIterator.cc:
##########
@@ -551,8 +551,13 @@ std::unordered_map<std::string, std::string>
WholeStageResultIterator::getQueryC
configs[velox::core::QueryConfig::kSparkPartitionId] =
std::to_string(taskInfo_.partitionId);
- // Enable Spark legacy date formatter if spark.sql.legacy.timeParserPolicy
is set to 'LEGACY'.
- if (veloxCfg_->get<std::string>(kSparkLegacyTimeParserPolicy, "") ==
"LEGACY") {
+ // Enable Spark legacy date formatter if spark.sql.legacy.timeParserPolicy
is set to 'LEGACY'
+ // or 'legacy'.
+ auto sparkPolicy =
veloxCfg_->get<std::string>(kSparkLegacyTimeParserPolicy, "");
+ for (auto& c : sparkPolicy) {
+ c = std::toupper(static_cast<unsigned char>(c));
+ }
+ if (sparkPolicy == "LEGACY") {
Review Comment:
Seems we can just do this check: `if (sparkPolicy == "LEGACY" || sparkPolicy
== "legacy")`. No need to convert to lower case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
