Github user janewangfb commented on the issue:
https://github.com/apache/spark/pull/18023
@gatorsmile, regarding:
> @janewangfb If we turn on the flag
spark.sql.parser.quotedRegexColumnNames by default, the > following test cases
failed. Could you do some investigations? Thanks!
>
> org.apache.spark.sql.SQLQuerySuite
the struct type was not supported now in the regex
some special characters has different meaning in regex.
> org.apache.spark.sql.DataFrameSuite
some special characters has different meaning in regex.
> org.apache.spark.sql.SingleLevelAggregateHashMapSuite
> org.apache.spark.sql.DataFrameAggregateSuite
> org.apache.spark.sql.TwoLevelAggregateHashMapSuite
> org.apache.spark.sql.TwoLevelAggregateHashMapWithVectorizedMapSuite
> org.apache.spark.sql.DataFrameNaFunctionsSuite
> org.apache.spark.sql.DataFrameStatSuite
These four failed for the same testcase. in AS alias, regex is not allowed.
> org.apache.spark.sql.SQLQueryTestSuite
This suite has the same behavior wether
spark.sql.parser.quotedRegexColumnNames default value is true/false.
> org.apache.spark.sql.execution.datasources.json.JsonSuite
for map struct, regex should not be allowed in A[B] part.
> org.apache.spark.sql.DatasetSuite
Expected. Explicitly set the spark.sql.parser.quotedRegexColumnNames =
false to false for those tests.
> org.apache.spark.sql.sources.TableScanSuite
some special characters has different meaning in regex
> org.apache.spark.sql.execution.datasources.parquet.ParquetFilterSuite
regex is not allowed in where.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
