AngersZhuuuu edited a comment on pull request #30850:
URL: https://github.com/apache/spark/pull/30850#issuecomment-748601417
> Which case you pointed out? This one?
>
> ```
> scala> sql("select rand()").write.save("/tmp/test")
> org.apache.spark.sql.AnalysisException: Attribute name "rand()" contains
invalid character(s) among " ,;{}()\n\t=". Please use alias to rename it.
> at
org.apache.spark.sql.execution.datasources.parquet.ParquetSchemaConverter$.checkConversionRequirement(ParquetSchemaConverter.scala:579)
> at
org.apache.spark.sql.execution.datasources.parquet.ParquetSchemaConverter$.checkFieldName(ParquetSchemaConverter.scala:570)
> ```
Yea, but InsertIntoHiveDir with parquet format don't have this check. If we
don't concern make it work with hive < 2.3.
Maybe we should add similar check in InsertIntoHiveDirCommand?
And avro schema validate method is private statistic, we need to add this
method on DDLUnit too to check for avro schema?
```
private static String validateName(String name) {
if (!(Boolean)validateNames.get()) {
return name;
} else {
int length = name.length();
if (length == 0) {
throw new SchemaParseException("Empty name");
} else {
char first = name.charAt(0);
if (!Character.isLetter(first) && first != '_') {
throw new SchemaParseException("Illegal initial character: " +
name);
} else {
for(int i = 1; i < length; ++i) {
char c = name.charAt(i);
if (!Character.isLetterOrDigit(c) && c != '_') {
throw new SchemaParseException("Illegal character in: " +
name);
}
}
return name;
}
}
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
