dongkelun commented on pull request #3380:
URL: https://github.com/apache/hudi/pull/3380#issuecomment-898210263
> > @pengzhiwei2018 Hi,When I test Spark3, I find that Spark SQL for Hoodie
with Spark3 uses the source code of Spark, but columns aliases in Merge Into is
not supported in Spark3, it will throw the following exception: 'Columns
aliases are not allowed in MERGE.'.I think there are two solutions, one is to
modify the source code of Spark3 to make Spark support, the other is to write
code in hudi-spark3 to implement Spark SQL for Hoodie, but I personally feel
that this is a big change, I do not know if I understand correctly. So I was
hoping you could help with some advice.
> > ` // org.apache.spark.sql.catalyst.parser.AstBuilder
> > val sourceTableAlias = getTableAliasWithoutColumnAlias(ctx.sourceAlias,
"MERGE")
> > private def getTableAliasWithoutColumnAlias(
> > ctx: TableAliasContext, op: String): Option[String] = {
> > if (ctx == null) {
> > None
> > } else {
> > val ident = ctx.strictIdentifier()
> > if (ctx.identifierList() != null) {
> > throw new ParseException(s"Columns aliases are not allowed in $op.",
ctx.identifierList())
> > }
> > if (ident != null) Some(ident.getText) else None
> > }
> > }`
>
> I think we can support this feature only for spark2 currently. You can
change the test case by `HoodieSqlUtils#isSpark3`, if it spark3, use the
`checkException` to validate the exception for spark3, For spark2, check the
answer.
Ok, I'll change the test case as you suggested
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
