freamdx commented on PR #1701:
URL: https://github.com/apache/sedona/pull/1701#issuecomment-2500182222
> > > Will the parser fallback to delegated parser when parsing fails? We
had #1547 to enable this fallback, otherwise the parser extensions of Iceberg
or Delta won't work correctly if they are registered prior to the
initialization of Sedona.
> >
> >
> >
> > 1. iceberg needs to be modified to support geometry type，integrate
sedona is the best solution
> > 2. the parser of iceberg code, like this:
> > class IcebergSparkSqlExtensionsParser(delegate: ParserInterface)
extends ParserInterface ... {
> > ......
> > // if use SedonaSqlParser, then delegate is SedonaSqlParser
> > override def parseDataType(sqlText: String): DataType = {
> > delegate.parseDataType(sqlText)
> > }
> >
> > ...... // override def parsePlan() in SedonaSqlParser is useless
override def parsePlan(sqlText: String): LogicalPlan = { val
sqlTextAfterSubstitution = substitutor.substitute(sqlText) if
(isIcebergCommand(sqlTextAfterSubstitution)) { parse(sqlTextAfterSubstitution)
{ parser => astBuilder.visit(parser.singleStatement())
}.asInstanceOf[LogicalPlan] } else { ...... } } ...... }
>
> If the iceberg extension is initialized after the initialization of
Sedona, `IcebergSparkSqlExtensionsParser` will certainly work correctly, since
`delegate` of `IcebergSparkSqlExtensionsParser` will be SedonaSqlParser. What
if Sedona got initialized after iceberg's Spark SQL extension, will SQL parsing
be delegated to `IcebergSparkSqlExtensionsParser` in this case?
>
> If initializing Sedona defeats any previously registered SQL parser, and
it is the desired behavior, we'd better document it somewhere to inform the
user.
-----------------
look sedona's ParserRegistrator register(): {
val parser = ParserFactory.getParser(parserClassName, delegate)
val field =
sparkSession.sessionState.getClass.getDeclaredField("sqlParser")
field.setAccessible(true)
field.set(sparkSession.sessionState, parser)
}
so, SessionState Field("sqlParser") is always SedonaSqlParser, it doesn't
matter the inject order
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
