[
https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013914#comment-15013914
]
Santiago M. Mola commented on SPARK-11855:
------------------------------------------
They have public visibility and no @DeveloperApi or @Experimental annotations,
so I always assumed they are. I have been working with catalyst on a day-to-day
basis for almost 1 year now. I understand that catalyst might not offer the
same kind of backwards compatibility as spark-core, but it would be good to
avoid breaking backwards compatibility, specially in cases where it is easy to
do (which are most of the cases I encounter).
I think part of the solution is also marking some parts as @Experimental. For
example, UnsafeArrayData interface changed wildly, and it's probably not viable
to maintain backwards compatibility, but it should be marked as @Experimental
if more breakage is expected before 2.0.
> Catalyst breaks backwards compatibility in branch-1.6
> -----------------------------------------------------
>
> Key: SPARK-11855
> URL: https://issues.apache.org/jira/browse/SPARK-11855
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Santiago M. Mola
> Priority: Critical
>
> There's a number of APIs broken in catalyst 1.6.0. I'm trying to compile most
> cases:
> *UnresolvedRelation*'s constructor has been changed from taking a Seq to a
> TableIdentifier. A deprecated constructor taking Seq would be needed to be
> backwards compatible.
> {code}
> case class UnresolvedRelation(
> - tableIdentifier: Seq[String],
> + tableIdentifier: TableIdentifier,
> alias: Option[String] = None) extends LeafNode {
> {code}
> It is similar with *UnresolvedStar*:
> {code}
> -case class UnresolvedStar(table: Option[String]) extends Star with
> Unevaluable {
> +case class UnresolvedStar(target: Option[Seq[String]]) extends Star with
> Unevaluable {
> {code}
> Spark 1.5 already broke backwards compatibility of part of catalyst API with
> respect to 1.4. I understand there are good reasons for some cases, but we
> should try to minimize backwards compatibility breakages for 1.x. Specially
> now that 2.x is on the horizon and there will be a near opportunity to remove
> deprecated stuff.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
