Github user ankurdave commented on the pull request:
https://github.com/apache/spark/pull/9089#issuecomment-147625512
@marmbrus I addressed your comments from the review about a month ago:
1. Foreign key references now store the referenced relation directly as a
logical plan rather than requiring a catalog lookup.
2. We now use `semanticEquals` and `AttributeSet` for attributes instead of
normal equality.
There were a few comments that didn't make sense on second thought:
1. Move the attribute equivalence check in `ForeignKeyFinder` to a method
on `LogicalPlan`. We thought this would simplify the logic, but it turned out
not to (still need to maintain the disjoint-set data structure, and the logic
gets split between `LogicalPlan` and `Project`).
2. Move foreign key attribute resolution to its own rule that runs at the
end of analysis. This would work fine, but it seems to fit well within
`ResolveReferences`.
Finally, the new DataFrame methods should probably be marked as alpha
somehow, but I'm not sure of the best way. Maybe a new ScalaDoc group?
cc @rxin, @jkbradley
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
