Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15316
@hvanhovell @rxin
@liancheng listed the reason in the original PR.
> Dataset always do eager analysis now
> We used to support disabling DataFrame eager analysis to help reporting
partially analyzed malformed logical plan on analysis failure. However, Dataset
encoders requires eager analysis during Dataset construction. To preserve the
error reporting feature, AnalysisException now takes an extra
Option[LogicalPlan] argument to hold the partially analyzed plan, so that we
can check the plan tree when reporting test failures. This plan is passed by
QueryExecution.assertAnalyzed.
However, there is a bug in AnslysisException output. It does not output the
plan. I just fixed it. See the updated PR description.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
