Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20444
Yeah, I see. LGTM.
On Wed, Jan 31, 2018, 1:03 PM Xiao Li <[email protected]> wrote:
> *@gatorsmile* commented on this pull request.
> ------------------------------
>
> In
>
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceExceptWithFilter.scala
> <https://github.com/apache/spark/pull/20444#discussion_r164948893>:
>
> > val filterCondition =
>
InferFiltersFromConstraints(combineFilters(right)).asInstanceOf[Filter].condition
>
> val attributeNameMap: Map[String, Attribute] = left.output.map(x =>
(x.name, x)).toMap
>
> - filterCondition.transform { case a : AttributeReference =>
attributeNameMap(a.name) }
> + if (filterCondition.references.forall(r =>
attributeNameMap.contains(r.name))) {
> + Some(filterCondition.transform { case a: AttributeReference =>
attributeNameMap(a.name) })
>
> Yes. There are multiple potential cases we can improve for this case. If
> we make it more complicated, it just takes a longer time to review the
> work. This blocks the 2.3 RC. Thus, I would like to fix it in a
> conservative way.
>
> â
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/20444#discussion_r164948893>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AAEM9wQriQJupnqPBxd7KgoAI-Jro-T8ks5tP-YLgaJpZM4RzQaq>
> .
>
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
