Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/21449
yes @daniel-shields, you are right with your analysis. The problem was that
we were sometimes using `==`, sometimes `semanticEquals`. And `equals` has the
problem you mentioned.
I think this is the only way for addressing the problem described here is
to reference which dataset the column is coming from. I think adding a metadata
for it is the cleanest way. We may also add a new attribute to the `Attribute`
class instead of using metadata, but honestly this way seemed cleaner to me.
What do you think? Do you have other suggestions?
cc @cloud-fan @hvanhovell @gatorsmile
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
