[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736645#comment-16736645 ] Dongjoon Hyun commented on SPARK-25823: --- +1. Thank you, [~hyukjin.kwon]. > map_filter can

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736582#comment-16736582 ] Hyukjin Kwon commented on SPARK-25823: -- Looks [~Thincrs] bot is still active. I'm going to ask

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736446#comment-16736446 ] Thincrs commented on SPARK-25823: - A user of thincrs has selected this issue. Deadline: Mon, Jan 14,

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662899#comment-16662899 ] Dongjoon Hyun commented on SPARK-25823: --- Right. That one is cosmetic. I send my opinion on dev

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662891#comment-16662891 ] Sean Owen commented on SPARK-25823: --- I was referring to your comment at

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662760#comment-16662760 ] Dongjoon Hyun commented on SPARK-25823: --- For me, this is a `data correctness` issue at

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662743#comment-16662743 ] Dongjoon Hyun commented on SPARK-25823: --- Sorry, but who says this issue is cosmetic? I think you

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662737#comment-16662737 ] Sean Owen commented on SPARK-25823: --- Got it, you're saying that's cosmetic. But I understood from here

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662735#comment-16662735 ] Dongjoon Hyun commented on SPARK-25823: --- [~srowen] and [~cloud_fan]. To be clear, - This issue

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662726#comment-16662726 ] Dongjoon Hyun commented on SPARK-25823: --- [~srowen]. I commend on SPARK-25824. Please see there. >

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662717#comment-16662717 ] Sean Owen commented on SPARK-25823: --- Hm, another tough one. It's not so much about what these new

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662673#comment-16662673 ] Dongjoon Hyun commented on SPARK-25823: --- Ah, got it. I thought a different one. For mine, I

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662666#comment-16662666 ] Wenchen Fan commented on SPARK-25823: - Anyway we should make map lookup and toString/collect

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662660#comment-16662660 ] Dongjoon Hyun commented on SPARK-25823: --- BTW, [~cloud_fan]. I'm looking around this issue. I'll

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662646#comment-16662646 ] Wenchen Fan commented on SPARK-25823: - [~dongjoon] good catch! I think we should update collect to

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662618#comment-16662618 ] Dongjoon Hyun commented on SPARK-25823: --- Right. This is a long lasting issue when CreateMap added.

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662616#comment-16662616 ] Wenchen Fan commented on SPARK-25823: - BTW one improvement we can do is to remove duplicated map

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662613#comment-16662613 ] Wenchen Fan commented on SPARK-25823: - I did a few experiments {code} scala> sql("SELECT

[jira] [Commented] (SPARK-25823) map_filter can generate incorrect data

[ https://issues.apache.org/jira/browse/SPARK-25823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662583#comment-16662583 ] Dongjoon Hyun commented on SPARK-25823: --- cc [~cloud_fan], [~srowen], [~smilegator],