[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-10-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209990#comment-17209990 ] Yang Jie commented on SPARK-32989: -- I found that if  stringToMap use codegen, the optimization of

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-10-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209979#comment-17209979 ] Yang Jie commented on SPARK-32989: -- [~ondrej] You're right, It will execute n times with

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-30 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204740#comment-17204740 ] Yang Jie commented on SPARK-32989: -- [~hyukjin.kwon] master also has this problem, may be related to

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-27 Thread Ondrej Kokes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202812#comment-17202812 ] Ondrej Kokes commented on SPARK-32989: -- [~hyukjin.kwon] I've added a file generator in the python

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202810#comment-17202810 ] Hyukjin Kwon commented on SPARK-32989: -- Can you upload the full CSV file? > Performance regression

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-27 Thread Ondrej Kokes (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202801#comment-17202801 ] Ondrej Kokes commented on SPARK-32989: -- [~hyukjin.kwon] I wanted to do so originally, but removing

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202791#comment-17202791 ] Hyukjin Kwon commented on SPARK-32989: -- >From a cursory look, there are two JIRAs that might need

[jira] [Commented] (SPARK-32989) Performance regression when selecting from str_to_map

2020-09-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202790#comment-17202790 ] Hyukjin Kwon commented on SPARK-32989: -- [~ondrej] would you mind showing a self-contained