[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-07-27 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559339#comment-16559339 ] Xiao Li commented on SPARK-24288: - [~hyukjin.kwon]I updated the JIRA description. > Enable preventing

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-07-25 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556801#comment-16556801 ] Hyukjin Kwon commented on SPARK-24288: -- [~smilegator] should we resolve this {{Won't fix}} for now?

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-07-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556119#comment-16556119 ] Apache Spark commented on SPARK-24288: -- User 'maryannxue' has created a pull request for this

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-07-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554109#comment-16554109 ] Tomasz Gawęda commented on SPARK-24288: --- [~smilegator] Yes, you are right. If we don't want to use

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-07-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553566#comment-16553566 ] Xiao Li commented on SPARK-24288: - [~TomaszGaweda] Based on my understanding, when the predicates fail

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-18 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480237#comment-16480237 ] Maryann Xue commented on SPARK-24288: - Thank you for pointing this out, [~cloud_fan]! I made 

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480232#comment-16480232 ] Apache Spark commented on SPARK-24288: -- User 'maryannxue' has created a pull request for this issue:

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-17 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478671#comment-16478671 ] Wenchen Fan commented on SPARK-24288: - You can take a look at `ResolveReferences#dedupRight`, which

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478534#comment-16478534 ] Maryann Xue commented on SPARK-24288: - Yes, sure. This is more of a demonstration than a patch. So

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478495#comment-16478495 ] Wenchen Fan commented on SPARK-24288: - Thanks for the quick patch! can you also send a PR? that will

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477747#comment-16477747 ] Maryann Xue commented on SPARK-24288: - Thank you, [~cloud_fan] and [~TomaszGaweda] for the input!

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477663#comment-16477663 ] Wenchen Fan commented on SPARK-24288: - We need to determine the scope first: 1. add a config to

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477160#comment-16477160 ] Tomasz Gawęda commented on SPARK-24288: --- Hi, my idea was to have some specialized operator, as

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476985#comment-16476985 ] Maryann Xue commented on SPARK-24288: - The special operator would just look like {{sql("SELECT * FROM

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476932#comment-16476932 ] Wenchen Fan commented on SPARK-24288: - if we add this special logic operator, how would the

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476928#comment-16476928 ] Xiao Li commented on SPARK-24288: - The proposal is something like the traditional query hint like

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-16 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476900#comment-16476900 ] Maryann Xue commented on SPARK-24288: - After taking a closer look at the problem, I realized that we

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-15 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476372#comment-16476372 ] Maryann Xue commented on SPARK-24288: - I agree with you, [~TomaszGaweda], that making this a special

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-15 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476368#comment-16476368 ] Maryann Xue commented on SPARK-24288: - Feel free to assign this to me then, [~smilegator]. > Enable

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

2018-05-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476303#comment-16476303 ] Xiao Li commented on SPARK-24288: - cc [~maryannxue] You might be interested in this. > Enable