[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-10-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #97865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97865/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-10-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #97828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97828/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-10-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #97845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97845/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-30 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/2 @cloud-fan @rdblue I want to leave some comments and thoughts during looking into this again, hope these can help us deciding the next step plan. Currently all the plan assumed input

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/2 Got it, I'll revert the changes in file source in this commit, thanks for your reply. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/2 can we do it for data source v2 first? It seems hard to fix the file source, as its reader function may lie about the return type. Let's see what's the simplest fix to remove the hack for

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95422/ Test PASSed. ---

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95422/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95422/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-29 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/2 @cloud-fan Thanks for your reply Wenchen, I'm trying to achieve this in this commit, please take a look, thanks. --- - To

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/2 +1 on @rdblue 's idea. One point is, we should use `ColumnarBatchScan.supportsBatch` to indicate columnar scan or not, instead of asking the RDD to report it. ---

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-27 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/2 @xuanyuanking, while this does remove the hack, it doesn't address the underlying problem. The problem is that there is a single RDD, which may contain InternalRow or may contain ColumnarBatch.

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95261/ Test PASSed. ---

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95261/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95261/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/2 cc @cloud-fan and @rdblue have a look when you have time. If this PR doesn't coincide with your expect, I'll close this soon. Thanks! ---

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95217/ Test PASSed. ---

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95217/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/2 **[Test build #95217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95217/testReport)** for PR 2 at commit

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22222: [SPARK-25083][SQL] Remove the type erasure hack in data ...

2018-08-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/2 Test PASSed. Refer to this link for build results (access rights to CI server needed):