[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-12-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 Issues, which this PR addresses, have been solved by other approaches (i.e. use `UnsafePrimitiveArray`). --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-12-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13758 I think we can close this now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-12-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 I think that it is good time to close this when #13909 is closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-12-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 shall we close it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 I'm sorry that if the TODO in `GenericArrayData` mislead you, but after https://github.com/apache/spark/pull/13680 , I don't think this TODO makes sense anymore, as `GenericArrayData` is meant to

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 > This PR also optimize generate code of projection for an primitive type array. While we know primitive type array does not require null check and has contiguous data region, this PR performs

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68397/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68397/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68397/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan yes, we could take the same approach as #15044. When I have just implement it in my local environment, it can achieve similar performance improvement. I will submit that approach to

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 @kiszk, can you update the PR description to explain where is the performance benefits of this PR? For the given example, `df.selectExpr("Array(value + 1.1d, value + 2.2d)").show`, can we just

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68361/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68361/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68362/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68362/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @hvanhovell Yes, as you said, an issue in a projection to write an array was resolved by #15044. I updated this PR to show another use case of this PR. What do you think? --- If your project is

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68362/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68361/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68349/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68349/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68344/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68344/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68349/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68345/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68345/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68345/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #68344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68344/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread oza
Github user oza commented on the issue: https://github.com/apache/spark/pull/13758 @kiszk it looks that this PR needs rebasing. Could you rebase it on master? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13758 @kiszk do we still need this one? We now have a much better UnsafeArrayData format, and the datasets conversions have also been improved. --- If your project is set up for it, you can reply to

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-11-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan @rxin Could you please review this? The latest implementation has less changes than previous implementation that introduced a factory method. --- If your project is set up for it, you can

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66217/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66211/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66210/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #66210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66210/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-10-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #66209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66209/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65293/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65293/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65293/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65277/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65277/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65277/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65239/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65239/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65239/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65238/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65238/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65238/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65215/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65215/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65215/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65208/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65208/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65208/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan , I see. First, I will create another PR to focus on projection by making the new `UnsafeArrayData`. After merging the PR, let us be back to this PR. --- If your project is set up for

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-16 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 no, by generating unsafe array directly, we will have an additional copy. But I don't think that matters too much, and worth such a big change. Anyway let's make the new unsafe array in

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 You are right. I missed `UnsafeArrayData` is a subclass of `ArrayData`. We can pass `UnsafeArrayData` to an projection. I have one question. When we directly generate `UnsafeArrayData`

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 you can take a look at `GenerateUnsafeProjection`, if the `ArrayData` is already an unsafe array, we will copy it directly, no iteration is needed. --- If your project is set up for it, you can

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 Thank you for your comment. We are on the same page to address the problem. In this case, you are right. This is because we finally would like to write an primitive int array

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13758 The new `UnsafeArrayData` should be as fast as primitive array right? The problem you pointed out is valid, but can we improve the encoder framework to create unsafe array directly for primitive

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 I am sorry for missing something in my description. [The PR](https://github.com/apache/spark/pull/13680) for `UnsafeArrayData` and this PR and address different issues. [The

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13758 @kiszk I have the feeling this is not that useful anymore since we are getting close to committing your work on the UnsafeArrayData. This seems like it is addressing an edge case. What are the

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @hvanhovell , @cloud-fan , @davies could you please review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @hvanhovell , @cloud-fan , @davies would it be possible to review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @hvanhovell , @cloud-fan , @davies could you please review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @hvanhovell and @cloud-fan could you please look at this again since I resolved conflict. Since #13704 is close to merge, this PR can improve performance in more cases. --- If your project is

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62826/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62826/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62826/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan would it be possible to review this? If I have to prepare additional test cases, benchmark results or etc., could you please let me know? --- If your project is set up for it, you can

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62531/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62531/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62531/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62523/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62523/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62523/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13758 @cloud-fan , would it be possible to review this? If I have to prepare additional benchmark results or etc., could you please let me know? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62109/ Test PASSed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62109 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62109/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #62109 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62109/consoleFull)** for PR 13758 at commit

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62103/ Test FAILed. ---

  1   2   3   >