[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8907 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-144296080 @davies @cloud-fan Thanks for the review! I'm merging this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-29 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-144252020 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-29 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-144217445 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-25 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8907#discussion_r40451726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala --- @@ -145,7 +145,13 @@ final class Decimal extends Ordered[Decimal] with S

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143096240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143096239 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143096170 [Test build #42992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42992/console) for PR 8907 at commit [`851f91f`](https://github.

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143085812 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143085813 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143085733 [Test build #42985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42985/console) for PR 8907 at commit [`6d85e69`](https://github.

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143078472 [Test build #42992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42992/consoleFull) for PR 8907 at commit [`851f91f`](https://gith

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143078376 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143078366 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143078102 The [last commit] [1] brings another ~5% performance boost. We should probably just use Java `BigDecimal` directly within Catalyst `Decimal`. I guess this would make d

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143075010 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143075012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143074577 [Test build #42984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42984/console) for PR 8907 at commit [`d59daf6`](https://github.

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143070006 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143070007 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143069919 [Test build #42979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42979/console) for PR 8907 at commit [`b99158e`](https://github.

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143064861 [Test build #42985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42985/consoleFull) for PR 8907 at commit [`6d85e69`](https://gith

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143063693 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143063722 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143062763 Hm, after some profiling, I found that stealing byte arrays using `ByteBuffer` is actual faster than the `ByteArrayThief` hack I employed in the 2nd commit. This is pr

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143057207 [Test build #42984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42984/consoleFull) for PR 8907 at commit [`d59daf6`](https://gith

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143056520 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143056483 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8907#discussion_r40373071 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystRowConverter.scala --- @@ -302,7 +302,13 @@ private[parquet] class

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8907#discussion_r40372370 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystRowConverter.scala --- @@ -302,7 +302,13 @@ private[parquet] c

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143048674 [Test build #42979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42979/consoleFull) for PR 8907 at commit [`b99158e`](https://gith

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/8907#discussion_r40371826 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystRowConverter.scala --- @@ -302,7 +302,13 @@ private[parquet] class

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143047536 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8907#issuecomment-143047485 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/8907#discussion_r40371067 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/CatalystRowConverter.scala --- @@ -332,24 +338,30 @@ private[parquet]

[GitHub] spark pull request: [SPARK-10811] [SQL] Eliminates unnecessary byt...

2015-09-24 Thread liancheng
GitHub user liancheng opened a pull request: https://github.com/apache/spark/pull/8907 [SPARK-10811] [SQL] Eliminates unnecessary byte array copying When reading Parquet string and binary-backed decimal values, Parquet `Binary.getBytes` always returns a copied byte array, which is u