[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-19 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46529253 Ah yeah, this might not have been super clear, but this has at least been my assumption. We do want to make both SQL and GraphX be non-alpha soon though, perhaps as early

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-18 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46436035 Hmmm, that doesn't precisely match my recollection or understanding. Certainly we discussed that alpha components aren't required to maintain a stable API, but I

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46352124 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46352148 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46360655 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15853/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46360653 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46363656 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46363827 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46363845 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46372514 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46372515 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15855/ --- If your project

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13891473 --- Diff: docs/sql-programming-guide.md --- @@ -91,14 +91,33 @@ of its decedents. To create a basic SQLContext, all you need is a SparkContext. {%

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13891482 --- Diff: docs/sql-programming-guide.md --- @@ -91,14 +91,33 @@ of its decedents. To create a basic SQLContext, all you need is a SparkContext. {%

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13891733 --- Diff: docs/sql-programming-guide.md --- @@ -297,50 +328,152 @@ JavaSchemaRDD teenagers = sqlCtx.sql(SELECT name FROM parquetFile WHERE age = div

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13892635 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -123,4 +125,53 @@ abstract class QueryPlan[PlanType :

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13892709 --- Diff: sql/core/pom.xml --- @@ -54,6 +61,11 @@ version${parquet.version}/version /dependency dependency +

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13892874 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -99,6 +97,37 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13892881 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -99,6 +97,35 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13893161 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala --- @@ -342,13 +344,34 @@ class SchemaRDD( def toJavaSchemaRDD: JavaSchemaRDD = new

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13893257 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala --- @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13893273 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala --- @@ -0,0 +1,399 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46380653 This looks to me overall. Only few nitpicks. I think we should merge it after you addressed the couple comments I had. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46383321 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46383326 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46387956 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46387957 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15862/ --- If your project

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46389105 Thanks. I'm merging this in master branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/999 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2060][SQL] Querying JSON Datasets with ...

2014-06-17 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46389597 Is that the basic strategy we are going to use with AlphaComponents -- merging new APIs at both the minor and maintenance levels? I don't know that I have any

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222354 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222342 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222401 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15821/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222400 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46222714 Have made a few changes: * Removed the special SchemaRDD (JsonRDD) for JSON datasets. Now, when users call `jsonFile` and `jsonRDD`, a SchemaRDD is returned. * Added

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13826595 --- Diff: docs/sql-programming-guide.md --- @@ -17,20 +17,20 @@ Spark. At the core of this component is a new type of RDD,

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13826671 --- Diff: docs/sql-programming-guide.md --- @@ -62,10 +62,10 @@ descendants. To create a basic SQLContext, all you need is a SparkContext. {%

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827260 --- Diff: docs/sql-programming-guide.md --- @@ -98,7 +98,9 @@ sqlCtx = SQLContext(sc) /div -## Running SQL on RDDs +# Data Sources

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827230 --- Diff: docs/sql-programming-guide.md --- @@ -98,7 +98,9 @@ sqlCtx = SQLContext(sc) /div -## Running SQL on RDDs +# Data Sources

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827652 --- Diff: docs/sql-programming-guide.md --- @@ -310,37 +325,190 @@ parquetFile = sqlCtx.parquetFile(people.parquet) # Parquet files can also be

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827707 --- Diff: docs/sql-programming-guide.md --- @@ -310,37 +325,190 @@ parquetFile = sqlCtx.parquetFile(people.parquet) # Parquet files can also be

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13827732 --- Diff: docs/sql-programming-guide.md --- @@ -310,37 +325,190 @@ parquetFile = sqlCtx.parquetFile(people.parquet) # Parquet files can also be

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828272 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala --- @@ -108,19 +118,18 @@ trait HiveTypeCoercion {

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828440 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizerTest.scala --- @@ -17,39 +17,10 @@ package

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/package.scala --- @@ -17,8 +17,55 @@ package

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828571 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -99,6 +97,35 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828608 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -99,6 +97,35 @@ class SQLContext(@transient val sparkContext: SparkContext)

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828715 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala --- @@ -0,0 +1,402 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13828804 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala --- @@ -0,0 +1,402 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13829492 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala --- @@ -122,4 +122,10 @@ private[sql] trait SchemaRDDLike { @Experimental

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13829475 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -123,4 +124,10 @@ abstract class QueryPlan[PlanType :

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13829803 --- Diff: examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQL.java --- @@ -56,6 +57,8 @@ public static void main(String[] args) throws

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13830011 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -123,4 +124,10 @@ abstract class QueryPlan[PlanType :

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13830083 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala --- @@ -122,4 +122,10 @@ private[sql] trait SchemaRDDLike { @Experimental

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13830433 --- Diff: docs/sql-programming-guide.md --- @@ -62,10 +62,10 @@ descendants. To create a basic SQLContext, all you need is a SparkContext. {%

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13834977 --- Diff: docs/sql-programming-guide.md --- @@ -310,37 +325,190 @@ parquetFile = sqlCtx.parquetFile(people.parquet) # Parquet files can also be registered

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46249486 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46249650 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46250558 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46250551 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46255972 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15831/ --- If your project

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46255971 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13839460 --- Diff: docs/sql-programming-guide.md --- @@ -91,14 +91,33 @@ of its decedents. To create a basic SQLContext, all you need is a SparkContext.

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13839520 --- Diff: docs/sql-programming-guide.md --- @@ -170,12 +191,11 @@ A schema can be applied to an existing RDD by calling `applySchema` and providin for

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13839554 --- Diff: docs/sql-programming-guide.md --- @@ -64,8 +64,8 @@ descendants. To create a basic SQLContext, all you need is a SparkContext. val sc:

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13840123 --- Diff: docs/sql-programming-guide.md --- @@ -170,12 +191,11 @@ A schema can be applied to an existing RDD by calling `applySchema` and providin for

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13840206 --- Diff: docs/sql-programming-guide.md --- @@ -297,50 +328,152 @@ JavaSchemaRDD teenagers = sqlCtx.sql(SELECT name FROM parquetFile WHERE age = div

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46260432 I just did a really light pass on the docs and public interfaces exposed. From that perspective, this looks good to me! --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13840531 --- Diff: docs/sql-programming-guide.md --- @@ -170,12 +191,11 @@ A schema can be applied to an existing RDD by calling `applySchema` and providin for the

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46261318 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46261327 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46264603 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46264604 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15837/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-15 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46111039 @mateiz, thats a good point and actually there is only a single implicit conversion needed for all the non-DSL examples (from RDD - SchemaRDD). Perhaps we could import

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-14 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46101559 Hey Yin, a few comments on the docs: - You should mention JSON as a data source in the first paragraph of the Spark SQL doc (right now it only mentions Parquet and

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-14 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46101589 Also another note for @marmbrus as well: I'd really try to minimize the use of top-level SQLContext methods being called without `context.` in front of them due to `import

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46046033 Programming guide: http://yhuai.github.io/site/sql-programming-guide.html Scala doc of SQLContext:

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46046107 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46046123 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46046141 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46046144 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15768/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46047235 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46047249 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46047263 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15769/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46047262 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46061901 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46061886 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46061920 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-46061921 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15773/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-12 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13690133 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala --- @@ -0,0 +1,485 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45814358 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45814366 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45814385 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15694/ --- If your project is set up for it, you can

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13601226 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonTable.scala --- @@ -0,0 +1,364 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13601819 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala --- @@ -0,0 +1,371 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13602167 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala --- @@ -0,0 +1,371 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13603527 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/json/JsonSuite.scala --- @@ -0,0 +1,371 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45637957 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/999#discussion_r13604518 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/json/JsonTable.scala --- @@ -0,0 +1,364 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45637977 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark 2060][SQL] Querying JSON Datasets with ...

2014-06-10 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/999#issuecomment-45642845 API doc for sql/core http://yhuai.github.io/spark-sql-core/api/#package --- If your project is set up for it, you can reply to this email and have your reply appear on

  1   2   >