[
https://issues.apache.org/jira/browse/BIGTOP-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311598#comment-14311598
]
jay vyas commented on BIGTOP-1653:
----------------------------------
Update. For the first part of this, I've implemented a simple spark sql query
in {{PetStoreStatistics.scala}} like this.
{noformat}
def totalTransactions(r:(_,_,_,_,RDD[Transaction]), sc: SparkContext):
Statistics = {
val sqlContext = new org.apache.spark.sql.SQLContext(sc);
import sqlContext._;
r._5.registerTempTable("transactions");
val results: SchemaRDD = sql("SELECT * FROM transactions")
return Statistics(results.count());
{noformat}
This fails currently, possibly a serialization error... ?
{noformat}
java.util.Calendar (of class scala.reflect.internal.Types$TypeRef$$anon$6)
scala.MatchError: java.util.Calendar (of class
scala.reflect.internal.Types$TypeRef$$anon$6)
at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:53)
at
org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:64)
at
org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:62)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:62)
at
org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:50)
at
org.apache.spark.sql.catalyst.ScalaReflection$.attributesFor(ScalaReflection.scala:44)
at
org.apache.spark.sql.execution.ExistingRdd$.fromProductRdd(basicOperators.scala:229)
at org.apache.spark.sql.SQLContext.createSchemaRDD(SQLContext.scala:94)
at
org.apache.bigtop.bigpetstore.spark.generator.PetStoreStatistics$.totalTransactions(PetStoreStatistics.scala:76)
{noformat}
will look more later
> Add queries for customer, state, and product statistics w/ d3 friendly JSON
> output to analytics phase.
> -------------------------------------------------------------------------------------------------------
>
> Key: BIGTOP-1653
> URL: https://issues.apache.org/jira/browse/BIGTOP-1653
> Project: Bigtop
> Issue Type: Improvement
> Components: blueprints
> Affects Versions: 0.9.0
> Reporter: jay vyas
> Fix For: 0.9.0
>
>
> Follow on to BIGTOP-1536, this time we can use a scala json library if a good
> one exists.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
