Re: Spark 1.6.1

I'm not going to be able to do anything until after the Spark Summit, but I will kick off RC1 after that (end of week). Get your patches in before then! On Sat, Feb 13, 2016 at 4:57 PM, Jong Wook Kim wrote: > Is 1.6.1 going to be ready this week? I see that the two last

Re: Dataset in spark 2.0.0-SNAPSHOT missing columns

yeah i was surprised with the Option. it works in 1.6.0-SNAPSHOT, and its a pretty neat way to indicate nullability i guess. i will file jira. i saw similar behavior with other types than Option. this was just the easiest to show. On Mon, Feb 15, 2016 at 3:52 PM, Reynold Xin

Re: Dataset in spark 2.0.0-SNAPSHOT missing columns

Looks like a bug. I'm also not sure whether we support Option yet. (If not, we should definitely support that in 2.0.) Can you file a JIRA ticket? On Mon, Feb 15, 2016 at 7:12 AM, Koert Kuipers wrote: > i noticed some things stopped working on datasets in spark

Subscribe

Subscribe

Dataset in spark 2.0.0-SNAPSHOT missing columns

i noticed some things stopped working on datasets in spark 2.0.0-SNAPSHOT, and with a confusing error message (cannot resolved some column with input columns []). for example in 1.6.0-SNAPSHOT: scala> val ds = sc.parallelize(1 to 10).toDS ds: org.apache.spark.sql.Dataset[Int] = [value: int]

Re: Spark 1.6.0 + Hive + HBase

anyone took a look at this issue: https://issues.apache.org/jira/browse/HIVE-11166 i got same exception by inserting into hbase table -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-1-6-0-Hive-HBase-tp16128p16332.html Sent from the Apache Spark