Excellent. Explicit toDF() works.
a) employees.toDF().registerTempTable("Employees") - works
b) Also affects saveAsParquetFile - orders.toDF().saveAsParquetFile
Adding to my earlier tests:
4.0 SQL from Scala and Python
4.1 result = sqlContext.sql("SELECT * from Employees WHERE State = 'WA'") OK
4.2 result = sqlContext.sql("SELECT
OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
4.3 result = sqlContext.sql("SELECT ShipCountry, Sum(OrderDetails.UnitPrice
* Qty * Discount) AS ProductSales FROM Orders INNER JOIN OrderDetails ON
Orders.OrderID = OrderDetails.OrderID GROUP BY ShipCountry") OK
4.4 saveAsParquetFile OK
4.5 Read and verify the 4.4 save - sqlContext.parquetFile,
registerTempTable, sql OK
Cheers & thanks Michael
<k/>
On Thu, Feb 19, 2015 at 12:02 PM, Michael Armbrust <mich...@databricks.com>
wrote:
> P.S: For some reason replacing "import sqlContext.createSchemaRDD" with "
>> import sqlContext.implicits._" doesn't do the implicit conversations.
>> registerTempTable
>> gives syntax error. I will dig deeper tomorrow. Has anyone seen this ?
>
>
> We will write up a whole migration guide before the final release, but I
> can quickly explain this one. We made the implicit conversion
> significantly less broad to avoid the chance of confusing conflicts.
> However, now you have to call .toDF in order to force RDDs to become
> DataFrames.
>
