Re: A question about creating persistent table when in-memory catalog is used

I see, thanks for the info! On Mon, Jan 23, 2017 at 4:12 PM, Xiao Li wrote: > Reynold mentioned the direction we are heading. You can see many PRs the > community submitted are for this target. To achieve this, a lot of works we > need to do. > > For example, for some

Re: A question about creating persistent table when in-memory catalog is used

Reynold mentioned the direction we are heading. You can see many PRs the community submitted are for this target. To achieve this, a lot of works we need to do. For example, for some serde, Hive metastore will infer the schema when the schema is not provided, but our InMemoryCatalog does not have

Re: A question about creating persistent table when in-memory catalog is used

Agree. : ) 2017-01-22 11:20 GMT-08:00 Reynold Xin : > To be clear there are two separate "hive" we are talking about here. One > is the catalog, and the other is the Hive serde and UDF support. We want to > get to a point that the choice of catalog does not impact the

Re: A question about creating persistent table when in-memory catalog is used

To be clear there are two separate "hive" we are talking about here. One is the catalog, and the other is the Hive serde and UDF support. We want to get to a point that the choice of catalog does not impact the functionality in Spark other than where the catalog is stored. On Sun, Jan 22, 2017

Re: A question about creating persistent table when in-memory catalog is used

We have a pending PR to block users to create the Hive serde table when using InMemroyCatalog. See: https://github.com/apache/spark/pull/16587 I believe it answers your question. BTW, we still can create the regular data source tables and insert the data into the tables. The major difference is

A question about creating persistent table when in-memory catalog is used

Hi all, Currently when the in-memory catalog is used, e.g. through `--conf spark.sql.catalogImplementation=in-memory`, we can create a persistent table, but inserting into this table would fail with error message "Hive support is required to insert into the following tables..". sql("create