Its possible that the format of your table is not compatible with your
version of hive, so Spark saved it in a way such that only Spark can read
it. When this happens it prints out a very visible warning letting you know
this has happened.
We've seen it most frequently when trying to save a parquet file with a
column in date format into a Hive table. In older versions of hive, its
parquet reader/writer did not support Date formats (among a couple others).
Nicholas Szandor Hakobian, Ph.D.
Staff Data Scientist
On Fri, Feb 9, 2018 at 9:59 AM, Prakash Joshi <prakashcjos...@gmail.com>
> Can you send the result of
> Show create table your_table_name
> On Feb 9, 2018 8:20 PM, "☼ R Nair (रविशंकर नायर)" <
> ravishankar.n...@gmail.com> wrote:
>> It has been three days continuously I am on this issue. Not getting any
>> Environment: Spark 2.2.x, all configurations are correct. hive-site.xml
>> is in spark's conf.
>> 1) Step 1: I created a data frame DF1 reading a csv file.
>> 2) Did manipulations on DF1. Resulting frame is passion_df.
>> 3) passion_df.write.format("orc").saveAsTable("sampledb.passion")
>> 4) The metastore shows the hive table., when I do "show tables" in HIVE,
>> I can see table name
>> 5) I can't select in HIVE, though I can select from SPARK as
>> spark.sql("select * from sampledb.passion")
>> Whats going on here? Please help. Why I am not seeing data from HIVE
>> The "describe formatted " command on the table in HIVE shows he data is
>> is in default warehouse location ( /user/hive/warehouse) since I set it.
>> I am not getting any definite answer anywhere. Many suggestions and
>> answers given in Stackoverflow et al.Nothing really works.
>> So asking experts here for some light on this, thanks