Its possible that the format of your table is not compatible with your version of hive, so Spark saved it in a way such that only Spark can read it. When this happens it prints out a very visible warning letting you know this has happened.
We've seen it most frequently when trying to save a parquet file with a column in date format into a Hive table. In older versions of hive, its parquet reader/writer did not support Date formats (among a couple others). Nicholas Szandor Hakobian, Ph.D. Staff Data Scientist Rally Health nicholas.hakob...@rallyhealth.com On Fri, Feb 9, 2018 at 9:59 AM, Prakash Joshi <prakashcjos...@gmail.com> wrote: > Ravi, > > Can you send the result of > Show create table your_table_name > > Thanks > Prakash > > On Feb 9, 2018 8:20 PM, "☼ R Nair (रविशंकर नायर)" < > ravishankar.n...@gmail.com> wrote: > >> All, >> >> It has been three days continuously I am on this issue. Not getting any >> clue. >> >> Environment: Spark 2.2.x, all configurations are correct. hive-site.xml >> is in spark's conf. >> >> 1) Step 1: I created a data frame DF1 reading a csv file. >> >> 2) Did manipulations on DF1. Resulting frame is passion_df. >> >> 3) passion_df.write.format("orc").saveAsTable("sampledb.passion") >> >> 4) The metastore shows the hive table., when I do "show tables" in HIVE, >> I can see table name >> >> 5) I can't select in HIVE, though I can select from SPARK as >> spark.sql("select * from sampledb.passion") >> >> Whats going on here? Please help. Why I am not seeing data from HIVE >> prompt? >> The "describe formatted " command on the table in HIVE shows he data is >> is in default warehouse location ( /user/hive/warehouse) since I set it. >> >> I am not getting any definite answer anywhere. Many suggestions and >> answers given in Stackoverflow et al.Nothing really works. >> >> So asking experts here for some light on this, thanks >> >> Best, >> Ravion >> >> >>