we are on 1.6.1 version of spark under CDH5.7.1
On Mon, Feb 6, 2017 at 2:53 PM, Xiao Li <gatorsm...@gmail.com> wrote:
> Which Spark version are you using?
>
> 2017-02-06 12:25 GMT-05:00 vaquar khan <vaquar.k...@gmail.com>:
>
>> Did you try MSCK REPAIR TABLE ?
>>
>> Regards,
>> Vaquar Khan
>>
>> On Feb 6, 2017 11:21 AM, "KhajaAsmath Mohammed" <mdkhajaasm...@gmail.com>
>> wrote:
>>
>>> I dont think so, i was able to insert overwrite other created tables in
>>> hive using spark sql. The only problem I am facing is, spark is not able
>>> to recognize hive view name. Very strange but not sure where I am doing
>>> wrong in this.
>>>
>>> On Mon, Feb 6, 2017 at 11:03 AM, Jon Gregg <coble...@gmail.com> wrote:
>>>
>>>> Confirming that Spark can read newly created views - I just created a
>>>> test view in HDFS and I was able to query it in Spark 1.5 immediately after
>>>> without a refresh. Possibly an issue with your Spark-Hive connection?
>>>>
>>>> Jon
>>>>
>>>> On Sun, Feb 5, 2017 at 9:31 PM, KhajaAsmath Mohammed <
>>>> mdkhajaasm...@gmail.com> wrote:
>>>>
>>>>> Hi Khan,
>>>>>
>>>>> It didn't work in my case. used below code. View is already present in
>>>>> Hive but I cant read that in spark sql. Throwing exception that table not
>>>>> found
>>>>>
>>>>> sqlCtx.refreshTable("schema.hive_view")
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Asmath
>>>>>
>>>>>
>>>>> On Sun, Feb 5, 2017 at 7:56 PM, vaquar khan <vaquar.k...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ashmath,
>>>>>>
>>>>>> Try refresh table
>>>>>>
>>>>>> // spark is an existing SparkSession
>>>>>> spark.catalog.refreshTable("my_table")
>>>>>>
>>>>>>
>>>>>> http://spark.apache.org/docs/latest/sql-programming-guide.ht
>>>>>> ml#metadata-refreshing
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Vaquar khan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Feb 5, 2017 at 7:19 PM, KhajaAsmath Mohammed <
>>>>>> mdkhajaasm...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have a hive view which is basically set of select statements on
>>>>>>> some tables. I want to read the hive view and use hive builtin functions
>>>>>>> available in spark sql.
>>>>>>>
>>>>>>> I am not able to read that hive view in spark sql but can retreive
>>>>>>> data in hive shell.
>>>>>>>
>>>>>>> can't spark access hive views?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Asmath
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Vaquar Khan
>>>>>> +1 -224-436-0783 <(224)%20436-0783>
>>>>>>
>>>>>> IT Architect / Lead Consultant
>>>>>> Greater Chicago
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
