Re: 答复: Difference between MANAGED_TABLE and EXTERNAL_TABLE in org.apache.hadoop.hive.metastore.TableType

2016-12-02 Thread Patrick Duin
Hi,

I've noticed the same thing we set the table parameter as well to make sure
the table is External.
replica.putToParameters("EXTERNAL", "TRUE")

Not sure if the tableType is actually used anywhere, we set it anyway as
well as the table parameter just to be sure when using the Metastore API.

No clue if it is a bug but setting the table parameter will work.

Patrick

2016-12-02 8:38 GMT+01:00 Huang Meilong :

> Thanks for your detailed explanations.
>
>
> I'm not asking what the concepts of INTERNAL TABLE and EXTERNAL TABLE in
> hive, I'm just confused what does the value "EXTERNAL_TABLE" in class
> org.apache.hadoop.hive.TableType mean?
>
>
> I create a table setting table type TableType.EXTERNAL_TABLE by calling
> hive metastore api like this:
>
>
> final HiveMetaStoreClient client = new HiveMetaStoreClient(c);
>
> Table newTable = new Table();
>
> newTable.setTableType(TableType.EXTERNAL_TABLE.toString());
>
> ...
>
> client.createTable(newTable);
>
>
> then I describe the created table with hive metastore api:
>
> client.getTable("dbName", "tableName");
>
>
> I can see that the table type is still "MANAGED_TABLE", is this a bug of
> metastore api?
>
>
>
>
>
> --
> *发件人:* Mich Talebzadeh 
> *发送时间:* 2016年12月2日 5:29
> *收件人:* user
> *主题:* Re: Difference between MANAGED_TABLE and EXTERNAL_TABLE in
> org.apache.hadoop.hive.metastore.TableType
>
> Adding to Alan's points external tables are used often as a staging area.
> For example, ingesting data from HDFS location on a daily basis and putting
> that data into Hive managed tables. That location of that external table
> can change pointing to a new HDFS directory created by say Flume etc
> through ALTER table like below
>
> ALTER TABLE ${DATABASE}.EXTERNALMARKETDATA set location
> 'hdfs://rhes564:9000/data/prices/${TODAY}';
>
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * 
> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> *
>
>
>
> http://talebzadehmich.wordpress.com
> 
> Mich Talebzadeh 
> talebzadehmich.wordpress.com
> Technical Architecture, Big Data, Oracle, Sybase, CEP, IMDB and Data Grid
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 1 December 2016 at 17:53, Alan Gates  wrote:
>
>> Hive does not assume that it owns the data for an external table.  Thus
>> when an external table is dropped, the data is not deleted.  People often
>> use this as a way to load data into a directory in HDFS and then “cast” a
>> table structure over it by creating an external table with that directory
>> as its location.
>>
>> Alan.
>>
>> > On Dec 1, 2016, at 06:15, Huang Meilong  wrote:
>> >
>> > Hi all,
>> >
>> > I found an enum TableType in package org.apache.hadoop.hive.metastore.
>> What's the difference between MANAGED_TABLE and EXTERNAL_TABLE?
>> >
>> > Will the table be an EXTERNAL TABLE with setting table type
>> EXTERNAL_TABLE when creating table?
>> >
>> > I found the code to determine whether a table is an external table in
>> MetaStoreUtils.java
>> > https://github.com/apache/hive/blob/master/metastore/src/
>> java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L1425
>> >
>> > I'm confused what is EXTERNAL_TABLE in TableType for?
>>
>>
>


答复: Difference between MANAGED_TABLE and EXTERNAL_TABLE in org.apache.hadoop.hive.metastore.TableType

2016-12-01 Thread Huang Meilong
Thanks for your detailed explanations.


I'm not asking what the concepts of INTERNAL TABLE and EXTERNAL TABLE in hive, 
I'm just confused what does the value "EXTERNAL_TABLE" in class 
org.apache.hadoop.hive.TableType mean?


I create a table setting table type TableType.EXTERNAL_TABLE by calling hive 
metastore api like this:


final HiveMetaStoreClient client = new HiveMetaStoreClient(c);

Table newTable = new Table();

newTable.setTableType(TableType.EXTERNAL_TABLE.toString());

...

client.createTable(newTable);


then I describe the created table with hive metastore api:

client.getTable("dbName", "tableName");


I can see that the table type is still "MANAGED_TABLE", is this a bug of 
metastore api?






发件人: Mich Talebzadeh 
发送时间: 2016年12月2日 5:29
收件人: user
主题: Re: Difference between MANAGED_TABLE and EXTERNAL_TABLE in 
org.apache.hadoop.hive.metastore.TableType

Adding to Alan's points external tables are used often as a staging area. For 
example, ingesting data from HDFS location on a daily basis and putting that 
data into Hive managed tables. That location of that external table can change 
pointing to a new HDFS directory created by say Flume etc through ALTER table 
like below

ALTER TABLE ${DATABASE}.EXTERNALMARKETDATA set location 
'hdfs://rhes564:9000/data/prices/${TODAY}';


HTH


Dr Mich Talebzadeh



LinkedIn  
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com

[https://secure.gravatar.com/blavatar/eeed9e1445477d955b8dd2470c042649?s=200&ts=1480663659]

Mich Talebzadeh
talebzadehmich.wordpress.com
Technical Architecture, Big Data, Oracle, Sybase, CEP, IMDB and Data Grid




Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
damage or destruction of data or any other property which may arise from 
relying on this email's technical content is explicitly disclaimed. The author 
will in no case be liable for any monetary damages arising from such loss, 
damage or destruction.



On 1 December 2016 at 17:53, Alan Gates 
mailto:alanfga...@gmail.com>> wrote:
Hive does not assume that it owns the data for an external table.  Thus when an 
external table is dropped, the data is not deleted.  People often use this as a 
way to load data into a directory in HDFS and then “cast” a table structure 
over it by creating an external table with that directory as its location.

Alan.

> On Dec 1, 2016, at 06:15, Huang Meilong 
> mailto:ims...@outlook.com>> wrote:
>
> Hi all,
>
> I found an enum TableType in package org.apache.hadoop.hive.metastore. What's 
> the difference between MANAGED_TABLE and EXTERNAL_TABLE?
>
> Will the table be an EXTERNAL TABLE with setting table type EXTERNAL_TABLE 
> when creating table?
>
> I found the code to determine whether a table is an external table in 
> MetaStoreUtils.java
> https://github.com/apache/hive/blob/master/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java#L1425
>
> I'm confused what is EXTERNAL_TABLE in TableType for?