[
https://issues.apache.org/jira/browse/HIVE-25648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LiuJun updated HIVE-25648:
--------------------------
Description:
{code:java}
// source code in HiveMetaStoreClient.java -- createTable func
public void createTable(Table tbl, EnvironmentContext envContext) throws
AlreadyExistsException,
InvalidObjectException, MetaException, NoSuchObjectException, TException {
if (!tbl.isSetCatName()) {
tbl.setCatName(getDefaultCatalog(conf));
}
HiveMetaHook hook = getHook(tbl);
if (hook != null) {
hook.preCreateTable(tbl);
}
boolean success = false;
try {
// Subclasses can override this step (for example, for temporary tables)
create_table_with_environment_context(tbl, envContext); *//create
metadata record*
if (hook != null) {
hook.commitCreateTable(tbl); *//create table in external catalog*
}
success = true;
}
finally {
if (!success && (hook != null)) {
try {
* // roll back from external catalog but without roll back from hive meta*
hook.rollbackCreateTable(tbl);
} catch (Exception e){
LOG.error("Create rollback failed with", e);
}
}
}
}
{code}
Accoriding to the source code above, when implementing hivemetastore's
HiveMetaHook to create external catalog tables(may be hbase)，firstly create
meta records to the database such as pg, then call the commitCreateTable
function to create table in hbase. Here comes the question: What if exception
thrown when creating the real table in hbase, because meta data has been
created so it is not in sync between Hive's metastore and hbase.
I think it is necessary to rollback metadata from hivemetastore when
failed to create table in external catalog by calling commitCreateTable, so
that we can keep external catalog in sync with Hive's metastore.
Please let me know if my idea is correct or I had an misunderstanding on
how to use the HiveMetaHook mechanism correctly!
was:
{code:java}
// source code in HiveMetaStoreClient.java -- createTable func
public void createTable(Table tbl, EnvironmentContext envContext) throws
AlreadyExistsException,
InvalidObjectException, MetaException, NoSuchObjectException, TException {
if (!tbl.isSetCatName()) {
tbl.setCatName(getDefaultCatalog(conf));
}
HiveMetaHook hook = getHook(tbl);
if (hook != null) {
hook.preCreateTable(tbl);
}
boolean success = false;
try {
// Subclasses can override this step (for example, for temporary tables)
create_table_with_environment_context(tbl, envContext); *//create
metadata record*
if (hook != null) {
hook.commitCreateTable(tbl); *//create table in external catalog*
}
success = true;
}
finally {
if (!success && (hook != null)) {
try {
* // roll back from external catalog but without roll back from hive meta*
hook.rollbackCreateTable(tbl);
} catch (Exception e){
LOG.error("Create rollback failed with", e);
}
}
}
}
{code}
Accoriding to the source code above, when implementing hivemetastore's
HiveMetaHook to create external catalog tables(may be hbase)，firstly create
meta records to the database such as pg, then call the commitCreateTable
function to create table in hbase. Here comes the question: What if exception
thrown when creating the real table in hbase, because meta data has been
created so it is not in sync between Hive's metastore and hbase.
I think it is necessary to rollback metadata from hivemetastore when
failed to create table in external catalog by calling commitCreateTable, so
that we can keep external catalog in sync with Hive's metastore.
Please let me know if my idea is correct or I had an misunderstanding on
how to use the HiveMetaHook mechanism correctly!
Summary: HiveMetaHook not work well in HiveMetaStoreClient when
commitCreateTable table failed! (was: Hook not work when commitdrop table
failed)
> HiveMetaHook not work well in HiveMetaStoreClient when commitCreateTable
> table failed!
> --------------------------------------------------------------------------------------
>
> Key: HIVE-25648
> URL: https://issues.apache.org/jira/browse/HIVE-25648
> Project: Hive
> Issue Type: Bug
> Components: API, Hooks, Standalone Metastore
> Affects Versions: 3.1.2
> Reporter: LiuJun
> Priority: Major
>
> {code:java}
> // source code in HiveMetaStoreClient.java -- createTable func
> public void createTable(Table tbl, EnvironmentContext envContext) throws
> AlreadyExistsException,
> InvalidObjectException, MetaException, NoSuchObjectException,
> TException {
> if (!tbl.isSetCatName()) {
> tbl.setCatName(getDefaultCatalog(conf));
> }
> HiveMetaHook hook = getHook(tbl);
> if (hook != null) {
> hook.preCreateTable(tbl);
> }
> boolean success = false;
> try {
> // Subclasses can override this step (for example, for temporary tables)
> create_table_with_environment_context(tbl, envContext); *//create
> metadata record*
> if (hook != null) {
> hook.commitCreateTable(tbl); *//create table in external catalog*
> }
> success = true;
> }
> finally {
> if (!success && (hook != null)) {
> try {
> * // roll back from external catalog but without roll back from hive meta*
> hook.rollbackCreateTable(tbl);
> } catch (Exception e){
> LOG.error("Create rollback failed with", e);
> }
> }
> }
> }
> {code}
> Accoriding to the source code above, when implementing hivemetastore's
> HiveMetaHook to create external catalog tables(may be hbase)，firstly create
> meta records to the database such as pg, then call the commitCreateTable
> function to create table in hbase. Here comes the question: What if exception
> thrown when creating the real table in hbase, because meta data has been
> created so it is not in sync between Hive's metastore and hbase.
> I think it is necessary to rollback metadata from hivemetastore when
> failed to create table in external catalog by calling commitCreateTable, so
> that we can keep external catalog in sync with Hive's metastore.
> Please let me know if my idea is correct or I had an misunderstanding on
> how to use the HiveMetaHook mechanism correctly!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
