Hi Carl
To reproduce it, I added one line
throw new MetaException("test");
at line 628 in ObjectStore.java and comment the following line
//commited = commitTransaction();
I also added some codes for tracing the behaviors of
open/commit/rollback transaction and executeWithRetry.
Here are the results I saw in my console:
hive>
> create table test(str string);
12/04/20 11:01:18 ERROR metastore.HiveMetaStore: execute with retry: 0
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: opening
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter commit transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction
12/04/20 11:01:18 ERROR metastore.ObjectStore: rolling back
12/04/20 11:01:18 ERROR metastore.ObjectStore: enter rollback transaction
FAILED: Error in metadata: MetaException(message:test)
12/04/20 11:01:18 ERROR exec.DDLTask: FAILED: Error in metadata:
MetaException(message:test)
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:test)
And then I use "describe test;", I got this:
hive> describe test;
12/04/20 11:11:31 ERROR metastore.HiveMetaStore: execute with retry: 0
12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:11:31 ERROR metastore.ObjectStore: opening
12/04/20 11:11:31 ERROR metastore.ObjectStore: enter open transaction
12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction
12/04/20 11:11:31 ERROR metastore.ObjectStore: enter commit transaction
12/04/20 11:11:31 ERROR metastore.ObjectStore: commiting
OK
str string
Time taken: 0.132 seconds
It seems the table has been created successfully although
the transaction has never been committed and has been rolled back.
I think this could relate to JDO and MySQL.
Because I cannot reproduce it when using Derby.
My MySQL version is 5.1.18
Thanks.
Kai Zhang
2012/4/20 Carl Steinbach <[email protected]>
> Hi Kai,
>
> I agree that this sounds like a bug. Can you describe the steps required
> to reproduce this behavior on 0.7.1?
>
> Thanks.
>
> Carl
>
>
> On Wed, Apr 18, 2012 at 11:03 PM, Zhang Kai <[email protected]> wrote:
>
>> Hi all,
>>
>> I got a problem when using Hive 0.7.1.
>> After I created a table with JDO failure, I can still find the table by
>> using 'show tables;'.
>>
>> I grep the code and find there is a transaction mechanism when hive
>> interacts with metastore.
>> However, it seems that the transaction doesn't work.
>>
>> I have tested this for several times.
>> I manually throw an exception before ObjectStore commit the 'CREATE
>> TABLE' transaction.
>> But I can find that a new record in TBLS in mysql.
>> So I guess meta data didn't rollback correctly.
>>
>> I also notice this could also happen on Hive 0.8 and 0.9.
>> However, I haven't tested it.
>>
>> I think this is a fatal error and can make hive metastore unavailable.
>> Is there anyone can help me solve this problem?
>>
>> Thanks.
>>
>> Kai Zhang
>>
>
>
