[
https://issues.apache.org/jira/browse/HIVE-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355394#comment-14355394
]
Alan Gates commented on HIVE-9874:
----------------------------------
Actually, the above causes problems not just because of the shared storage
descriptors, but also because the hbase metastore is caching tables. The above
works with ObjectStore because each call to ms.getTable() returns a new table
object. HBaseStore returns the cached object again and again, so doing
{code}
while ... {
t = ms.getTable();
p.setSd(t.getSd());
p.getSd().setLocation(...);
}
{code}
causes problems.
This patch actually needs to fix three issues:
# This re-use of table storage descriptors in partitions without copying;
# HBaseStore is currently attaching the storage descriptor from HBase directly
to the table and partitions, which is wrong. It should instead be attaching a
SharedStorageDescriptor with a unique location and a reference to the storage
descriptor from HBase.
# The SharedStorageDescriptor needs copy-on-write semantics so that when a user
does {{table.getSd().setInputFormat("something different")}} it only changes
the storage descriptor for that table rather than for everything that shares a
storage descriptor with that table (eg all its partitions).
> Partition storage descriptors being set from table sd without copying
> [hbase-metastore branch]
> ----------------------------------------------------------------------------------------------
>
> Key: HIVE-9874
> URL: https://issues.apache.org/jira/browse/HIVE-9874
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Alan Gates
> Assignee: Alan Gates
>
> There are a number of places in the code where something like the following
> is done:
> {code}
> partition.setSd(table.getSd());
> {code}
> This causes problems in the HBase metastore case because of the way it shares
> storage descriptors when they are identical. This means when using a storage
> descriptor as a template for another we need to actually create a new one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
