PK violation during Hive add partition

[Karlen Lie]

Hello,

We are running into intermittent errors while running the below query.  Some 
background on this, our table (tbl_someTable) that we're altering is an 
external table, and the query below is run concurrently by multiple oozie 
workflows.

ALTER TABLE tbl_someTable ADD IF NOT EXISTS PARTITION(cluster_address = 
'${CLUSTERADDRESS}', upload_date = '${PREVIOUSDATE}' , upload_hour = 
'${PREVIOUSHOUR}')
LOCATION 
'asv://${RAWLOGSCONTAINER}/${CLUSTERADDRESS}/someLog/${PREVIOUSDATE}/${PREVIOUSHOUR}';

The errors we're getting are below.

Is this a known issue and is there a workaround for it?

Thanks
karlen

stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use 
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in 
jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/5662320028645753518_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history 
file=/tmp/dssxuser/hive_job_log_dssxuser_201212070113_1149932084.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object 
"org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f<mailto:org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f>"
 using statement "INSERT INTO PARTITIONS 
(PART_ID,CREATE_TIME,SD_ID,PART_NAME,LAST_ACCESS_TIME,TBL_ID) VALUES 
(?,?,?,?,?,?)" failed : Violation of PRIMARY KEY constraint 
&apos;PK_partitions_PART_ID&apos;. Cannot insert duplicate key in object 
&apos;dbo.PARTITIONS&apos;. The duplicate key value is (221).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY 
constraint &apos;PK_partitions_PART_ID&apos;. Cannot insert duplicate key in 
object &apos;dbo.PARTITIONS&apos;. The duplicate key value is (221).
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], 
exit code [9]


stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use 
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in 
jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/2751940372978647467_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/dssxuser/hive_job_log_dssxuser_201212071515_173032638.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object 
"org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5<mailto:org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5>"
 using statement "INSERT INTO SERDES (SERDE_ID,SLIB,"NAME") VALUES (?,?,?)" 
failed : Violation of PRIMARY KEY constraint &apos;PK_serdes_SERDE_ID&apos;. 
Cannot insert duplicate key in object &apos;dbo.SERDES&apos;. The duplicate key 
value is (2006).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY 
constraint &apos;PK_serdes_SERDE_ID&apos;. Cannot insert duplicate key in 
object &apos;dbo.SERDES&apos;. The duplicate key value is (2006).
FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], 
exit code [9]