[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13548051#comment-13548051 ] Hudson commented on HIVE-3443: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3443. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key (Shreepadma Venugopalan via cws) (Revision 1388098) Backout HIVE-3443 (cws) (Revision 1386857) HIVE-3443. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key (Shreepadma Venugopalan via cws) (Revision 1386782) Result = ABORTED cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1388098 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1386857 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1386782 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Fix For: 0.10.0 Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt, HIVE-3443.4.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13460241#comment-13460241 ] Hudson commented on HIVE-3443: -- Integrated in Hive-trunk-h0.21 #1687 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1687/]) HIVE-3443. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key (Shreepadma Venugopalan via cws) (Revision 1388098) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1388098 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Fix For: 0.10.0 Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt, HIVE-3443.4.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458029#comment-13458029 ] Shreepadma Venugopalan commented on HIVE-3443: -- Updated patch available on both JIRA and review board. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt, HIVE-3443.4.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458126#comment-13458126 ] Carl Steinbach commented on HIVE-3443: -- +1 Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt, HIVE-3443.4.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13457237#comment-13457237 ] Carl Steinbach commented on HIVE-3443: -- +1. Will commit if tests pass. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13457328#comment-13457328 ] Hudson commented on HIVE-3443: -- Integrated in Hive-trunk-h0.21 #1676 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1676/]) HIVE-3443. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key (Shreepadma Venugopalan via cws) (Revision 1386782) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1386782 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13457583#comment-13457583 ] Hudson commented on HIVE-3443: -- Integrated in Hive-trunk-h0.21 #1678 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1678/]) Backout HIVE-3443 (cws) (Revision 1386857) Result = FAILURE cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1386857 Files : * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java * /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/tools/HiveMetaTool.java * /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaTool.java Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13455004#comment-13455004 ] Shreepadma Venugopalan commented on HIVE-3443: -- Updated patch available for review at : https://reviews.apache.org/r/6962/ Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt, HIVE-3443.3.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452326#comment-13452326 ] Jakob Homan commented on HIVE-3443: --- I don't believe Apache Hive is obligated to support automating this relatively simple task. Are you also going to change to the Serde class, since its name changed? Are you going to update input and outputformat at the same time? People writing tools on github (including me) that eventually get merged in some format to the main Hive branch do not obligate Hive to support the old github version in any way. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452350#comment-13452350 ] Shreepadma Venugopalan commented on HIVE-3443: -- @Jakob: This tool has been added to support migration to HA NN for Hive customers. I'm not sure I understand what you mean by change to the Sedere class since its name changed. Hive Metatool replies on the key schema.url to update avro schema url records that contain a reference to the NN. Given the recent change from schema.url to avro.schema.url, unless Hive Metatool allows the user to pass in the serdeParamKey, it can't reliably update all metastore records to refer the the right NN. Failing to update some of the metastore records will render some tables in the warehouse inaccessible. This is in no way a minor bug. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Minor Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452358#comment-13452358 ] Jakob Homan commented on HIVE-3443: --- You're attempting to introduce code designed to support the change from Haivvreo to AvroSerde. Once again, it is not Hive's responsibility to support non-ASF code like Haivvreo. If you want this functionality, it should be added to Haivvreo (and I'd be happy to review any pull requests there), not Hive. If users chose to use Haivvreo (or had it via vendor-supplied packages), it's their (or their vendor's) responsibility to update as necessary. There is no need to add code or complexity to Hive to support non-ASF code. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452362#comment-13452362 ] Carl Steinbach commented on HIVE-3443: -- @Jakob: the code that we're trying to add isn't specific to Haivvreo or the AvroSerDe, but making the transition easier is one of the use cases we are trying to address. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[
https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452367#comment-13452367
]
Jakob Homan commented on HIVE-3443:
---
{code}+if (key != null) {
+ LOG.info(Looking for location in the value field of + key + key in
SERDES table...);
+} else {
+ LOG.info(Looking for location in the value field of
schema.url/avro.schema.url key in +
+ SERDES table...);
+}{code}
Seems pretty Haivvreo or AvroSerDe specific...
Hive Metatool should take serde_param_key from the user to allow for changes
to avro serde's schema url key
---
Key: HIVE-3443
URL: https://issues.apache.org/jira/browse/HIVE-3443
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.10.0
Reporter: Shreepadma Venugopalan
Assignee: Shreepadma Venugopalan
Priority: Critical
Attachments: HIVE-3443.1.patch.txt
Hive Metatool should take serde_param_key from the user to allow for chanes
to avro serde's schema url key. In the past avro.schema.url key used to be
called schema.url.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452370#comment-13452370 ] Carl Steinbach commented on HIVE-3443: -- Thanks for pointing that out. Shreepadma, please remove the AvroSerDe/Haivvreo specific stuff from this patch. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452372#comment-13452372 ] Shreepadma Venugopalan commented on HIVE-3443: -- I'm not adding any code that is specific to Haivvreo to Hive, nor is this patch supporting the change from Haivvreo to AvroSerde. Note that this tool is specifically intended for migration to HA and **not** to support change from Haivvreo to AvroSerde. This patch makes the metatool more flexible and generic to allow the users to update both Haivvreo and AvroSerde specific metadata that resides in the metastore. If its not Hive's responsibility to support non-ASF code such as Haivvreo, then Haivvreo specific property such as schema.url shouldn't be past of the metastore in the first place. Given that it is and users need a way to migrate to HA, there is not a better and a convenient way to do this migration other than make Metatool flexible enough to take in serdeParamKey from the user. This change is fixing what you are specifically objecting to i.e., Hive shouldn't know about Haivvreo. With this patch, Hive Metatool will be oblivious to whether its updating Haivvreo or AvroSerde metadata. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452376#comment-13452376 ] Shreepadma Venugopalan commented on HIVE-3443: -- @Carl, Jakob: Note that the code Jakob is pointing to is from the patch I uploaded prior to this discussion. I've not uploaded the latest patch yet, but it doesn't have anything that is Avro specific. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452564#comment-13452564 ] Shreepadma Venugopalan commented on HIVE-3443: -- Updated patch available for review at : https://reviews.apache.org/r/6962/ Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452616#comment-13452616 ] Carl Steinbach commented on HIVE-3443: -- Comments on reviewboard. Please change the status to 'patch available' when this ticket is ready for review. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt, HIVE-3443.2.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451695#comment-13451695 ] Shreepadma Venugopalan commented on HIVE-3443: -- @Carl: Good catch. Yup, looks like we have to take in tableProp instead of serdeProp. I need to look into this a little more carefully but what I have in mind at this time is, metatool -updateLocation new-HA-NN old-NN -tableProp propKey=propValue I think in addition to addressing the avro use case, it will make the tool a lot more generic. However, we also have to consider users who are using schema.url as a serde property at this time. I think its safe to attempt an update of serde prop schema.url irrespective of whether the user passes in the tableProp. What are your thoughts on this? Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451706#comment-13451706 ] Carl Steinbach commented on HIVE-3443: -- Are users required to the specify the propValue field? If someone is trying to replace the NN address for a bunch of Avro tables, the actual propValue is going to be slightly different for each table since it includes the path as well as the host. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451718#comment-13451718 ] Shreepadma Venugopalan commented on HIVE-3443: -- @Carl: StorageDescriptor has a param key/value pair. When I say propValue I mean avro.schema.url i.e., value of the field param key in storagedescriptor. I guess this is confusing. How about -tableProp propKey where a valid propKey is avro.schema.url? Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451724#comment-13451724 ] Carl Steinbach commented on HIVE-3443: -- Yup, thanks for clarifying. That makes sense. bq. I think its safe to attempt an update of serde prop schema.url irrespective of whether the user passes in the tableProp. If the table or partition record is using the Haivvreo SerDe and has a schema.url property defined, then I agree that it's probably safe to update this field even if the user didn't specify it. However, one of the points Jakob made in his comment was that the name schema.url is pretty generic, and there's the chance that other SerDes may use the same key for other purposes. I think it would be safer all around to force the user to manually specify the serdeproperties and tableproperties that they want to update. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451725#comment-13451725 ] Shreepadma Venugopalan commented on HIVE-3443: -- @Carl: Are you suggesting that metatool should take in the serdeParamKey in addition to tableProp to update potential serde location? Just so that I've understood you are suggesting the following, metatool new-NN-loc old-NN-loc -tableProp param_key -serdeParamKey param_key Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451726#comment-13451726 ] Carl Steinbach commented on HIVE-3443: -- Yup, I think that makes sense. Is there another approach that might work better? Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13451729#comment-13451729 ] Shreepadma Venugopalan commented on HIVE-3443: -- I think this approach may be the safest, even though users have to be aware of the fact that avro serde's schema url property is stored as table prop and other serde props are stored as serde props. I can't think of an approach that is cleaner. If you have other suggestions, I'm open to hearing them. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Attachments: HIVE-3443.1.patch.txt Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3443) Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
[ https://issues.apache.org/jira/browse/HIVE-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13450242#comment-13450242 ] Shreepadma Venugopalan commented on HIVE-3443: -- Support for Hive MetaTool was added in HIVE-3056. Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key --- Key: HIVE-3443 URL: https://issues.apache.org/jira/browse/HIVE-3443 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Priority: Critical Hive Metatool should take serde_param_key from the user to allow for chanes to avro serde's schema url key. In the past avro.schema.url key used to be called schema.url. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
