[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-13596: -- Labels: (was: TODOC2.1) > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.1.0 > > Attachments: HIVE-13596.01.patch, HIVE-13596.02.patch, > HIVE-13596.03.patch, HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-13596: -- Labels: TODOC2.1 (was: ) > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Labels: TODOC2.1 > Fix For: 2.1.0 > > Attachments: HIVE-13596.01.patch, HIVE-13596.02.patch, > HIVE-13596.03.patch, HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Committed to master - thanks for the review! > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 2.1.0 > > Attachments: HIVE-13596.01.patch, HIVE-13596.02.patch, > HIVE-13596.03.patch, HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Attachment: HIVE-13596.03.patch > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13596.01.patch, HIVE-13596.02.patch, > HIVE-13596.03.patch, HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Attachment: HIVE-13596.02.patch Updated the patch to add it to the system registry instead, after some discussion. I changed the synchronized methods to use a lock object without any logic changes wrt locking, and then took the MS call (and resource acquisition) out of the lock to avoid bottlenecking on the system registry. > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13596.01.patch, HIVE-13596.02.patch, > HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Attachment: HIVE-13596.01.patch > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13596.01.patch, HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Attachment: HIVE-13596.patch This restores some of the functionality, with a config flag, and accounting for other registry changes. [~jdere] can you take a look? > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin > Attachments: HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Assignee: Sergey Shelukhin Status: Patch Available (was: Open) > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-13596.patch > > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Description: When multiple HS2s are run, creating a permanent fn is only executed on one of them, and the other HS2s don't get the new function. Unlike say with tables, where we always get stuff from db on demand, fns are registered at certain points in the code and if the new one is not registered, it will not be available. We should restore the pre-HIVE-2573 behavior of being able to refresh the UDFs on demand. was: When multiple HS2s are run, creating a permanent fn is only executed on one of them, and the other HS2s don't get the new function. Unlike say with tables, where we always get stuff from db on demand, fns are registered at certain points in the code and if the new one is not registered, it will not be available. We could change the code to refresh the udf by name if it's missing, similar to getting a table or whatever; or we could refresh UDFs when a session is started in multi-HS2 case, or at some other convenient point. > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We should restore the pre-HIVE-2573 behavior of being able to refresh the > UDFs on demand. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13596) HS2 should be able to get UDFs on demand from metastore
[ https://issues.apache.org/jira/browse/HIVE-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-13596: Summary: HS2 should be able to get UDFs on demand from metastore (was: HS2 should refresh UDFs more frequently(?), at least in multi-HS2 case) > HS2 should be able to get UDFs on demand from metastore > --- > > Key: HIVE-13596 > URL: https://issues.apache.org/jira/browse/HIVE-13596 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin > > When multiple HS2s are run, creating a permanent fn is only executed on one > of them, and the other HS2s don't get the new function. Unlike say with > tables, where we always get stuff from db on demand, fns are registered at > certain points in the code and if the new one is not registered, it will not > be available. > We could change the code to refresh the udf by name if it's missing, similar > to getting a table or whatever; or we could refresh UDFs when a session is > started in multi-HS2 case, or at some other convenient point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
