[
https://issues.apache.org/jira/browse/DRILL-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16633753#comment-16633753
]
ASF GitHub Bot commented on DRILL-6762:
---------------------------------------
arina-ielchiieva commented on issue #1482: DRILL-6762: Dynamic UDFs registered
on 1 Drillbit not visible on others
URL: https://github.com/apache/drill/pull/1482#issuecomment-425840592
Closing the PR, since https://github.com/apache/drill/pull/1484 was opened
which contains full fix of the problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Dynamic UDFs registered on one Drillbit are not visible on other Drillbits
> --------------------------------------------------------------------------
>
> Key: DRILL-6762
> URL: https://issues.apache.org/jira/browse/DRILL-6762
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.13.0
> Reporter: Kunal Khatua
> Assignee: Arina Ielchiieva
> Priority: Critical
> Fix For: 1.15.0
>
> Attachments: Dynamic UDFs issue.pdf
>
>
> Originally Reported :
> https://stackoverflow.com/questions/52480160/dynamic-udf-in-apache-drill-cluster
> When using a 4-node Drill 1.14 cluster, UDF jars registered on one node are
> not usable on other nodes despite the {{/registry}} and ZK showing the UDFs
> as registered.
> This was previously working on 1.13.0
> *Root cause*
> 1. {{VersionedDelegatingStore}} was starting with version -1 and local
> function registry with version 0. This caused issues when
> {{LocalPersistentStore}} already existed on the file system. When adding jars
> into remote registry its versioned was bumped to 0 and synchronization did
> not happen since local registry had the same version.
> *Fix*: start {{VersionedDelegatingStore}} with version 0, local function
> registry with undefined version (-2) thus first sync will always happen.
> 2. {{PersistentStoreProvider.getOrCreateVersionedStore}} was wrapping stores
> into VersionedDelegatingStore for those store providers that did not override
> this method. Only Zookeeper store was overriding it. But
> {{VersionedDelegatingStore}} is only keeps versioning locally and thus can be
> applied only for local stores, i.e. Hbase, Mongo cannot use it.
> {{CachingPersistentStoreProvider}} did not override
> {{getOrCreateVersionedStore}} either. Mostly all stores in Drill are created
> using {{CachingPersistentStoreProvider}}. Thus all stores where wrapped into
> {{VersionedDelegatingStore}}, even Zookeeper one which caused function
> registries version synchronization issues.
> *Fix*: Add {{UndefinedVersionDelegatingStore}} for those stores that do not
> support versioning and wrap into it by default in
> {{PersistentStoreProvider.getOrCreateVersionedStore}} if this method is not
> overriden. {{UndefinedVersionDelegatingStore}} will return UNDEFINED version
> (-2). During sync between remote and local registries if remote registry has
> UNDEFINED version sync will be done immediately, on the contrary with
> NOT_AVAILABLE version (-1) which indicates that remote function registry is
> not accessible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
