[
https://issues.apache.org/jira/browse/DRILL-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464657#comment-15464657
]
ASF GitHub Bot commented on DRILL-4726:
---------------------------------------
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/574#discussion_r77498708
--- Diff: protocol/src/main/protobuf/UserBitShared.proto ---
@@ -298,3 +298,17 @@ enum CoreOperatorType {
NESTED_LOOP_JOIN = 35;
AVRO_SUB_SCAN = 36;
}
+
+message Func {
+ optional string name = 1;
+ repeated common.MajorType major_type = 2;
+}
+
+message Jar {
+ optional string name = 1;
+ repeated Func function = 2;
+}
+
+message Registry {
--- End diff --
Registry is stored at Zookeeper. Unfortunately, we can't consider different
structure, since each time we try to register UDF we need all registry to
validate against it and since we don't impose any locks on registry during
validation, we use ZK versioning feature. Basically, we take registry from ZK
with its version, validate against it, update it and try to store updated
version in ZK. If registry version has changed by that time, we re-validate
again (see 5.1.3 Registration). Versioning only works for object stored under
znode but not on child znodes. I have tried to add in remote registry only
required information to reduce it's size (jar name, function name, list of in
parameters). Eventually, the size of registry might grow too big, in this case
we expect user periodically move verified dynamic UDFs to static UDFs, remove
unused UDFs.
> Dynamic UDFs support
> --------------------
>
> Key: DRILL-4726
> URL: https://issues.apache.org/jira/browse/DRILL-4726
> Project: Apache Drill
> Issue Type: New Feature
> Affects Versions: 1.6.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Allow register UDFs without restart of Drillbits.
> Design is described in document below:
> https://docs.google.com/document/d/1FfyJtWae5TLuyheHCfldYUpCdeIezR2RlNsrOTYyAB4/edit?usp=sharing
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
