Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/574#discussion_r77498708
--- Diff: protocol/src/main/protobuf/UserBitShared.proto ---
@@ -298,3 +298,17 @@ enum CoreOperatorType {
NESTED_LOOP_JOIN = 35;
AVRO_SUB_SCAN = 36;
}
+
+message Func {
+ optional string name = 1;
+ repeated common.MajorType major_type = 2;
+}
+
+message Jar {
+ optional string name = 1;
+ repeated Func function = 2;
+}
+
+message Registry {
--- End diff --
Registry is stored at Zookeeper. Unfortunately, we can't consider different
structure, since each time we try to register UDF we need all registry to
validate against it and since we don't impose any locks on registry during
validation, we use ZK versioning feature. Basically, we take registry from ZK
with its version, validate against it, update it and try to store updated
version in ZK. If registry version has changed by that time, we re-validate
again (see 5.1.3 Registration). Versioning only works for object stored under
znode but not on child znodes. I have tried to add in remote registry only
required information to reduce it's size (jar name, function name, list of in
parameters). Eventually, the size of registry might grow too big, in this case
we expect user periodically move verified dynamic UDFs to static UDFs, remove
unused UDFs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
