[
https://issues.apache.org/jira/browse/DRILL-4726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15468944#comment-15468944
]
ASF GitHub Bot commented on DRILL-4726:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/574#discussion_r77737314
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillFunctionRegistry.java
---
@@ -64,62 +76,134 @@
.put("CONVERT_FROM", Pair.of(2, 2))
.put("FLATTEN", Pair.of(1, 1)).build();
+ /** Registers all functions present in Drill classpath on start-up. All
functions will be marked as built-in.*/
public DrillFunctionRegistry(ScanResult classpathScan) {
+ validate(BUILT_IN, classpathScan);
+ register(BUILT_IN, classpathScan, this.getClass().getClassLoader());
+ if (logger.isTraceEnabled()) {
+ StringBuilder allFunctions = new StringBuilder();
+ for (DrillFuncHolder method:
registryHolder.getAllFunctionsWithHolders().values()) {
+ allFunctions.append(method.toString()).append("\n");
+ }
+ logger.trace("Registered functions: [\n{}]", allFunctions);
+ }
+ }
+
+ /**
+ * Validates all functions, present in jars.
+ * Will throw {@link FunctionValidationException} if:
+ * 1. Jar with the same name has been already registered.
+ * 2. Conflicting function with the similar signature is found.
+ * 3. Aggregating function is not deterministic.
+ *
+ * @return list of validated functions
+ */
+ public List<Func> validate(String jarName, ScanResult classpathScan) {
+ List<Func> functions = Lists.newArrayList();
FunctionConverter converter = new FunctionConverter();
List<AnnotatedClassDescriptor> providerClasses =
classpathScan.getAnnotatedClasses();
- // Hash map to prevent registering functions with exactly matching
signatures
- // key: Function Name + Input's Major Type
- // value: Class name where function is implemented
- //
- final Map<String, String> functionSignatureMap = new HashMap<>();
+ if (registryHolder.containsJar(jarName)) {
+ throw new FunctionValidationException(String.format("Jar %s is
already registered", jarName));
+ }
+
+ final ListMultimap<String, String> allFuncWithSignatures =
registryHolder.getAllFunctionsWithSignatures();
+
for (AnnotatedClassDescriptor func : providerClasses) {
- DrillFuncHolder holder = converter.getHolder(func);
+ DrillFuncHolder holder = converter.getHolder(func,
ClassLoader.getSystemClassLoader());
if (holder != null) {
- // register handle for each name the function can be referred to
- String[] names = holder.getRegisteredNames();
- // Create the string for input types
String functionInput = "";
+ List<MajorType> types = Lists.newArrayList();
for (DrillFuncHolder.ValueReference ref : holder.parameters) {
functionInput += ref.getType().toString();
+ types.add(ref.getType());
}
+
+ String[] names = holder.getRegisteredNames();
for (String name : names) {
String functionName = name.toLowerCase();
- registeredFunctions.put(functionName, holder);
String functionSignature = functionName + functionInput;
- String existingImplementation;
- if ((existingImplementation =
functionSignatureMap.get(functionSignature)) != null) {
- throw new AssertionError(
- String.format(
- "Conflicting functions with similar signature found.
Func Name: %s, Class name: %s " +
- " Class name: %s", functionName, func.getClassName(),
existingImplementation));
- } else if (holder.isAggregating() && !holder.isDeterministic() )
{
- logger.warn("Aggregate functions must be deterministic, did
not register function {}", func.getClassName());
+
+ if
(allFuncWithSignatures.get(functionName).contains(functionSignature)) {
--- End diff --
Similar race condition. Getting the list of function names & signatures is
thread-safe. But, iterating though the list is not: two threads may do the same
functions at the same time, resulting in conflicts.
> Dynamic UDFs support
> --------------------
>
> Key: DRILL-4726
> URL: https://issues.apache.org/jira/browse/DRILL-4726
> Project: Apache Drill
> Issue Type: New Feature
> Affects Versions: 1.6.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Fix For: Future
>
>
> Allow register UDFs without restart of Drillbits.
> Design is described in document below:
> https://docs.google.com/document/d/1FfyJtWae5TLuyheHCfldYUpCdeIezR2RlNsrOTYyAB4/edit?usp=sharing
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
