[
https://issues.apache.org/jira/browse/DRILL-8474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17810070#comment-17810070
]
ASF GitHub Bot commented on DRILL-8474:
---------------------------------------
cgivre commented on PR #2836:
URL: https://github.com/apache/drill/pull/2836#issuecomment-1906689793
> > > @cgivre @paul-rogers is there an example of a Drill UDF that is not
part of the drill repository tree?
> > > I'd like to understand the mechanisms for distributing any jar files
and dependencies of the UDF that drill uses. I can't find any such in the
quasi-USFs that are in the Drill tree, because well, since they are part of
Drill, and so are their dependencies, this problem doesn't exist.
> >
> >
> > @mbeckerle Here's an example:
https://github.com/datadistillr/drill-humanname-functions. I'm sorry we weren't
able to connect last week.
>
> If I understand this correctly, if a jar is on the classpath and has
drill-module.conf in its root dir, then drill will find it and read that HOCON
file to get the package to add to drill.classpath.scanning.packages.
I believe that is correct.
>
> Drill then appears to scan jars for class files for those packages. Not
sure what it is doing with the class files. I imagine it is repackaging them
somehow so Drill can use them on the drill distributed nodes. But it isn't yet
clear to me how this aspect works. Do these classes just get loaded on the
distributed drill nodes? Or is the classpath augmented in some way on the drill
nodes so that they see a jar that contains all these classes?
>
> I have two questions:
>
> (1) what about dependencies? The UDF may depend on libraries which depend
on other libraries, etc.
So UDFs are a bit of a special case, but if they do have dependencies, you
have to also include those JAR files in the UDF directory, or in Drill's 3rd
party JAR folder. I'm not that good with maven, but I've often wondered about
making a so-called fat-JAR which includes the dependencies as part of the UDF
JAR file.
>
> (2) what about non-class files, e.g., things under src/main/resources of
the project that go into the jar, but aren't "class" files? How do those things
also get moved? How would code running in the drill node access these? The
usual method is to call getResource(URL) with a URL that gives the path within
a jar file to the resource in question.
Take a look at this UDF.
https://github.com/datadistillr/drill-geoip-functions
This UDF has a few external resources including a CSV file and the MaxMind
databases.
>
> Thanks for any info.
> Add Daffodil Format Plugin
> --------------------------
>
> Key: DRILL-8474
> URL: https://issues.apache.org/jira/browse/DRILL-8474
> Project: Apache Drill
> Issue Type: New Feature
> Affects Versions: 1.21.1
> Reporter: Charles Givre
> Priority: Major
> Fix For: 1.22.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
