Julien,
I have an empty drill-module.conf file in the udf jar file. Below are the
contents of my global drill-override.conf file
drill.classpath.scanning.packages : ${?drill.classpath.scanning.packages} [
org.apache.drill.udfs ]
drill.exec: {
cluster-id: "rahul_cluster_com-drillbits",
zk.connect: "localhost:5181"
}
With this, the drillbit fails to start with the error I posted in the first
email. May be I am getting the syntax wrong?
The reason I am insisting an empty drill-module.conf file is because, udf's
developed prior to the "classpath scanning change" had an empty
drill-module.conf file and we used to override the udf package name in the
global conf file.
- Rahul
On Mon, Jan 11, 2016 at 10:46 AM, Julien Le Dem <[email protected]> wrote:
> Yes I believe that should work:
> - add an empty drill-module.conf in the root of the udf jar
> - add the package to drill.classpath.scanning.packages in the drill conf
> (possibly using drill-override.conf)
>
> However if you are adding the drill-module.conf file to the jar, you might
> as well add the package in it. (unless there's some other reason)
>
> On Mon, Jan 11, 2016 at 10:28 AM, rahul challapalli <
> [email protected]> wrote:
>
> > Just to be sure, If I have an empty drill-module.conf in the root of my
> udf
> > jar, then there is no way to add the package information to the global
> > drill-override.conf file?
> >
> > On Mon, Jan 11, 2016 at 10:26 AM, Julien Le Dem <[email protected]>
> wrote:
> >
> > > You are correct:
> > > The jar containing the UDFs should have a drill-module.conf at the root
> > > adding your package to the property drill.classpath.scanning.packages
> for
> > > scanning
> > > drill.classpath.scanning.packages :
> > ${?drill.classpath.scanning.packages} [
> > > my.package.containing.my.udfs
> > > ]
> > > Jars that don't contain a drill-module.conf will not get scanned.
> > >
> > >
> > >
> > > On Mon, Jan 11, 2016 at 10:17 AM, rahul challapalli <
> > > [email protected]> wrote:
> > >
> > > > Thanks for your reply Jason.
> > > >
> > > > If we cannot override the global configuration file, then for
> existing
> > > > UDF's we have to re-compile them by modifying the drill-module.conf
> > file.
> > > > If so our UDF's are not backward compatible. Appreciate it if someone
> > can
> > > > confirm this.
> > > >
> > > > - Rahul
> > > >
> > > > On Mon, Jan 11, 2016 at 9:59 AM, Jason Altekruse <
> > > [email protected]
> > > > >
> > > > wrote:
> > > >
> > > > > Rahul,
> > > > >
> > > > > The error message you are seeing is in reading a storage plugin
> > > > > configuration file. I am planning to fix these kinds of messages to
> > > > > actually direct users at the file that is failing parsing. I have
> > seen
> > > > this
> > > > > in the past when the classpath was incorrect and one of the plugins
> > > (like
> > > > > Hbase) was not included.
> > > > >
> > > > > Julien can confirm, but I think this might be intentional to have
> the
> > > > paths
> > > > > read out of the modules configuration rather than the global one to
> > > save
> > > > > time when scanning the path (rather than scanning all of the jars
> for
> > > all
> > > > > paths given in the override file).
> > > > >
> > > > > On Fri, Jan 8, 2016 at 4:32 PM, rahul challapalli <
> > > > > [email protected]> wrote:
> > > > >
> > > > > > Before 1.2, my udfs project contained an empty
> drill-override.conf
> > > file
> > > > > and
> > > > > > I used to update the drill-override.conf on all the drillbits to
> > > > specify
> > > > > > the package of my UDF. This is no longer working for me. I tried
> a
> > > few
> > > > > > things and below is how my drill-override.conf file looks now
> > > > > >
> > > > > > drill.classpath.scanning.packages :
> > > > > ${?drill.classpath.scanning.packages} [
> > > > > > org.apache.drill.udfs ]
> > > > > > drill.exec: {
> > > > > > cluster-id: "rahul_cluster_com-drillbits",
> > > > > > zk.connect: "localhost:5181"
> > > > > > }
> > > > > >
> > > > > > When I restart the drillbits, I get this strange error " Caused
> by:
> > > > > > com.fasterxml.jackson.databind.JsonMappingException: Could not
> > > resolve
> > > > > type
> > > > > > id 'hbase' into a subtype of [simple type, class
> > > > > > org.apache.drill.common.logical.StoragePluginConfig]"
> > > > > >
> > > > > > If I moved the package information to the drill-module.conf in my
> > > udf's
> > > > > > project, then things are working fine. However this requires
> > > > re-compiling
> > > > > > the udfs which is not desirable. Is there any other way around
> > this ?
> > > > > >
> > > > > > - Rahul
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Julien
> > >
> >
>
>
>
> --
> Julien
>
