The error you posted originally is from deserializing a storage plugin config, not the drill-override or drill-module files. That being said, we find the list of available storage plugins using classpath scanning.
I don't know why the inclusion or exclusion of contents inside of a module.conf file would impact our scanning for storage plugins, but I believe this is the issue you are seeing. Somehow the package that contains the hbase plugin is being removed from the list of packages to scan when you add the empty file. Do you want to jump on slack to chat about this? - Jason On Mon, Jan 11, 2016 at 10:52 AM, rahul challapalli < [email protected]> wrote: > Julien, > > I have an empty drill-module.conf file in the udf jar file. Below are the > contents of my global drill-override.conf file > > drill.classpath.scanning.packages : ${?drill.classpath.scanning.packages} [ > org.apache.drill.udfs ] > drill.exec: { > cluster-id: "rahul_cluster_com-drillbits", > zk.connect: "localhost:5181" > } > > With this, the drillbit fails to start with the error I posted in the first > email. May be I am getting the syntax wrong? > > The reason I am insisting an empty drill-module.conf file is because, udf's > developed prior to the "classpath scanning change" had an empty > drill-module.conf file and we used to override the udf package name in the > global conf file. > > - Rahul > > On Mon, Jan 11, 2016 at 10:46 AM, Julien Le Dem <[email protected]> wrote: > > > Yes I believe that should work: > > - add an empty drill-module.conf in the root of the udf jar > > - add the package to drill.classpath.scanning.packages in the drill conf > > (possibly using drill-override.conf) > > > > However if you are adding the drill-module.conf file to the jar, you > might > > as well add the package in it. (unless there's some other reason) > > > > On Mon, Jan 11, 2016 at 10:28 AM, rahul challapalli < > > [email protected]> wrote: > > > > > Just to be sure, If I have an empty drill-module.conf in the root of my > > udf > > > jar, then there is no way to add the package information to the global > > > drill-override.conf file? > > > > > > On Mon, Jan 11, 2016 at 10:26 AM, Julien Le Dem <[email protected]> > > wrote: > > > > > > > You are correct: > > > > The jar containing the UDFs should have a drill-module.conf at the > root > > > > adding your package to the property drill.classpath.scanning.packages > > for > > > > scanning > > > > drill.classpath.scanning.packages : > > > ${?drill.classpath.scanning.packages} [ > > > > my.package.containing.my.udfs > > > > ] > > > > Jars that don't contain a drill-module.conf will not get scanned. > > > > > > > > > > > > > > > > On Mon, Jan 11, 2016 at 10:17 AM, rahul challapalli < > > > > [email protected]> wrote: > > > > > > > > > Thanks for your reply Jason. > > > > > > > > > > If we cannot override the global configuration file, then for > > existing > > > > > UDF's we have to re-compile them by modifying the drill-module.conf > > > file. > > > > > If so our UDF's are not backward compatible. Appreciate it if > someone > > > can > > > > > confirm this. > > > > > > > > > > - Rahul > > > > > > > > > > On Mon, Jan 11, 2016 at 9:59 AM, Jason Altekruse < > > > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > Rahul, > > > > > > > > > > > > The error message you are seeing is in reading a storage plugin > > > > > > configuration file. I am planning to fix these kinds of messages > to > > > > > > actually direct users at the file that is failing parsing. I have > > > seen > > > > > this > > > > > > in the past when the classpath was incorrect and one of the > plugins > > > > (like > > > > > > Hbase) was not included. > > > > > > > > > > > > Julien can confirm, but I think this might be intentional to have > > the > > > > > paths > > > > > > read out of the modules configuration rather than the global one > to > > > > save > > > > > > time when scanning the path (rather than scanning all of the jars > > for > > > > all > > > > > > paths given in the override file). > > > > > > > > > > > > On Fri, Jan 8, 2016 at 4:32 PM, rahul challapalli < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > Before 1.2, my udfs project contained an empty > > drill-override.conf > > > > file > > > > > > and > > > > > > > I used to update the drill-override.conf on all the drillbits > to > > > > > specify > > > > > > > the package of my UDF. This is no longer working for me. I > tried > > a > > > > few > > > > > > > things and below is how my drill-override.conf file looks now > > > > > > > > > > > > > > drill.classpath.scanning.packages : > > > > > > ${?drill.classpath.scanning.packages} [ > > > > > > > org.apache.drill.udfs ] > > > > > > > drill.exec: { > > > > > > > cluster-id: "rahul_cluster_com-drillbits", > > > > > > > zk.connect: "localhost:5181" > > > > > > > } > > > > > > > > > > > > > > When I restart the drillbits, I get this strange error " Caused > > by: > > > > > > > com.fasterxml.jackson.databind.JsonMappingException: Could not > > > > resolve > > > > > > type > > > > > > > id 'hbase' into a subtype of [simple type, class > > > > > > > org.apache.drill.common.logical.StoragePluginConfig]" > > > > > > > > > > > > > > If I moved the package information to the drill-module.conf in > my > > > > udf's > > > > > > > project, then things are working fine. However this requires > > > > > re-compiling > > > > > > > the udfs which is not desirable. Is there any other way around > > > this ? > > > > > > > > > > > > > > - Rahul > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Julien > > > > > > > > > > > > > > > -- > > Julien > > >
