I am all for this, it would make Mesos easier as well. (I do have to merge the 3rd party code and the libjpam.so into my archive at upgrade) . That said, I'd be interested in the choice to merge a conf directory with code. Personally, putting on my admin hat, I'd like the ability to add a Code directory, and have that location be seperate from my conf directory. I may have different qa processes, change management processes, etc between the two. I am very in favor of what you are proposing, I would just prefer it being a separate option to append to my 3rd party jar search locations. Please let me know if I am not being clear, I am on my iPad, and my thoughts always appear more jumbled (more than usual) when I reread my iPad posts. I blame Woz.
John On Sunday, May 29, 2016, Paul Rogers <[email protected]> wrote: > Hi All, > > The discussion with John and Charles about the drillbit scripts reminded > me to get your thoughts on another change we’re working on. > > Today, drillbit.sh has the --config option so you can put your config > files in a location separate from DRILL_HOME: > > $DRILL_HOME/bin/drillbit.sh —config /some/path/to/conf start > > This is handy, but it only holds config files (drill-env.sh, > drill-override.sh). If you have custom code, it still must go into > $DRILL_HOME/jars/3rdparty. > > This presents two challenges: > > * On upgrades, you have to grab your files from the old $DRILL_HOME and > copy them into the new one. > * With YARN, we have to create an archive of your entire $DRILL_HOME just > to grab your “site” files. > > So, we propose to extend the —config option to include code as well as > config. We call this “complete” set of files the “site” directory (using > Hadoop terminology.) (See DRILL-4591.) This way: > > * Upgrade is easy, throw away the old $DRILL_HOME and extract the Drill > archive to create the new one. > * With YARN, we upload the “stock” drill archive plus your (much smaller) > site files. > * We can more easily support multiple Drill “clusters” (each with its own > site files, including assigned ports.) > > With YARN, you only need one copy of the DRILL_HOME and site directory; > YARN copies (“localizes”) the files to all your worker nodes. Without YARN, > you have to do the copy, probably with your favorite system admin tool. > > So, the question is this: is the site directory a help for those of you > that won’t be using YARN? Or, does everyone just copy site files from one > DRILL_HOME to the next on upgrade, then push the merged directory to all > your worker nodes? > > Thoughts? > > Thanks, > > - Paul > > -- Sent from my iThing
