How’s about this? The proposed structure is: $DRILL_CONF_DIR |- drill-override.conf |- drill-env.sh |- jars |- your.jar
So, could you replace jars with a link to the (shared) code? $DRILL_CONF_DIR |- … |- jars —> /your/shared/code With YARN, the step to archive the site directory will follow sym links. With non-YARN, sym links should “just work.” Will this work for the Mesos case? - Paul > On May 29, 2016, at 6:41 PM, John Omernik <[email protected]> wrote: > > I am all for this, it would make Mesos easier as well. (I do have to merge > the 3rd party code and the libjpam.so into my archive at upgrade) . That > said, I'd be interested in the choice to merge a conf directory with > code. Personally, putting on my admin hat, I'd like the ability to add a > Code directory, and have that location be seperate from my conf directory. > I may have different qa processes, change management processes, etc between > the two. I am very in favor of what you are proposing, I would just prefer > it being a separate option to append to my 3rd party jar search locations. > Please let me know if I am not being clear, I am on my iPad, and my > thoughts always appear more jumbled (more than usual) when I reread my iPad > posts. I blame Woz. > > John > > On Sunday, May 29, 2016, Paul Rogers <[email protected]> wrote: > >> Hi All, >> >> The discussion with John and Charles about the drillbit scripts reminded >> me to get your thoughts on another change we’re working on. >> >> Today, drillbit.sh has the --config option so you can put your config >> files in a location separate from DRILL_HOME: >> >> $DRILL_HOME/bin/drillbit.sh —config /some/path/to/conf start >> >> This is handy, but it only holds config files (drill-env.sh, >> drill-override.sh). If you have custom code, it still must go into >> $DRILL_HOME/jars/3rdparty. >> >> This presents two challenges: >> >> * On upgrades, you have to grab your files from the old $DRILL_HOME and >> copy them into the new one. >> * With YARN, we have to create an archive of your entire $DRILL_HOME just >> to grab your “site” files. >> >> So, we propose to extend the —config option to include code as well as >> config. We call this “complete” set of files the “site” directory (using >> Hadoop terminology.) (See DRILL-4591.) This way: >> >> * Upgrade is easy, throw away the old $DRILL_HOME and extract the Drill >> archive to create the new one. >> * With YARN, we upload the “stock” drill archive plus your (much smaller) >> site files. >> * We can more easily support multiple Drill “clusters” (each with its own >> site files, including assigned ports.) >> >> With YARN, you only need one copy of the DRILL_HOME and site directory; >> YARN copies (“localizes”) the files to all your worker nodes. Without YARN, >> you have to do the copy, probably with your favorite system admin tool. >> >> So, the question is this: is the site directory a help for those of you >> that won’t be using YARN? Or, does everyone just copy site files from one >> DRILL_HOME to the next on upgrade, then push the merged directory to all >> your worker nodes? >> >> Thoughts? >> >> Thanks, >> >> - Paul >> >> > > -- > Sent from my iThing
