what I don't understand is the substitution in general. Why have
export SERVER_GC_OPTS=${SERVER_GC_OPTS/"-Xloggc:<FILE-
PATH>"/"-Xloggc:${loggc}"}
instead of
export SERVER_GC_OPTS="${SERVER_GC_OPTS/} -Xloggc:${loggc}"
The latter seems much more straight forward and understandable, and less
prone to odd ball issues. Maybe, one other if test to ensure that ${loggc}
is set as well.
if [ -n "$SERVER_GC_OPTS" && -n "${loggc}" ]; then
export SERVER_GC_OPTS="${SERVER_GC_OPTS/} -Xloggc:${loggc}"
fi
I guess I am just a big fan of simplification...
On Mon, May 30, 2016 at 5:01 PM, Paul Rogers <[email protected]> wrote:
> Hi John,
>
> The Drill scripts need quite a bit of TLC. (See DRILL-4581.)
> drill-config.sh tries to set up both the Drillbit (server) and sqlline
> (client). Work was needed to fully separate the two. The CLIENT_GC_OPTS are
> only for sqlline, SERVER_GC_OPTS are for the drillbit.
>
> The problem is that SERVER_GC_OPTS does two things that conflict. If it
> only did logging, it would work:
>
> $ loggc=/foo/bar.log
> $ export SERVER_GC_OPTS="-Xloggc:<FILE-PATH>”
> $ echo ${SERVER_GC_OPTS/"-Xloggc:<FILE-PATH>"/"-Xloggc:${loggc}”}
> -Xloggc:/foo/bar.log
>
> But, current version of drill-env.sh helpfully adds other stuff to
> SERVER_GC_OPTS, which makes the substitution fail:
>
> export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled -XX:+UseG1GC "
>
> Sigh… More bugs to fix… I’ve added this issue as a comment to DRILL-4581.
>
> For now, just work around the problem using DRILL_JAVA_OPTS. The following
> exists today in drill-env.sh:
>
> export DRILL_JAVA_OPTS="-Xms$DRILL_HEAP -Xmx$DRILL_HEAP…
>
> Add another line:
>
> export DRILL_JAVA_OPTS=“$DRILL_JAVA_OPTS -Xloggc:/path/to/gc.log"
>
> You’ll have to specify the log path, but it sounds like you do that anyway
> for your Mesos setup.
>
> By the way, another change we’re making for DoY is to split drill-env.sh
> into three parts: Drill defaults move into drill-config.sh,
> distribution-specific stuff moves into its own file, and drill-env.sh will
> contain only site-specific settings.
>
> - Paul
>
> > On May 30, 2016, at 5:28 AM, John Omernik <[email protected]> wrote:
> >
> > More importantly, I am not sure how the strings inside the curly braces
> > actually works either, based on testing... (echoing out the
> SERVER_GC_OPTS
> > and CLIENT_GC_OPTS) It's not actually working
> >
> > If I am reading the bash correctly, than it's looking to, if
> SERVER_GC_OPTS
> > (or CLIENT) is set (-n = return true if the length of the string is
> > nonzero, since the Variable is interpreted, we are checking wether there
> is
> > something in the variable) then we should be adding the xloggc (both of
> > them) to the SERVER_GC_OPTS (and client).
> >
> > As you can see with the testing, the SERVER_GC_OPTS is only the value
> that
> > I am setting from my drill-env.sh (default setting) which is loaded by
> > drill-config.sh sourced earlier in the drillbit.sh. Thus, this code in
> > drillbit.sh is effectively doing nothing ... I guess my thought process
> > here would be to have someone help decide what is intended here, (I am
> not
> > sure "Nothing" is intended based on the amount of code) and then we can
> do
> > some updating here to clarify and ensure efficacy.
> >
> >
> > Testing:
> >
> > if [ -n "$SERVER_GC_OPTS" ]; then
> >
> > export
> SERVER_GC_OPTS=${SERVER_GC_OPTS/"-Xloggc:<FILE-PATH>"/"-Xloggc:${loggc}"}
> >
> > fi
> >
> > if [ -n "$CLIENT_GC_OPTS" ]; then
> >
> > export
> CLIENT_GC_OPTS=${CLIENT_GC_OPTS/"-Xloggc:<FILE-PATH>"/"-Xloggc:${loggc}"}
> >
> > fi
> >
> > echo "Server: $SERVER_GC_OPTS"
> >
> > echo "Client: $CLIENT_GC_OPTS"
> >
> > exit 1
> >
> >
> > Server: -XX:+CMSClassUnloadingEnabled -XX:+UseG1GC
> >
> > Client:
> >
> > On Mon, May 30, 2016 at 6:43 AM, John Omernik <[email protected]> wrote:
> >
> >> So based on Paul's drilbit.sh comment and this, I decided to go ensure I
> >> was enabling the proper GC logging because I am skipping the
> drillbit.sh.
> >> I looked at the drillbit.sh, and frankly, It looks like a goofy error
> may
> >> be in that... the <FILE-PATH> seems to be in documentation for other
> >> hadoop-ish projects, but I don't think Java or BASH does anything with
> it.
> >> Thus having that in the drillbit.sh (which to me shouldn't be changed)
> >> seems to be a mistake... (Yes the -Xloggc after it may just overwrite
> what
> >> was passed in the <FILE-PATH> but am I correct in saying that this is
> >> actually just a mistake that in the drillbit.sh, and all it does is add
> >> confusion? I hope I am wrong here and I get to learn something, but I
> ust
> >> can't see how <FILE-PATH> is interpreted by bash or java....
> >>
> >>
> >> John
> >>
> >>
> >>
> >> if [ -n "$SERVER_GC_OPTS" ]; then
> >>
> >> export SERVER_GC_OPTS=${SERVER_GC_OPTS/"-Xloggc:<FILE-PATH>"/
> >> "-Xloggc:${loggc}"}
> >>
> >> fi
> >>
> >> if [ -n "$CLIENT_GC_OPTS" ]; then
> >>
> >> export CLIENT_GC_OPTS=${CLIENT_GC_OPTS/"-Xloggc:<FILE-PATH>"/
> >> "-Xloggc:${loggc}"}
> >>
> >> fi
> >>
> >> On Mon, May 30, 2016 at 3:42 AM, Ted Dunning <[email protected]>
> >> wrote:
> >>
> >>> On Sun, May 29, 2016 at 2:29 PM, John Omernik <[email protected]>
> wrote:
> >>>
> >>>> (It's a very weird situation that the bits get into,
> >>>> everything hangs, somethings work, other things seem to be a in
> >>>> an in-between between working and not working etc. Like describe
> table
> >>>> operations eventually return but after 10+seconds. I resolve this by
> >>>> restarting all bits, and then things are right as rain.
> >>>>
> >>>
> >>> Sounds like GC pressure, possibly.
> >>>
> >>> The GC logging that was mentioned in connection with drill.sh would be
> >>> helpful here.
> >>>
> >>
> >>
>
>
