[
https://issues.apache.org/jira/browse/KYLIN-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359948#comment-15359948
]
Richard Calaba edited comment on KYLIN-1830 at 7/2/16 3:18 AM:
---------------------------------------------------------------
Okay - fair enough - but I prefer having configuration parameters for sizing in
one location (/conf ) as we there other performance and execution env. related
settings.
Thus I have found for you a generic solution you can use for ALL variables in
setenv.sh - using this solution you can override the parameters from setenv.sh
by values specified in the conf/kylin.properties file using same name of the
environment variable.
See attached & updated setenv.sh and kylin.properties files (both from Kylin
1.5.2.1).
------------ Background info about implementation:
The trick is that I have defined 2 bash functions which try to read the
property value override from conf/kylin.properties - and if found - it uses the
value specified there ; if not found (or the value override is commented by #)
then it uses the default value as specified in the bin/setenv.sh script.
Feel free to include it to the standard packaging - was thinking to provide
patch here - but didn't find setenv.sh in source code - probably generated ?
Functions Defined - see setenv.sh attachment.
Example from setenv.sh - how the override of the variable KYLIN_JVM_SETTINGS:
Instead of:
export KYLIN_JVM_SETTINGS="<default_value>"
Use:
export_property_override KYLIN_JVM_SETTINGS "<default_value>"
I.e:
export_property_override KYLIN_JVM_SETTINGS "-Xms1024M -Xmx4096
-XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
Then in the conf/kylin.properties you can put line:
KYLIN_JVM_SETTINGS="-Xms1024M -Xmx16g -XX:MaxPermSize=512M -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$
-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
Which will override the default value provided in setenv.sh
Tested and working. Enjoy!
was (Author: [email protected]):
Okay - fair enough - but I prefer having configuration parameters for sizing in
one location (/conf ) as we there other performance and execution env. related
settings.
Thus I have found for you a generic solution you can use for ALL variables in
setenv.sh - using this solution you can override the parameters from setenv.sh
by values specified in the conf/kylin.properties file using same name of the
environment variable.
See attached & updated setenv.sh and kylin.properties files (both from Kylin
1.5.2.1).
------------ Background info about implementation:
The trick is that I have defined 2 bash functions which try to read the
property value override from conf/kylin.properties - and if found - it uses the
value specified there ; if not found (or the value override is commented by #)
then it uses the default value as specified in the bin/setenv.sh script.
Feel free to include it to the standard packaging - was thinking to provide
patch here - but didn't find setenv.sh in source code - probably generated ?
Functions Defined:
function parse_properties() {
local param_value=$(awk -F "=" "(!/^(\$|[[:space:]]*#)/) && (/$2/) {
idx = index(\$0,\"=\"); print substr(\$0,idx+1)}" "$1" | sed -e 's/^"//' -e
's/"$//')
if [[ -z "${param_value}" ]]; then
echo $3
else
echo `eval echo "${param_value}"`
fi
}
function export_property_override() {
# default Kylin property file location - for environment values override
local kylin_property_file=${KYLIN_HOME}/conf/kylin.properties
export "$1"="$(parse_properties "${kylin_property_file}" "$1" "$2" )"
}
# Example from setenv.sh - how the override of the variable KYLIN_JVM_SETTINGS:
Instead of:
export KYLIN_JVM_SETTINGS="<default_value>"
Use:
export_property_override KYLIN_JVM_SETTINGS "<default_value>"
I.e:
export_property_override KYLIN_JVM_SETTINGS "-Xms1024M -Xmx4096
-XX:MaxPermSize=512M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
Then in the conf/kylin.properties you can put line:
KYLIN_JVM_SETTINGS="-Xms1024M -Xmx16g -XX:MaxPermSize=512M -verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$
-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
Which will override the default value provided in setenv.sh
Tested and working. Enjoy!
> Put KYLIN_JVM_SETTINGS to kylin.properties
> ------------------------------------------
>
> Key: KYLIN-1830
> URL: https://issues.apache.org/jira/browse/KYLIN-1830
> Project: Kylin
> Issue Type: Improvement
> Reporter: Richard Calaba
> Priority: Minor
> Attachments: kylin.properties, setenv.sh
>
>
> Currently is the KYLIN_JVM_SETTINGS variable stored in the ,/bin/setenv.sh
> ... which is not wrong, but as we have also some other memory specific
> setting in ./conf/kylin.properties file (like i.e
> kylin.job.mapreduce.default.reduce.input.mb or kylin.table.snapshot.max_mb)
> it might be good idea to have those performance and sizing related parameters
> in one location.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
