Re: Storm PROD Server log folder configuration issue

[Harsha]

Hi Yiming,

           This is a known bug in 0.9.2
. [1]https://issues.apache.org/jira/browse/STORM-279.

The bug was that supervisor not forwarding
storm.server.log.path opt to the worker.

[2]https://github.com/apache/incubator-storm/commit/598acf97109
20028ed0c240dc6add02a895f2f48#diff-8a8d97993ededcb27c19504b9e88
9e6f . From 0.9.3 users can define STORM_LOG_DIR and all the
logs will be in that location. By default this would be under
STORM_HOME/logs.

-Harsha





On Fri, Aug 15, 2014, at 08:32 AM, Fang, Yiming  wrote:

Hi All,



I am currently working on a PROD server setup task on storm
0.9.2 .

Trying to configure storm server log folder inside storm python
script:



def exec_storm_class(klass, jvmtype="-server", jvmopts=[],
extrajars=[], args=[], fork=False):

    global CONFFILE

    all_args = [

        "java", jvmtype, get_config_opts(),

        "-Dstorm.home=" + STORM_DIR,

"-Dstorm.server.log.path=" + STORM_SERVER_LOG_PATH,

        "-Djava.library.path=" + confvalue("java.library.path",
extrajars),

        "-Dstorm.conf.file=" + CONFFILE,

        "-cp", get_classpath(extrajars),

    ] + jvmopts + [klass] + list(args)

    print "Running: " + " ".join(all_args)

    if fork:

        os.spawnvp(os.P_WAIT, "java", all_args)

    else:

        os.execvp("java", all_args) # replaces the current
process and never returns



I pass STORM_SERVER_LOG_PATHas system env parameter and then
inside cluster.xml I

Replace the file with new config

<configurationscan="true"scanPeriod="60 seconds">

<appender name="A1"
class="ch.qos.logback.core.rolling.RollingFileAppender">

<file>${storm.server.log.path}/logs/${logfile.name}</file>



Result:

I could have all following logs in new place:

access.log, metrics.log, ui.log, nimbus.log, supervisor.log



but the worker-6702 6703 log just in the original server
location at the following place

when I setup my topology:



bash-3.2$ cd /opt/gpf/realtime/storm/0.9.2/bin

bash-3.2$ ls

storm  storm.cmd  storm-config.cmd
storm.server.log.path_IS_UNDEFINED

bash-3.2$ cd storm.server.log.path_IS_UNDEFINED/

bash-3.2$ ls

logs

bash-3.2$ cd logs

bash-3.2$ ls

access.log  metrics.log  worker-6702.log  worker-6703.log



Can anyone help?



Thanks and regards,

Yiming

References

1. https://issues.apache.org/jira/browse/STORM-279
2. 
https://github.com/apache/incubator-storm/commit/598acf9710920028ed0c240dc6add02a895f2f48#diff-8a8d97993ededcb27c19504b9e889e6f