Dzahn has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392079 )
Change subject: Gerrit: Fix up logstash configuation ...................................................................... Gerrit: Fix up logstash configuation Switch to using socket instead of tcp. Using socket over tcp works. Logstash side can parse the info it gets. Tested locally on: http://gerrit-logstash.wmflabs.org/ Also switch from properties syntax to xml syntax. xml log4j syntax has more advanced features which we want to use. Also include configuation updates to improve the logging. As it was logging alot of useless things. Also we have to make sure gc_log and sshd_log and error_log are created now as gerrit was prevously doing that in java, but some how using a custom log4j file disables it gerrit's side so log4j will throw a error but gerrit will still start, it just wont log. Also use java_options from jetty.pp which will allow us to define the command once in puppet and then it's added to gerrit.config and gerrit.service thus making the files look pretty as they wont have long lines Bug: T141324 Change-Id: I76245284285c6eb2377df23f1a75284dc0616d36 --- M modules/gerrit/manifests/jetty.pp M modules/gerrit/templates/initscripts/gerrit.systemd.erb D modules/gerrit/templates/log4j.properties.erb A modules/gerrit/templates/log4j.xml.erb 4 files changed, 189 insertions(+), 110 deletions(-) Approvals: jenkins-bot: Verified Dzahn: Looks good to me, approved diff --git a/modules/gerrit/manifests/jetty.pp b/modules/gerrit/manifests/jetty.pp index e4e7d40..dc35759 100644 --- a/modules/gerrit/manifests/jetty.pp +++ b/modules/gerrit/manifests/jetty.pp @@ -52,6 +52,8 @@ $ldap_proxyagent_pass = $ldapconfig['proxypass'] $java_options = [ + "-Xmx${heap_limit} -Xms${heap_limit}", + '-Dlog4j.configuration=file:///var/lib/gerrit2/review_site/etc/log4j.xml', # These settings apart from the bottom control logging for gc # '-Xloggc:/srv/gerrit/jvmlogs/jvm_gc.%p.log', # '-XX:+PrintGCApplicationStoppedTime', @@ -62,7 +64,6 @@ # '-XX:+UseGCLogFileRotation', # '-XX:NumberOfGCLogFiles=10', # '-XX:GCLogFileSize=2M', - # '-Dlog4j.configuration=file:///var/lib/gerrit2/review_site/etc/log4j.properties', ] require_package([ @@ -245,8 +246,8 @@ require => File['/var/lib/gerrit2/review_site/etc'], } - file { '/var/lib/gerrit2/review_site/etc/log4j.properties': - content => template('gerrit/log4j.properties.erb'), + file { '/var/lib/gerrit2/review_site/etc/log4j.xml': + content => template('gerrit/log4j.xml.erb'), owner => 'gerrit2', group => 'gerrit2', mode => '0444', @@ -277,6 +278,13 @@ require => File['/var/lib/gerrit2/review_site/etc'], } + file { '/var/lib/gerrit2/review_site/logs': + ensure => directory, + owner => 'gerrit2', + group => 'gerrit2', + mode => '0755', + } + file { '/var/lib/gerrit2/review_site/static': ensure => directory, recurse => remote, @@ -284,6 +292,7 @@ group => 'gerrit2', mode => '0444', source => 'puppet:///modules/gerrit/static', + require => File['/var/lib/gerrit2/review_site'], } file { '/var/lib/gerrit2/review_site/plugins': diff --git a/modules/gerrit/templates/initscripts/gerrit.systemd.erb b/modules/gerrit/templates/initscripts/gerrit.systemd.erb index 13e2b3a..9860c60 100644 --- a/modules/gerrit/templates/initscripts/gerrit.systemd.erb +++ b/modules/gerrit/templates/initscripts/gerrit.systemd.erb @@ -7,7 +7,7 @@ Group=gerrit2 Type=simple EnvironmentFile=/etc/default/gerrit -ExecStart=<%= @java_home %>/bin/java -Xmx<%= @heap_limit %> -Xms<%= @heap_limit %> -jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site<%- if @slave -%> --slave<% end %> +ExecStart=<%= @java_home %>/bin/java <% @java_options.each do |jvm_opt| %><%= jvm_opt %> <% end %> -jar /var/lib/gerrit2/review_site/bin/gerrit.war daemon -d /var/lib/gerrit2/review_site<%- if @slave -%> --slave<% end %> KillSignal=SIGINT # NOFILE : GERRIT_FDS, determined by "core.packedGitOpenFiles" in the script LimitNOFILE=<%= @git_open_files %> diff --git a/modules/gerrit/templates/log4j.properties.erb b/modules/gerrit/templates/log4j.properties.erb deleted file mode 100644 index 429bba2..0000000 --- a/modules/gerrit/templates/log4j.properties.erb +++ /dev/null @@ -1,106 +0,0 @@ -# Copyright (C) 2008 The Android Open Source Project -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -log4j.rootLogger=DEBUG, error_log<% if @log_host %>, tcp<% end %> -log4j.logger.com.google.gerrit.sshd.SshLog=DEBUG, sshd_log -log4j.logger.com.google.gerrit.pgm.http.jetty.HttpLog=DEBUG, httpd_log -log4j.logger.gc_log=DEBUG, gc_log -# Prevent gc_log from outputting to error_log -log4j.additivity.gc_log=false -<%- if @log_host -%> - -### Logstash -log4j.appender.tcp=org.apache.log4j.net.SocketAppender -log4j.appender.tcp.Port=<%= @log_port %> -log4j.appender.tcp.RemoteHost=<%= @log_host %> -log4j.appender.tcp.ReconnectionDelay=500 -log4j.appender.tcp.Application=gerrit -<%- end -%> - -# Configure log appenders as defaults -log4j.appender.error_log=org.apache.log4j.DailyRollingFileAppender -log4j.appender.error_log.File=/var/lib/gerrit2/review_site/logs/error_log -log4j.appender.error_log.layout=org.apache.log4j.PatternLayout -log4j.appender.error_log.layout.ConversionPattern=[%d] %-5p %c %x: %m%n -log4j.appender.error_log.encoding=UTF-8 -log4j.appender.error_log.threshold=INFO -log4j.appender.error_log.immediateFlush=true -log4j.appender.error_log.fileAppend=true -log4j.appender.httpd_log=org.apache.log4j.DailyRollingFileAppender -log4j.appender.httpd_log.File=/var/lib/gerrit2/review_site/logs/httpd_log -log4j.appender.httpd_log.layout=com.google.gerrit.pgm.http.jetty.HttpLogLayout -log4j.appender.httpd_log.encoding=UTF-8 -log4j.appender.httpd_log.threshold=INFO -log4j.appender.httpd_log.immediateFlush=true -log4j.appender.httpd_log.fileAppend=true -log4j.appender.sshd_log=org.apache.log4j.DailyRollingFileAppender -log4j.appender.sshd_log.File=/var/lib/gerrit2/review_site/logs/sshd_log -log4j.appender.sshd_log.layout=com.google.gerrit.sshd.SshLogLayout -log4j.appender.sshd_log.encoding=UTF-8 -log4j.appender.sshd_log.threshold=INFO -log4j.appender.sshd_log.immediateFlush=true -log4j.appender.sshd_log.fileAppend=true -log4j.appender.gc_log=org.apache.log4j.DailyRollingFileAppender -log4j.appender.gc_log.File=/var/log/gerrit/gc_log -log4j.appender.gc_log.layout=org.apache.log4j.PatternLayout -log4j.appender.gc_log.layout.ConversionPattern=[%d] %-5p %x: %m%n -log4j.appender.gc_log.encoding=UTF-8 -log4j.appender.gc_log.threshold=INFO -log4j.appender.gc_log.immediateFlush=true -log4j.appender.gc_log.fileAppend=true - -log4j.logger.com.google.gerrit=INFO - - -# Silence non-critical messages from MINA SSHD. -# -log4j.logger.org.apache.mina=WARN -log4j.logger.org.apache.sshd.common=WARN -log4j.logger.org.apache.sshd.server=WARN -# To be removed when we upgrade to gerrit 2.14. -log4j.logger.org.apache.sshd.common.keyprovider.AbstractFileKeyPairProvider=INFO -# Replaces the above -log4j.logger.org.apache.sshd.common.keyprovider.FileKeyPairProvider=INFO -log4j.logger.com.google.gerrit.sshd.GerritServerSession=WARN - -# Silence non-critical messages from mime-util. -# -log4j.logger.eu.medsea.mimeutil=WARN - -# Silence non-critical messages from openid4java -# -log4j.logger.org.apache.http=WARN -log4j.logger.org.apache.xml=WARN -log4j.logger.org.openid4java=WARN -log4j.logger.org.openid4java.consumer.ConsumerManager=FATAL -log4j.logger.org.openid4java.discovery.Discovery=ERROR -log4j.logger.org.openid4java.server.RealmVerifier=ERROR -log4j.logger.org.openid4java.message.AuthSuccess=ERROR - -# Silence non-critical messages from c3p0 (if used). -# -log4j.logger.com.mchange.v2.c3p0=WARN -log4j.logger.com.mchange.v2.resourcepool=WARN -log4j.logger.com.mchange.v2.sql=WARN - -# Silence non-critical messages from Velocity -# -log4j.logger.velocity=WARN - -# Silence non-critical messages from apache.http -# log4j.logger.org.apache.http=WARN - -# Silence non-fatal messages from cookie parsing -log4j.logger.org.apache.http.client.protocol.ResponseProcessCookies=FATAL diff --git a/modules/gerrit/templates/log4j.xml.erb b/modules/gerrit/templates/log4j.xml.erb new file mode 100644 index 0000000..d4f0ec4 --- /dev/null +++ b/modules/gerrit/templates/log4j.xml.erb @@ -0,0 +1,176 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd"> +<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/";> + <appender name="error_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/error_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="[%d] [%t] %-5p %c %x: %m%n"/> + </layout> + </appender> + <appender name="gc_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/gc_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="[%d] [%t] %-5p %x: %m%n"/> + </layout> + </appender> + <appender name="httpd_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/httpd_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="com.google.gerrit.pgm.http.jetty.HttpLogLayout"/> + </appender> + <appender name="replication_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/replication_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="[%d] [%t] %-5p %x: %m%n"/> + </layout> + </appender> + <appender name="sshd_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/sshd_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="com.google.gerrit.sshd.SshLogLayout"/> + </appender> + <appender name="plugin_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/plugin_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="[%d] [%t] %-5p %x: %m%n"/> + </layout> + </appender> + <appender name="delete_log" class="org.apache.log4j.DailyRollingFileAppender"> + <param name="encoding" value="UTF-8"/> + <param name="File" value="/var/lib/gerrit2/review_site/logs/delete_log"/> + <!--<param name="fileAppend" value="true"/>--> + <param name="immediateFlush" value="true"/> + <param name="threshold" value="INFO"/> + <layout class="org.apache.log4j.PatternLayout"> + <param name="ConversionPattern" value="[%d] [%t] %-5p %x: %m%n"/> + </layout> + </appender> + <% if @log_host %> + <!-- Logstash --> + <appender name="socket" class="org.apache.log4j.net.SocketAppender"> + <param name="port" value="<%= @log_port %>"/> + <param name="remoteHost" value="<%= @log_host %>"/> + </appender> + <appender name="async" class="org.apache.log4j.AsyncAppender"> + <param name="Blocking" value="false"/> + <param name="BufferSize" value="64"/> + <appender-ref ref="socket"/> + </appender> + <% end %> + <logger name="com.google.gerrit.pgm.http.jetty.HttpLog" additivity="false"> + <level value="DEBUG"/> + <appender-ref ref="httpd_log"/> + </logger> + <logger name="com.google.gerrit.sshd.GerritServerSession"> + <level value="WARN"/> + </logger> + <logger name="com.google.gerrit.sshd.SshLog"> + <level value="DEBUG"/> + <appender-ref ref="sshd_log"/> + </logger> + <logger name="com.mchange.v2.c3p0"> + <level value="WARN"/> + </logger> + <logger name="com.mchange.v2.resourcepool"> + <level value="WARN"/> + </logger> + <logger name="com.mchange.v2.sql"> + <level value="WARN"/> + </logger> + <logger name="eu.medsea.mimeutil"> + <level value="WARN"/> + </logger> + <logger name="gc_log" additivity="false"> + <level value="DEBUG"/> + <appender-ref ref="gc_log"/> + </logger> + <logger name="delete_log" additivity="false"> + <level value="INFO"/> + <appender-ref ref="delete_log"/> + </logger> + <logger name="replication_log" additivity="false"> + <level value="INFO"/> + <appender-ref ref="replication_log"/> + </logger> + <logger name="sshd_log" additivity="false"> + <level value="INFO"/> + <appender-ref ref="sshd_log"/> + </logger> + <logger name="org.apache.http"> + <level value="WARN"/> + </logger> + <logger name="org.apache.http.client.protocol.ResponseProcessCookies"> + <level value="FATAL"/> + </logger> + <logger name="org.apache.mina"> + <level value="WARN"/> + </logger> + <logger name="org.apache.sshd.common"> + <level value="WARN"/> + </logger> + <!-- To be removed when we upgrade to gerrit 2.14. --> + <logger name="org.apache.sshd.common.keyprovider.AbstractFileKeyPairProvider"> + <level value="INFO"/> + </logger> + <!-- Replaces the above in 2.14+ --> + <logger name="org.apache.sshd.common.keyprovider.FileKeyPairProvider"> + <level value="INFO"/> + </logger> + <logger name="org.apache.sshd.server"> + <level value="WARN"/> + </logger> + <logger name="org.apache.xml"> + <level value="WARN"/> + </logger> + <logger name="org.openid4java"> + <level value="WARN"/> + </logger> + <logger name="org.openid4java.consumer.ConsumerManager"> + <level value="FATAL"/> + </logger> + <logger name="org.openid4java.discovery.Discovery"> + <level value="ERROR"/> + </logger> + <logger name="org.openid4java.message.AuthSuccess"> + <level value="ERROR"/> + </logger> + <logger name="org.openid4java.server.RealmVerifier"> + <level value="ERROR"/> + </logger> + <logger name ="com.google.gerrit.server.plugins.PluginLoader"> + <level value="INFO"/> + <appender-ref ref="plugin_log"/> + </logger> + <!-- Silence non-critical messages from Velocity --> + <!-- to be removed in gerrit 2.16 / 3.0 --> + <logger name="velocity"> + <level value="WARN"/> + </logger> + <root> + <level value="INFO"/> + <appender-ref ref="error_log"/> + <% if @log_host %><appender-ref ref="async"/><% end %> + </root> +</log4j:configuration> -- To view, visit https://gerrit.wikimedia.org/r/392079 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I76245284285c6eb2377df23f1a75284dc0616d36 Gerrit-PatchSet: 21 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: Paladox <[email protected]> Gerrit-Reviewer: Chad <[email protected]> Gerrit-Reviewer: Dzahn <[email protected]> Gerrit-Reviewer: Gehel <[email protected]> Gerrit-Reviewer: Paladox <[email protected]> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
