Gehel has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/392621 )
Change subject: Upgrade logstash plugins to 5.5.2 ...................................................................... Upgrade logstash plugins to 5.5.2 - Changed license because ruby complained - removed sentry to pull it like other plugins (not sure about de dot it seems to contain some tweaks event.get vs event.include?) - install plugins first then prepare offline pack - added json_encode as it seems to be useful, one usecase I'd like to explore is to create a field that contains the event as json that could be indexed with the standard analyszer. Perhaps it'd help to reduce the need to have dynamic mappings by offering users a free text field they could search on to retrieve particular events. Bug: T178412 Change-Id: I3673cda254731a695b92cce0eb9fff135d52b45a --- M build.sh D logstash-filters-wikimedia/lib/logstash/outputs/sentry.rb M logstash-filters-wikimedia/logstash-filters-wikimedia.gemspec A target/releases/plugins-5.5.2.zip M target/releases/plugins-latest.zip 5 files changed, 18 insertions(+), 139 deletions(-) Approvals: Gehel: Verified; Looks good to me, approved diff --git a/build.sh b/build.sh index 8e07bb6..f8a6f6e 100755 --- a/build.sh +++ b/build.sh @@ -64,11 +64,24 @@ echo ------------------------ echo Build initial plugin pack with logstash-plugin prepare-offline-pack echo +pushd $LS_HOME # https://github.com/logstash-plugins/logstash-filter-anonymize/issues/11 + +# Install first otherwize prepare-offline-pack will fail +$LS_PLUGIN install \ + logstash-filter-anonymize \ + logstash-filter-multiline \ + logstash-filter-prune \ + logstash-filter-json_encode \ + logstash-output-sentry + $LS_PLUGIN prepare-offline-pack \ --output "$PLUGIN_PACK_PATH" \ logstash-filter-anonymize \ logstash-filter-multiline \ - logstash-filter-prune + logstash-filter-prune \ + logstash-filter-json_encode \ + logstash-output-sentry +popd echo echo ------------------------ diff --git a/logstash-filters-wikimedia/lib/logstash/outputs/sentry.rb b/logstash-filters-wikimedia/lib/logstash/outputs/sentry.rb deleted file mode 100644 index 584918b..0000000 --- a/logstash-filters-wikimedia/lib/logstash/outputs/sentry.rb +++ /dev/null @@ -1,135 +0,0 @@ -# from https://github.com/antho31/logstash-output-sentry/blob/master/lib/logstash/outputs/sentry.rb -# (C) 2014 Dave Clark, MIT license - -# encoding: utf-8 -require 'logstash/outputs/base' -require 'logstash/namespace' -require 'json' - -# Sentry is a modern error logging and aggregation platform. -# * https://getsentry.com/ -# -# It’s important to note that Sentry should not be thought of as a log stream, but as an aggregator. -# It fits somewhere in-between a simple metrics solution (such as Graphite) and a full-on log stream aggregator (like Logstash). -# -# Generate and inform your client key (Settings -> Client key) -# The client key has this form * https://[key]:[secret]@[host]/[project_id] * -# -# More informations : -# * https://sentry.readthedocs.org/en/latest/ - - -class LogStash::Outputs::Sentry < LogStash::Outputs::Base - - config_name 'sentry' - - - # The key of the client key - config :key, :validate => :string, :required => true - - # The secret of the client key - config :secret, :validate => :string, :required => true - - # The project id of the client key - config :project_id, :validate => :string, :required => true - - # The Sentry host - config :host, :validate => :string, :default => "https://app.getsentry.com";, :required => false - - # This sets the message value in Sentry (the title of your event) - config :msg, :validate => :string, :default => "Message from logstash", :required => false - - # This sets the level value in Sentry (the level tag) - config :level_tag, :validate => :string, :default => "error", :required => false - - # Is the protocole https ? By default yes (host is "https://app.getsentry.com";) - # If you have installed Sentry in your own machine, maybe you do use http, - # so you have to disable ssl ( "use_ssl" => false ) - config :use_ssl, :validate => :boolean, :default => true, :required => false - - # If set to true automatically map all logstash defined fields to Sentry extra fields. - # As an example, the logstash event: - # [source,ruby] - # { - # "@timestamp":"2013-12-10T14:36:26.151+0000", - # "@version": 1, - # "message":"log message", - # "host": "host.domain.com", - # "nested_field": { - # "key": "value" - # } - # } - # Is mapped to this Sentry event: - # [source,ruby] - # extra { - # "@timestamp":"2013-12-10T14:36:26.151+0000", - # "@version": 1, - # "message":"log message", - # "host": "host.domain.com", - # "nested_field": { - # "key": "value" - # } - # } - config :fields_to_tags, :validate => :boolean, :default => false, :required => false - - public - def register - require 'net/https' - require 'uri' - - @url = "#{host}/api/#{project_id}/store/" - @uri = URI.parse(@url) - - @client = Net::HTTP.new(@uri.host, @uri.port) - @client.use_ssl = use_ssl - @client.verify_mode = OpenSSL::SSL::VERIFY_NONE - - @logger.debug("Client", :client => @client.inspect) - end - - public - def receive(event) - return unless output?(event) - - require 'securerandom' - - packet = { - :event_id => SecureRandom.uuid.gsub('-', ''), - :timestamp => event['@timestamp'], - :message => event["#{msg}"] || "#{msg}" - } - - packet[:level] = "#{level_tag}" - packet[:platform] = 'logstash' - packet[:server_name] = event['host'] - packet[:extra] = event.to_hash - - if fields_to_tags == true - packet[:tags] = event.to_hash - end - - @logger.debug("Sentry packet", :sentry_packet => packet) - - auth_header = "Sentry sentry_version=5," + - "sentry_client=raven_logstash/1.0," + - "sentry_timestamp=#{event['@timestamp'].to_i}," + - "sentry_key=#{@key}," + - "sentry_secret=#{@secret}" - - request = Net::HTTP::Post.new(@uri.path) - - begin - request.body = packet.to_json - request.add_field('X-Sentry-Auth', auth_header) - - response = @client.request(request) - - @logger.info("Sentry response", :request => request.inspect, :response => response.inspect) - - raise unless response.code == '200' - rescue Exception => e - @logger.warn("Unhandled exception", :request => request.inspect, :response => response.inspect, :exception => e.inspect) - end - end -end - diff --git a/logstash-filters-wikimedia/logstash-filters-wikimedia.gemspec b/logstash-filters-wikimedia/logstash-filters-wikimedia.gemspec index f1d8f18..c7a03a0 100644 --- a/logstash-filters-wikimedia/logstash-filters-wikimedia.gemspec +++ b/logstash-filters-wikimedia/logstash-filters-wikimedia.gemspec @@ -1,8 +1,8 @@ Gem::Specification.new do |s| s.name = 'logstash-filters-wikimedia' - s.version = '0.5.0' - s.licenses = ['Apache License (2.0)'] + s.version = '0.5.5' + s.licenses = ['Apache-2.0'] s.summary = "Backports of logstash plugins for wikimedia installation. Includes the prune and de_dot filters" s.description = "This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install /path/to/gemfile. This gem is not a stand-alone program" s.authors = ["Elastic"] diff --git a/target/releases/plugins-5.5.2.zip b/target/releases/plugins-5.5.2.zip new file mode 100644 index 0000000..e2d886d --- /dev/null +++ b/target/releases/plugins-5.5.2.zip @@ -0,0 +1 @@ +#$# git-fat 1f5e6a030070629673ea86f1ca41ddc8eb9d2eef 198058 diff --git a/target/releases/plugins-latest.zip b/target/releases/plugins-latest.zip index aff2dd2..24405d5 120000 --- a/target/releases/plugins-latest.zip +++ b/target/releases/plugins-latest.zip @@ -1 +1 @@ -plugins-5.0.1.zip \ No newline at end of file +plugins-5.5.2.zip \ No newline at end of file -- To view, visit https://gerrit.wikimedia.org/r/392621 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3673cda254731a695b92cce0eb9fff135d52b45a Gerrit-PatchSet: 3 Gerrit-Project: operations/software/logstash/plugins Gerrit-Branch: master Gerrit-Owner: DCausse <[email protected]> Gerrit-Reviewer: EBernhardson <[email protected]> Gerrit-Reviewer: Gehel <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
