Hello community,
here is the log from the commit of package python-logreduce for
openSUSE:Factory checked in at 2018-07-10 16:15:41
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-logreduce (Old)
and /work/SRC/openSUSE:Factory/.python-logreduce.new (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-logreduce"
Tue Jul 10 16:15:41 2018 rev:2 rq:621700 version:0.1.3
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-logreduce/python-logreduce.changes
2018-05-29 10:39:53.279635900 +0200
+++ /work/SRC/openSUSE:Factory/.python-logreduce.new/python-logreduce.changes
2018-07-10 16:15:46.685503937 +0200
@@ -1,0 +2,13 @@
+Mon Jul 9 10:53:33 UTC 2018 - dmuel...@suse.com
+
+- update to 0.1.3:
+ * Update zuul-jobs log-classify role
+ * Fix ARA report directory link to ara-report
+ * Automatically set the zuul-web url for logs.rdoproject.org
+ * Add tripleo-ci job tokenization
+ * Update README todo lists
+ * Fix typo in ARA Records Ansible
+ * Start with a capitalized pypi summary
+ * Move flake8 from test-requirements to the linter env
+
+-------------------------------------------------------------------
Old:
----
logreduce-0.1.2.tar.gz
New:
----
logreduce-0.1.3.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-logreduce.spec ++++++
--- /var/tmp/diff_new_pack.ZwBQCm/_old 2018-07-10 16:15:47.301502982 +0200
+++ /var/tmp/diff_new_pack.ZwBQCm/_new 2018-07-10 16:15:47.301502982 +0200
@@ -19,7 +19,7 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
%define skip_python2 1
Name: python-logreduce
-Version: 0.1.2
+Version: 0.1.3
Release: 0
Summary: Extract anomalies from log files
License: Apache-2.0
++++++ logreduce-0.1.2.tar.gz -> logreduce-0.1.3.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/AUTHORS new/logreduce-0.1.3/AUTHORS
--- old/logreduce-0.1.2/AUTHORS 2018-04-26 06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/AUTHORS 2018-07-04 08:41:29.000000000 +0200
@@ -1,3 +1,5 @@
+David Moreau Simard <dmsim...@redhat.com>
+Dirk Mueller <dmuel...@suse.com>
Jenkins CI <jenk...@softwarefactory-project.io>
Tristan Cacqueray <tdeca...@redhat.com>
Zuul CI <z...@softwarefactory-project.io>
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/ChangeLog
new/logreduce-0.1.3/ChangeLog
--- old/logreduce-0.1.2/ChangeLog 2018-04-26 06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/ChangeLog 2018-07-04 08:41:29.000000000 +0200
@@ -1,6 +1,18 @@
CHANGES
=======
+0.1.3
+-----
+
+* Update zuul-jobs log-classify role
+* Fix ARA report directory link to ara-report
+* Automatically set the zuul-web url for logs.rdoproject.org
+* Add tripleo-ci job tokenization
+* Update README todo lists
+* Fix typo in ARA Records Ansible
+* Start with a capitalized pypi summary
+* Move flake8 from test-requirements to the linter env
+
0.1.2
-----
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/PKG-INFO new/logreduce-0.1.3/PKG-INFO
--- old/logreduce-0.1.2/PKG-INFO 2018-04-26 06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/PKG-INFO 2018-07-04 08:41:29.000000000 +0200
@@ -1,7 +1,7 @@
Metadata-Version: 1.1
Name: logreduce
-Version: 0.1.2
-Summary: extract anomalies from log files
+Version: 0.1.3
+Summary: Extract anomalies from log files
Home-page: https://logreduce.softwarefactory-project.io/
Author: Tristan Cacqueray
Author-email: tdeca...@redhat.com
@@ -200,13 +200,21 @@
Add --debug to display false positive and missing chunks.
- Roadmap/todo
- ------------
+ TODOs
+ -----
- * Add logstash filter module
- * Add daemon worker mode with MQTT event listener
+ * Add terminal colors output
+ * Add progress bar
+ * Better differentiate training debug from testing debug
+ * Add a starting log line and report written
* Add tarball traversal in utils.files_iterator
+ * Add logstash filter module
* Improve tokenization tests
+
+
+ Roadmap
+ -------
+ * Add daemon worker mode with MQTT event listener
* Discard files that are 100% anomalous
* Report mean diviation instead of absolute distances
* Investigate second stage model
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/README.rst
new/logreduce-0.1.3/README.rst
--- old/logreduce-0.1.2/README.rst 2018-04-26 06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/README.rst 2018-07-04 08:41:06.000000000 +0200
@@ -192,13 +192,21 @@
Add --debug to display false positive and missing chunks.
-Roadmap/todo
-------------
+TODOs
+-----
-* Add logstash filter module
-* Add daemon worker mode with MQTT event listener
+* Add terminal colors output
+* Add progress bar
+* Better differentiate training debug from testing debug
+* Add a starting log line and report written
* Add tarball traversal in utils.files_iterator
+* Add logstash filter module
* Improve tokenization tests
+
+
+Roadmap
+-------
+* Add daemon worker mode with MQTT event listener
* Discard files that are 100% anomalous
* Report mean diviation instead of absolute distances
* Investigate second stage model
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce/cmd.py
new/logreduce-0.1.3/logreduce/cmd.py
--- old/logreduce-0.1.2/logreduce/cmd.py 2018-04-26 06:07:05.000000000
+0200
+++ new/logreduce-0.1.3/logreduce/cmd.py 2018-07-04 08:41:06.000000000
+0200
@@ -62,6 +62,10 @@
# Convenient trick
if "logs.openstack.org" in kwargs.get('logs_url', ""):
self.zuul_web = "http://zuul.openstack.org/api";
+ if "logs.rdoproject.org" in kwargs.get('logs_url', ""):
+ self.zuul_web = "https://softwarefactory-project.io/zuul/api/"; \
+ "tenant/rdoproject.org"
+
try:
args.func(**kwargs)
except RuntimeError as e:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce/html_output.py
new/logreduce-0.1.3/logreduce/html_output.py
--- old/logreduce-0.1.2/logreduce/html_output.py 2018-04-26
06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/logreduce/html_output.py 2018-07-04
08:41:06.000000000 +0200
@@ -44,7 +44,7 @@
"<button type='button' id='debugbtn' "
"class='pull-right btn-xs btn-primary btn'>Show Debug</button>"
"<h4>--> <a href='./'>Full logs</a> // "
- "<a href='ara'>ARA Record Ansible</a> <--</h4>")
+ "<a href='ara-report'>ARA Records Ansible</a> <--</h4>")
# Results info
dom.append("<ul id='debuginfo'>")
dom.append(" <li>Command: %s</li>" % " ".join(sys.argv))
@@ -166,6 +166,6 @@
dom.append("<li><a href='%s'>%s</a></li>" % (fname[1], fname[0]))
dom.append("</ul>")
dom.append("<h4>--> <a href='./'>Full logs</a> // "
- "<a href='ara'>ARA Record Ansible</a> <--</h4>")
+ "<a href='ara-report'>ARA Records Ansible</a> <--</h4>")
dom.append("</body></html>")
return "\n".join(dom)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce/process.py
new/logreduce-0.1.3/logreduce/process.py
--- old/logreduce-0.1.2/logreduce/process.py 2018-04-26 06:07:05.000000000
+0200
+++ new/logreduce-0.1.3/logreduce/process.py 2018-07-04 08:41:06.000000000
+0200
@@ -24,6 +24,7 @@
from logreduce.models import models
from logreduce.tokenizer import remove_ansible_std_lines_lists
+from logreduce.tokenizer import Tokenizer
from logreduce.utils import files_iterator
from logreduce.utils import format_speed
from logreduce.utils import open_file
@@ -99,6 +100,8 @@
".xml", ".html", ".txt", ".py", ".json", ".yml"):
if ext in filename:
shortfilename += ext
+ # Remove UUID in filename
+ shortfilename = Tokenizer.uuid_re.subn('', shortfilename)[0]
# Remove numbers and symbols
return re.subn(r'[^a-zA-Z\/\._-]*', '', shortfilename)[0]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce/tokenizer.py
new/logreduce-0.1.3/logreduce/tokenizer.py
--- old/logreduce-0.1.2/logreduce/tokenizer.py 2018-04-26 06:07:05.000000000
+0200
+++ new/logreduce-0.1.3/logreduce/tokenizer.py 2018-07-04 08:41:06.000000000
+0200
@@ -83,6 +83,7 @@
r'|sshd.*: Failed password for'
# zuul random test
r'|zuul.*echo BECOME-SUCCESS-'
+ r'|^[^ ]{64}$'
# useless debug statement
r'|ovs-ofctl .* (dump-ports|dump-flows|show)\b'
r'|(ip|eb)tables .* -L\b'
@@ -90,9 +91,10 @@
ip_re = re.compile(r'(%s|%s|%s)' % (IPV4_RE, IPV6_RE, MAC_RE), re.I)
power2_re = re.compile(r'([0-9a-f]{128}|[0-9a-f+/]{64}|'
'[0-9a-f]{40}|[0-9a-f]{32})', re.I)
- uuid_re = re.compile(UUID_RE, re.I)
+ uuid_re = re.compile(r'(%s|tx[^ ]{32})' % UUID_RE, re.I)
date_re = re.compile('(%s|%s|%s|%s)' % (DAYS, SHORT_DAYS,
SHORT_MONTHS, MONTHS), re.I)
+ heat_re = re.compile("-[^ -]{12}[- $]", re.I)
comments = re.compile(r'([\s]*# |^%% |^#|^[\s]*id = ").*')
alpha_re = re.compile(r'[^a-zA-Z_\/\s]')
gitver_re = re.compile(r'git[a-z0-9]+', re.I)
@@ -118,6 +120,7 @@
# Remove words that are exactly 32, 64 or 128 character longs
strip = Tokenizer.power2_re.subn("RNGN", strip)[0]
# Remove uuid
+ strip = Tokenizer.heat_re.subn(" HEAT ", strip)[0]
strip = Tokenizer.uuid_re.subn("RNGU", strip)[0]
# Remove date
strip = Tokenizer.date_re.subn("DATE", strip)[0]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce/utils.py
new/logreduce-0.1.3/logreduce/utils.py
--- old/logreduce-0.1.2/logreduce/utils.py 2018-04-26 06:07:05.000000000
+0200
+++ new/logreduce-0.1.3/logreduce/utils.py 2018-07-04 08:41:06.000000000
+0200
@@ -31,6 +31,8 @@
'ara-report/',
'ara-sf/',
'ara/',
+ 'etc/hostname',
+ 'etc/nodepool/provider',
# sf-ci useless static files
"executor.*/trusted/",
# tripleo-ci static files
@@ -68,6 +70,7 @@
'iotop.txt',
'lastlog',
'last',
+ 'authkey',
'lsmod.txt',
'lsof.txt',
'lsof_network.txt',
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce.egg-info/PKG-INFO
new/logreduce-0.1.3/logreduce.egg-info/PKG-INFO
--- old/logreduce-0.1.2/logreduce.egg-info/PKG-INFO 2018-04-26
06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/logreduce.egg-info/PKG-INFO 2018-07-04
08:41:29.000000000 +0200
@@ -1,7 +1,7 @@
Metadata-Version: 1.1
Name: logreduce
-Version: 0.1.2
-Summary: extract anomalies from log files
+Version: 0.1.3
+Summary: Extract anomalies from log files
Home-page: https://logreduce.softwarefactory-project.io/
Author: Tristan Cacqueray
Author-email: tdeca...@redhat.com
@@ -200,13 +200,21 @@
Add --debug to display false positive and missing chunks.
- Roadmap/todo
- ------------
+ TODOs
+ -----
- * Add logstash filter module
- * Add daemon worker mode with MQTT event listener
+ * Add terminal colors output
+ * Add progress bar
+ * Better differentiate training debug from testing debug
+ * Add a starting log line and report written
* Add tarball traversal in utils.files_iterator
+ * Add logstash filter module
* Improve tokenization tests
+
+
+ Roadmap
+ -------
+ * Add daemon worker mode with MQTT event listener
* Discard files that are 100% anomalous
* Report mean diviation instead of absolute distances
* Investigate second stage model
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/logreduce.egg-info/pbr.json
new/logreduce-0.1.3/logreduce.egg-info/pbr.json
--- old/logreduce-0.1.2/logreduce.egg-info/pbr.json 2018-04-26
06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/logreduce.egg-info/pbr.json 2018-07-04
08:41:29.000000000 +0200
@@ -1 +1 @@
-{"git_version": "2b70fec", "is_release": true}
\ No newline at end of file
+{"git_version": "f071111", "is_release": true}
\ No newline at end of file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/logreduce-0.1.2/roles/log-classify/defaults/main.yaml
new/logreduce-0.1.3/roles/log-classify/defaults/main.yaml
--- old/logreduce-0.1.2/roles/log-classify/defaults/main.yaml 2018-04-26
06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/roles/log-classify/defaults/main.yaml 2018-07-04
08:41:06.000000000 +0200
@@ -1,3 +1,8 @@
+# Set optin to False to disable log-classify
+logclassify_optin: True
+# Set report to True to hijack the reported log url
+logclassify_report: False
+
# How many job logs to use for building the model
logclassify_model_job_count: 2
# The pipeline to use for building the model
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/logreduce-0.1.2/roles/log-classify/tasks/install.yaml
new/logreduce-0.1.3/roles/log-classify/tasks/install.yaml
--- old/logreduce-0.1.2/roles/log-classify/tasks/install.yaml 2018-04-26
06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/roles/log-classify/tasks/install.yaml 2018-07-04
08:41:06.000000000 +0200
@@ -11,8 +11,8 @@
block:
- name: Look for depends-on location
set_fact:
- _logclassify_dir: "{{
zuul.projects['managesf.softwarefactory-project.io/logreduce']['src_dir'] }}"
- when: "'managesf.softwarefactory-project.io/logreduce' in zuul.projects"
+ _logclassify_dir: "{{
zuul.projects['softwarefactory-project.io/logreduce']['src_dir'] }}"
+ when: "'softwarefactory-project.io/logreduce' in zuul.projects"
- name: Fetch sources
block:
@@ -22,7 +22,7 @@
- name: Set clone location
set_fact:
_logclassify_dir: /tmp/logreduce
- when: "'managesf.softwarefactory-project.io/logreduce' not in
zuul.projects"
+ when: "'softwarefactory-project.io/logreduce' not in zuul.projects"
- name: Install from sources
command: python3 setup.py develop --user
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/roles/log-classify/tasks/main.yaml
new/logreduce-0.1.3/roles/log-classify/tasks/main.yaml
--- old/logreduce-0.1.2/roles/log-classify/tasks/main.yaml 2018-04-26
06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/roles/log-classify/tasks/main.yaml 2018-07-04
08:41:06.000000000 +0200
@@ -4,4 +4,6 @@
block:
- import_tasks: install.yaml
- import_tasks: run.yaml
- when: not zuul_success | bool
+ when:
+ - not zuul_success | bool
+ - logclassify_optin | default(True) | bool
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/roles/log-classify/tasks/run.yaml
new/logreduce-0.1.3/roles/log-classify/tasks/run.yaml
--- old/logreduce-0.1.2/roles/log-classify/tasks/run.yaml 2018-04-26
06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/roles/log-classify/tasks/run.yaml 2018-07-04
08:41:06.000000000 +0200
@@ -1,4 +1,9 @@
---
+- name: Foce the model_per_project option for known job to have different
output per project
+ set_fact:
+ logclassify_model_per_project: true
+ when: "'tox' in zuul.job"
+
- name: Set model name
set_fact:
_model_name: "{% if logclassify_model_per_project %}{{ zuul.project.name
}}/{% endif %}{{ zuul.job }}/{{ zuul.branch }}.clf"
@@ -7,6 +12,10 @@
set_fact:
_model_path: "{{ logclassify_model_dir }}/{{ _model_name }}"
+- name: Set report name
+ set_fact:
+ _report_name: "{% if logclassify_report %}report{% else %}log-classify{%
endif %}"
+
- name: Create directories
file:
path: "{{ item }}"
@@ -79,7 +88,7 @@
{{ logclassify_local_dir }}/job-output.txt
{% endif %}
{% endif %}
- --html {{ logclassify_tmp_dir }}/report.html
+ --html {{ logclassify_tmp_dir }}/{{ _report_name }}.html
{% if logclassify_static_location %}--static-location {{
logclassify_static_location }}{% endif %}
{% if logclassify_logserver_dir %}--include-path {{
logclassify_logserver_dir }}{% endif %}
{% for _path in logclassify_exclude_paths %}--exclude-path {{ _path }}{%
endfor %}
@@ -91,8 +100,8 @@
- name: Fetch report
synchronize:
- src: "{{ logclassify_tmp_dir }}/report.{{ item }}"
- dest: "{{ zuul.executor.log_root }}/report.{{ item }}"
+ src: "{{ logclassify_tmp_dir }}/{{ _report_name }}.{{ item }}"
+ dest: "{{ zuul.executor.log_root }}/{{ _report_name }}.{{ item }}"
mode: pull
with_items:
- html
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/setup.cfg
new/logreduce-0.1.3/setup.cfg
--- old/logreduce-0.1.2/setup.cfg 2018-04-26 06:07:25.000000000 +0200
+++ new/logreduce-0.1.3/setup.cfg 2018-07-04 08:41:29.000000000 +0200
@@ -1,6 +1,6 @@
[metadata]
name = logreduce
-summary = extract anomalies from log files
+summary = Extract anomalies from log files
description-file = README.rst
requires-python = >=3.5
author = Tristan Cacqueray
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/test-requirements.txt
new/logreduce-0.1.3/test-requirements.txt
--- old/logreduce-0.1.2/test-requirements.txt 2018-04-26 06:07:05.000000000
+0200
+++ new/logreduce-0.1.3/test-requirements.txt 2018-07-04 08:41:06.000000000
+0200
@@ -1,2 +1 @@
-flake8
nose
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/logreduce-0.1.2/tox.ini new/logreduce-0.1.3/tox.ini
--- old/logreduce-0.1.2/tox.ini 2018-04-26 06:07:05.000000000 +0200
+++ new/logreduce-0.1.3/tox.ini 2018-07-04 08:41:06.000000000 +0200
@@ -11,4 +11,5 @@
commands = nosetests -v --cover-package=logreduce
[testenv:pep8]
+deps = flake8
commands = flake8-3 --ignore=E26,E501,E251,E225,E722 logreduce
