Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package python-passivetotal for
openSUSE:Factory checked in at 2021-08-04 22:28:58
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-passivetotal (Old)
and /work/SRC/openSUSE:Factory/.python-passivetotal.new.1899 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-passivetotal"
Wed Aug 4 22:28:58 2021 rev:11 rq:910121 version:2.5.3
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-passivetotal/python-passivetotal.changes
2021-07-10 22:55:24.555338646 +0200
+++
/work/SRC/openSUSE:Factory/.python-passivetotal.new.1899/python-passivetotal.changes
2021-08-04 22:29:40.941743618 +0200
@@ -1,0 +2,28 @@
+Wed Jul 28 10:11:21 UTC 2021 - Sebastian Wagner <sebix+novell....@sebix.at>
+
+- update to version 2.5.3:
+ - Enhancements:
+ - Better support for unit tests in client libraries with ability to set a
+ session to override default request methods.
+ - Add flexibility to library class instantiation to prefer keyword parameters
+ over config file keys.
+ - Support for new `create_date` Articles API data field and query parameter.
Enables
+ searching for most recent articles instead of returning all of them at
once, and
+ provides visiblity to situations where an article published in the past
was recently
+ added to the Articles collection.
+ - Breaking Changes:
+ - Previously, calls to `analyzer.AllArticles()` would return all articles
without a date
+ limit. Now, it will return only articles created after the starting date
set with
+ `analyzer.set_date_range()`. The current module-level default for all
date-bounded queries
+ is 90 days back, so now this function will return all articles created in
the last 90 days.
+ - `age` property of an Article analyzer object is now based on `create_date`
instead of publish
+ date.
+- update to version 2.5.2:
+ - Enhancements:
+ - Send new request headers for metrics and troubleshooting with the
`set_context`
+ method on the `analyzer` module and within the core API request libs.
+ - Abstract package version into a distinct file to consolidate updates and
ensure
+ consistency across docs and pypi. Add `get_version` method to `analyzer`
module
+ for easy access to the current version number.
+
+-------------------------------------------------------------------
Old:
----
passivetotal-2.5.1.tar.gz
New:
----
passivetotal-2.5.3.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-passivetotal.spec ++++++
--- /var/tmp/diff_new_pack.fsXwL4/_old 2021-08-04 22:29:41.377743087 +0200
+++ /var/tmp/diff_new_pack.fsXwL4/_new 2021-08-04 22:29:41.381743082 +0200
@@ -19,7 +19,7 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
%bcond_without test
Name: python-passivetotal
-Version: 2.5.1
+Version: 2.5.3
Release: 0
Summary: Client for the PassiveTotal REST API
License: GPL-2.0-only
++++++ passivetotal-2.5.1.tar.gz -> passivetotal-2.5.3.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/CHANGELOG.md
new/passivetotal-2.5.3/CHANGELOG.md
--- old/passivetotal-2.5.1/CHANGELOG.md 2021-06-29 20:04:38.000000000 +0200
+++ new/passivetotal-2.5.3/CHANGELOG.md 2021-07-28 01:47:09.000000000 +0200
@@ -1,5 +1,51 @@
# Changelog
+## v2.5.3
+
+#### Enhancements
+
+- Better support for unit tests in client libraries with ability to set a
+ session to override default request methods.
+- Add flexibility to library class instantiation to prefer keyword parameters
+ over config file keys.
+- Support for new `create_date` Articles API data field and query parameter.
Enables
+ searching for most recent articles instead of returning all of them at once,
and
+ provides visiblity to situations where an article published in the past was
recently
+ added to the Articles collection.
+
+
+#### Breaking Changes
+
+- Previously, calls to `analyzer.AllArticles()` would return all articles
without a date
+ limit. Now, it will return only articles created after the starting date set
with
+ `analyzer.set_date_range()`. The current module-level default for all
date-bounded queries
+ is 90 days back, so now this function will return all articles created in
the last 90 days.
+- `age` property of an Article analyzer object is now based on `create_date`
instead of publish
+ date.
+
+
+#### Bug Fixes
+
+[ none ]
+
+
+
+## v2.5.2
+
+#### Enhancements
+
+- Send new request headers for metrics and troubleshooting with the
`set_context`
+ method on the `analyzer` module and within the core API request libs.
+- Abstract package version into a distinct file to consolidate updates and
ensure
+ consistency across docs and pypi. Add `get_version` method to `analyzer`
module
+ for easy access to the current version number.
+
+
+#### Bug Fixes
+
+
+
+
## v2.5.1
#### Enhancements
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/PKG-INFO
new/passivetotal-2.5.3/PKG-INFO
--- old/passivetotal-2.5.1/PKG-INFO 2021-06-29 20:04:56.000000000 +0200
+++ new/passivetotal-2.5.3/PKG-INFO 2021-07-28 01:48:06.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: passivetotal
-Version: 2.5.1
+Version: 2.5.3
Summary: Library for the RiskIQ PassiveTotal and Illuminate API
Home-page: https://github.com/passivetotal/python_api
Author: RiskIQ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/docs/conf.py
new/passivetotal-2.5.3/docs/conf.py
--- old/passivetotal-2.5.1/docs/conf.py 2021-06-29 20:04:38.000000000 +0200
+++ new/passivetotal-2.5.3/docs/conf.py 2021-07-15 21:35:42.000000000 +0200
@@ -14,6 +14,7 @@
import sys
import os
+import re
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
@@ -55,14 +56,12 @@
copyright = u'2021, RiskIQ'
author = u'RiskIQ'
-# The version info for the project you're documenting, acts as replacement for
-# |version| and |release|, also used in various other places throughout the
-# built documents.
-#
-# The short X.Y version.
-version = '2.5'
-# The full version, including alpha/beta/rc tags.
-release = '2.5.1'
+# pylint: disable=locally-disabled, invalid-name
+with open('../passivetotal/_version.py', 'r') as fd:
+ v_match = re.search(r'^VERSION\s*=\s*[\'"]([^\'"]*)[\'"]', fd.read(),
re.MULTILINE)
+ release = v_match.group(1)
+ version = '.'.join(release.split('.')[0:-1])
+# pylint: enable=locally-disabled, invalid-name
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal/_version.py
new/passivetotal-2.5.3/passivetotal/_version.py
--- old/passivetotal-2.5.1/passivetotal/_version.py 1970-01-01
01:00:00.000000000 +0100
+++ new/passivetotal-2.5.3/passivetotal/_version.py 2021-07-28
01:47:09.000000000 +0200
@@ -0,0 +1 @@
+VERSION="2.5.3"
\ No newline at end of file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal/analyzer/__init__.py
new/passivetotal-2.5.3/passivetotal/analyzer/__init__.py
--- old/passivetotal-2.5.1/passivetotal/analyzer/__init__.py 2021-06-21
16:37:13.000000000 +0200
+++ new/passivetotal-2.5.3/passivetotal/analyzer/__init__.py 2021-07-28
01:47:09.000000000 +0200
@@ -3,6 +3,8 @@
from collections import namedtuple
from datetime import datetime, timezone, timedelta
from passivetotal import *
+from passivetotal._version import VERSION
+from passivetotal.api import Context
from passivetotal.analyzer._common import AnalyzerError, AnalyzerAPIError,
is_ip
DEFAULT_DAYS_BACK = 90
@@ -54,8 +56,9 @@
if 'username' in kwargs and 'api_key' in kwargs:
api_clients[name] = c(**kwargs)
else:
- api_clients[name] = c.from_config()
+ api_clients[name] = c.from_config(**kwargs)
api_clients[name].exception_class = AnalyzerAPIError
+
api_clients[name].set_context('python','passivetotal',VERSION,'analyzer')
config['is_ready'] = True
def get_api(name):
@@ -91,6 +94,23 @@
raise AnalyzerError('type must be IPAddress or Hostname')
return objs[type](input)
+def get_version():
+ """Get the current version of this package."""
+ return VERSION
+
+def set_context(provider, variant, version, feature=''):
+ """Define the application context for an implementation using the analyzer
module.
+
+ Sets a header to be sent in API requests that is used for metrics and
troubleshooting.
+
+ :param provider: The company, partner, provider or other top-level
application context.
+ :param variant: The specific app, libary subcomponent, or feature category.
+ :param version: Version of the app, feature or code setting the context.
+ :param feature: Optional sub-feature, dashboard or script name.
+ """
+ for client in api_clients.values():
+ client.set_context(provider, variant, version, feature)
+
def set_date_range(days_back=DEFAULT_DAYS_BACK, start=None, start_date=None,
end=None, end_date=None):
"""Set a range of dates for all date-bounded API queries.
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal/analyzer/_common.py
new/passivetotal-2.5.3/passivetotal/analyzer/_common.py
--- old/passivetotal-2.5.1/passivetotal/analyzer/_common.py 2021-06-29
20:04:38.000000000 +0200
+++ new/passivetotal-2.5.3/passivetotal/analyzer/_common.py 2021-07-28
01:47:09.000000000 +0200
@@ -429,7 +429,13 @@
self.json = self.response.json()
except Exception:
self.json = {}
- self.message = self.json.get('error', self.json.get('message',
str(response)))
+ if self.json is None:
+ self.message = 'No JSON data in API response'
+ else:
+ try:
+ self.message = self.json.get('error', self.json.get('message',
str(response)))
+ except Exception:
+ self.message = ''
def __str__(self):
return 'Error #{0.status_code} "{0.message}" ({0.url})'.format(self)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal/analyzer/articles.py
new/passivetotal-2.5.3/passivetotal/analyzer/articles.py
--- old/passivetotal-2.5.1/passivetotal/analyzer/articles.py 2021-06-21
16:37:13.000000000 +0200
+++ new/passivetotal-2.5.3/passivetotal/analyzer/articles.py 2021-07-28
01:47:09.000000000 +0200
@@ -3,7 +3,7 @@
from passivetotal.analyzer._common import (
RecordList, Record, ForPandas
)
-from passivetotal.analyzer import get_api
+from passivetotal.analyzer import get_api, get_config
@@ -71,20 +71,28 @@
By default, instantiating the class will automatically load the entire list
of threat intelligence articles. Pass autoload=False to the constructor to
disable
this functionality.
+
+ Only articles created after the start date specified in the
analyzer.set_date_range()
+ method will be returned unless a different created_after parameter is
supplied to the object
+ constructor.
"""
- def __init__(self, autoload = True):
+ def __init__(self, created_after=None, autoload=True):
"""Initialize a list of articles; will autoload by default.
-
:param autoload: whether to automatically load articles upon
instantiation (defaults to true)
"""
super().__init__()
if autoload:
- self.load()
+ self.load(created_after)
- def load(self):
- """Query the API for articles and load them into an articles list."""
- response = get_api('Articles').get_articles()
+ def load(self, created_after=None):
+ """Query the API for articles and load them into an articles list.
+
+ :param created_after: only return articles created after this date
(optional, defaults to date set by `analyzer.set_date_range()`
+ """
+ if created_after is None:
+ created_after = get_config('start_date')
+ response = get_api('Articles').get_articles(createdAfter=created_after)
self.parse(response)
@@ -98,6 +106,7 @@
self._summary = api_response.get('summary')
self._type = api_response.get('type')
self._publishdate = api_response.get('publishedDate')
+ self._createdate = api_response.get('createdDate')
self._link = api_response.get('link')
self._categories = api_response.get('categories')
self._tags = api_response.get('tags')
@@ -115,13 +124,14 @@
response = get_api('Articles').get_details(self._guid)
self._summary = response.get('summary')
self._publishdate = response.get('publishedDate')
+ self._createdate = response.get('createdDate')
self._tags = response.get('tags')
self._categories = response.get('categories')
self._indicators = response.get('indicators')
def _get_dict_fields(self):
- return ['guid','title','type','summary','str:date_published','age',
- 'link','categories','tags','indicators','indicator_count',
+ return
['guid','title','type','summary','str:date_published','str:date_created',
+ 'age',
'link','categories','tags','indicators','indicator_count',
'indicator_types','str:ips','str:hostnames']
def _ensure_details(self):
@@ -161,6 +171,7 @@
title = self._title,
type = self._type,
date_published = self._publishdate,
+ date_created = self._createdate,
summary = self._summary,
link = self._link,
categories = self._categories,
@@ -229,10 +240,17 @@
return date
@property
+ def date_created(self):
+ """Date the article was created in the RiskIQ database."""
+ self._ensure_details()
+ date = datetime.fromisoformat(self._createdate)
+ return date
+
+ @property
def age(self):
- """Age of the article in days."""
+ """Age of the article in days, measured from create date."""
now = datetime.now(timezone.utc)
- interval = now - self.date_published
+ interval = now - self.date_created
return interval.days
@property
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal/api.py
new/passivetotal-2.5.3/passivetotal/api.py
--- old/passivetotal-2.5.1/passivetotal/api.py 2021-06-29 20:04:38.000000000
+0200
+++ new/passivetotal-2.5.3/passivetotal/api.py 2021-07-28 01:47:09.000000000
+0200
@@ -1,13 +1,17 @@
"""PassiveTotal API Interface."""
-__author__ = 'Brandon Dixon (PassiveTotal)'
-__version__ = '1.0.0'
import json
import logging
import requests
import sys
+from urllib.parse import quote as urlquote
from passivetotal.config import Config
+from passivetotal._version import VERSION
+
+__author__ = 'Brandon Dixon (PassiveTotal)'
+__version__ = VERSION
+
class Client(object):
@@ -20,7 +24,8 @@
def __init__(self, username, api_key, server=DEFAULT_SERVER,
version=DEFAULT_VERSION, http_proxy=None, https_proxy=None,
- verify=True, headers=None, debug=False,
exception_class=Exception):
+ verify=True, headers=None, debug=False,
exception_class=Exception,
+ session=None):
"""Initial loading of the client.
:param str username: API username in email address format
@@ -58,18 +63,26 @@
if '127.0.0.1' in server:
self.verify = False
self.exception_class = exception_class
+ self.set_context('python','passivetotal',VERSION)
+ self.session = session or requests.Session()
@classmethod
- def from_config(cls):
- """Method to return back a loaded instance."""
+ def from_config(cls, **kwargs):
+ """Method to return back a loaded instance.
+
+ kwargs override configuration file variables if provided and are
passed to the object constructor.
+ """
+ arg_keys =
['username','api_key','server','version','http_proxy','https_proxy']
+ args = { k: kwargs.pop(k) if k in kwargs else None for k in arg_keys }
config = Config()
client = cls(
- username=config.get('username'),
- api_key=config.get('api_key'),
- server=config.get('api_server'),
- version=config.get('api_version'),
- http_proxy=config.get('http_proxy'),
- https_proxy=config.get('https_proxy'),
+ username = args.get('username') or config.get('username'),
+ api_key = args.get('api_key') or config.get('api_key'),
+ server = args.get('server') or config.get('api_server'),
+ version = args.get('version') or config.get('api_version'),
+ http_proxy = args.get('http_proxy') or config.get('http_proxy'),
+ https_proxy = args.get('https_proxy') or config.get('https_proxy'),
+ **kwargs
)
return client
@@ -78,6 +91,18 @@
self.logger.setLevel('DEBUG')
else:
self.logger.setLevel('INFO')
+
+ def set_context(self, provider, variant, version, feature=''):
+ """Set the context for this request.
+
+ :param provider: The company, partner, provider or other top-level
application context.
+ :param variant: The specific app, libary subcomponent, or feature
category.
+ :param version: Version of the app, feature or code setting the
context.
+ :param feature: Optional sub-feature, dashboard or script name.
+ """
+ context = Context(provider, variant, version, feature)
+ self.context = context
+ self.headers.update(context.get_header())
def _endpoint(self, endpoint, action, *url_args):
"""Return the URL for the action.
@@ -138,7 +163,7 @@
if self.proxies:
kwargs['proxies'] = self.proxies
self.logger.debug("Requesting: %s, %s" % (api_url, str(kwargs)))
- response = requests.get(api_url, **kwargs)
+ response = self.session.get(api_url, **kwargs)
return self._json(response)
def _get_special(self, endpoint, action, trail, data, *url_args,
**url_params):
@@ -158,7 +183,7 @@
'auth': (self.username, self.api_key)}
if self.proxies:
kwargs['proxies'] = self.proxies
- response = requests.get(api_url, **kwargs)
+ response = self.session.get(api_url, **kwargs)
return self._json(response)
def _send_data(self, method, endpoint, action,
@@ -179,5 +204,32 @@
'auth': (self.username, self.api_key)}
if self.proxies:
kwargs['proxies'] = self.proxies
- response = requests.request(method, api_url, **kwargs)
+ response = self.session.request(method, api_url, **kwargs)
return self._json(response)
+
+
+
+class Context:
+
+ """Integration context for a set of API requests."""
+
+ HEADER_NAME = 'X-RISKIQ-CONTEXT'
+
+ def __init__(self, provider, variant, version, feature = ''):
+ """Build a new context header.
+
+ :param provider: The company, partner, provider or other top-level
application context.
+ :param variant: The specific app, libary subcomponent, or feature
category.
+ :param version: Version of the app, feature or code setting the
context.
+ :param feature: Optional sub-feature, dashboard or script name.
+ """
+ self._fields = (provider, variant, version, feature)
+
+ def get_header_name(self):
+ return self.HEADER_NAME
+
+ def get_header_value(self):
+ return '/'.join(map(lambda f: urlquote(f, safe=''), self._fields))
+
+ def get_header(self):
+ return { self.get_header_name() : self.get_header_value() }
\ No newline at end of file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal.egg-info/PKG-INFO
new/passivetotal-2.5.3/passivetotal.egg-info/PKG-INFO
--- old/passivetotal-2.5.1/passivetotal.egg-info/PKG-INFO 2021-06-29
20:04:56.000000000 +0200
+++ new/passivetotal-2.5.3/passivetotal.egg-info/PKG-INFO 2021-07-28
01:48:06.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: passivetotal
-Version: 2.5.1
+Version: 2.5.3
Summary: Library for the RiskIQ PassiveTotal and Illuminate API
Home-page: https://github.com/passivetotal/python_api
Author: RiskIQ
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/passivetotal.egg-info/SOURCES.txt
new/passivetotal-2.5.3/passivetotal.egg-info/SOURCES.txt
--- old/passivetotal-2.5.1/passivetotal.egg-info/SOURCES.txt 2021-06-29
20:04:56.000000000 +0200
+++ new/passivetotal-2.5.3/passivetotal.egg-info/SOURCES.txt 2021-07-28
01:48:06.000000000 +0200
@@ -21,6 +21,7 @@
examples/whois_search.py
passivetotal/__init__.py
passivetotal/__main__.py
+passivetotal/_version.py
passivetotal/api.py
passivetotal/config.py
passivetotal/response.py
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/passivetotal-2.5.1/setup.py
new/passivetotal-2.5.3/setup.py
--- old/passivetotal-2.5.1/setup.py 2021-06-29 20:04:38.000000000 +0200
+++ new/passivetotal-2.5.3/setup.py 2021-07-15 21:35:42.000000000 +0200
@@ -1,14 +1,21 @@
#!/usr/bin/env python
import os
+import re
from setuptools import setup, find_packages
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
+# pylint: disable=locally-disabled, invalid-name
+with open('passivetotal/_version.py', 'r') as fd:
+ v_match = re.search(r'^VERSION\s*=\s*[\'"]([^\'"]*)[\'"]', fd.read(),
re.MULTILINE)
+ __version__ = v_match.group(1) if v_match else 'no version'
+# pylint: enable=locally-disabled, invalid-name
+
setup(
name='passivetotal',
- version='2.5.1',
+ version=__version__,
description='Library for the RiskIQ PassiveTotal and Illuminate API',
url="https://github.com/passivetotal/python_api";,
author="RiskIQ",
