[
https://issues.apache.org/jira/browse/BEAM-5339?focusedWorklogId=148573&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-148573
]
ASF GitHub Bot logged work on BEAM-5339:
----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Sep/18 03:38
Start Date: 27/Sep/18 03:38
Worklog Time Spent: 10m
Work Description: chamikaramj closed pull request #6430:
[BEAM-5339]/apply new policy on Beam dependency tooling
URL: https://github.com/apache/beam/pull/6430
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
index 58f70996868..4ece37987d8 100644
--- a/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
+++ b/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
@@ -16,17 +16,18 @@
# limitations under the License.
#
-import sys
+import logging
import os.path
import re
-import traceback
-import logging
import requests
-import time
+import sys
+import traceback
+import version_comparer
+
from datetime import datetime
from dependency_check.bigquery_client_utils import BigQueryClientUtils
-from jira_utils.jira_manager import JiraManager
from dependency_check.report_generator_config import ReportGeneratorConfig
+from jira_utils.jira_manager import JiraManager
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
@@ -135,11 +136,12 @@ def prioritize_dependencies(deps, sdk_type):
latest_ver,
curr_release_date,
latest_release_date)
- if (compare_dependency_versions(curr_ver, latest_ver) or
+ if (version_comparer.compare_dependency_versions(curr_ver, latest_ver) or
compare_dependency_release_dates(curr_release_date,
latest_release_date)):
# Create a new issue or update on the existing issue
- jira_issue = jira_manager.run(dep_name, latest_ver, sdk_type, group_id
= group_id)
- if (jira_issue.fields.status.name == 'Open' or
jira_issue.fields.status.name == 'Reopened'):
+ jira_issue = jira_manager.run(dep_name, curr_ver, latest_ver,
sdk_type, group_id = group_id)
+ if (jira_issue.fields.status.name == 'Open' or
+ jira_issue.fields.status.name == 'Reopened'):
dep_info += "<td><a href=\'{0}\'>{1}</a></td></tr>".format(
ReportGeneratorConfig.BEAM_JIRA_HOST+"browse/"+ jira_issue.key,
jira_issue.key)
@@ -153,39 +155,6 @@ def prioritize_dependencies(deps, sdk_type):
return high_priority_deps
-def compare_dependency_versions(curr_ver, latest_ver):
- """
- Compare the current using version and the latest version.
- Return true if a major version change was found, or 3 minor versions that
the current version is behind.
- Args:
- curr_ver
- latest_ver
- Return:
- boolean
- """
- if curr_ver is None or latest_ver is None:
- return True
- else:
- curr_ver_splitted = curr_ver.split('.')
- latest_ver_splitted = latest_ver.split('.')
- curr_major_ver = curr_ver_splitted[0]
- latest_major_ver = latest_ver_splitted[0]
- # compare major versions
- if curr_major_ver != latest_major_ver:
- return True
- # compare minor versions
- else:
- curr_minor_ver = curr_ver_splitted[1] if len(curr_ver_splitted) > 1 else
None
- latest_minor_ver = latest_ver_splitted[1] if len(latest_ver_splitted) >
1 else None
- if curr_minor_ver is not None and latest_minor_ver is not None:
- if (not curr_minor_ver.isdigit() or not latest_minor_ver.isdigit())
and curr_minor_ver != latest_minor_ver:
- return True
- elif int(curr_minor_ver) +
ReportGeneratorConfig.MAX_MINOR_VERSION_DIFF <= int(latest_minor_ver):
- return True
- # TODO: Comparing patch versions if needed.
- return False
-
-
def find_release_time_from_maven_central(group_id, artifact_id, version):
"""
Find release dates from Maven Central REST API.
diff --git a/.test-infra/jenkins/dependency_check/generate_report.sh
b/.test-infra/jenkins/dependency_check/generate_report.sh
index 644fa81581d..13c6a0f2eeb 100755
--- a/.test-infra/jenkins/dependency_check/generate_report.sh
+++ b/.test-infra/jenkins/dependency_check/generate_report.sh
@@ -49,6 +49,7 @@ pip install mock jira pyyaml
cd $WORKSPACE/src/.test-infra/jenkins
python -m dependency_check.dependency_check_report_generator_test
python -m jira_utils.jira_manager_test
+python -m dependency_check.version_comparer_test
echo "<html><body>" >
$WORKSPACE/src/build/dependencyUpdates/beam-dependency-check-report.html
diff --git a/.test-infra/jenkins/dependency_check/version_comparer.py
b/.test-infra/jenkins/dependency_check/version_comparer.py
new file mode 100644
index 00000000000..d8134ecf11d
--- /dev/null
+++ b/.test-infra/jenkins/dependency_check/version_comparer.py
@@ -0,0 +1,51 @@
+#!/usr/bin/env python
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from dependency_check.report_generator_config import ReportGeneratorConfig
+
+def compare_dependency_versions(curr_ver, latest_ver):
+ """
+ Compare the current using version and the latest version.
+ Return true if a major version change was found, or 3 minor versions that
the current version is behind.
+ Args:
+ curr_ver
+ latest_ver
+ Return:
+ boolean
+ """
+ if curr_ver is None or latest_ver is None:
+ return True
+ else:
+ curr_ver_splitted = curr_ver.split('.')
+ latest_ver_splitted = latest_ver.split('.')
+ curr_major_ver = curr_ver_splitted[0]
+ latest_major_ver = latest_ver_splitted[0]
+ # compare major versions
+ if curr_major_ver != latest_major_ver:
+ return True
+ # compare minor versions
+ else:
+ curr_minor_ver = curr_ver_splitted[1] if len(curr_ver_splitted) > 1 else
None
+ latest_minor_ver = latest_ver_splitted[1] if len(latest_ver_splitted) >
1 else None
+ if curr_minor_ver is not None and latest_minor_ver is not None:
+ if (not curr_minor_ver.isdigit() or not latest_minor_ver.isdigit())
and curr_minor_ver != latest_minor_ver:
+ return True
+ elif int(curr_minor_ver) +
ReportGeneratorConfig.MAX_MINOR_VERSION_DIFF <= int(latest_minor_ver):
+ return True
+ # TODO: Comparing patch versions if needed.
+ return False
diff --git a/.test-infra/jenkins/dependency_check/version_comparer_test.py
b/.test-infra/jenkins/dependency_check/version_comparer_test.py
new file mode 100644
index 00000000000..78b67e1967c
--- /dev/null
+++ b/.test-infra/jenkins/dependency_check/version_comparer_test.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import version_comparer
+import unittest
+
+class VersionComparerTest(unittest.TestCase):
+ """Tests for `version_comparer.py`."""
+
+ def setUp(self):
+ print("\n\nTest : " + self._testMethodName)
+
+ def test_compare_major_verison_true(self):
+ curr_ver = '1.0.0'
+ latest_ver = '2.0.0'
+ self.assertTrue(version_comparer.compare_dependency_versions(curr_ver,
latest_ver))
+
+ def test_compare_minor_version_true(self):
+ curr_ver = '1.0.0'
+ latest_ver = '1.3.0'
+ self.assertTrue(version_comparer.compare_dependency_versions(curr_ver,
latest_ver))
+
+ def test_compare_non_semantic_version_true(self):
+ curr_ver = '1.rc1'
+ latest_ver = '1.rc2'
+ self.assertTrue(version_comparer.compare_dependency_versions(curr_ver,
latest_ver))
+
+ def test_compare_minor_version_false(self):
+ curr_ver = '1.0.0'
+ latest_ver = '1.2.0'
+ self.assertFalse(version_comparer.compare_dependency_versions(curr_ver,
latest_ver))
+
+ def test_compare_same_version_false(self):
+ curr_ver = '1.0.0'
+ latest_ver = '1.0.0'
+ self.assertFalse(version_comparer.compare_dependency_versions(curr_ver,
latest_ver))
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/.test-infra/jenkins/jira_utils/jira_manager.py
b/.test-infra/jenkins/jira_utils/jira_manager.py
index 5ef364e6dac..b52f9cb67b6 100644
--- a/.test-infra/jenkins/jira_utils/jira_manager.py
+++ b/.test-infra/jenkins/jira_utils/jira_manager.py
@@ -17,14 +17,19 @@
#
import logging
-import yaml
+import os
import traceback
+import yaml
+import dependency_check.version_comparer as version_comparer
+
from datetime import datetime
from jira_client import JiraClient
+
_JIRA_PROJECT_NAME = 'BEAM'
_JIRA_COMPONENT = 'dependencies'
_ISSUE_SUMMARY_PREFIX = 'Beam Dependency Update Request: '
+_ISSUE_REOPEN_DAYS = 180
class JiraManager:
@@ -40,11 +45,16 @@ def __init__(self, jira_url, jira_username, jira_password,
owners_file):
logging.getLogger().setLevel(logging.INFO)
- def run(self, dep_name, dep_latest_version, sdk_type, group_id=None):
+ def run(self, dep_name,
+ dep_current_version,
+ dep_latest_version,
+ sdk_type,
+ group_id=None):
"""
Manage the jira issue for a dependency
Args:
dep_name,
+ dep_current_version,
dep_latest_version,
sdk_type: Java, Python
group_id (optional): only required for Java dependencies
@@ -67,8 +77,7 @@ def run(self, dep_name, dep_latest_version, sdk_type,
group_id=None):
logging.info("""Did not find existing issue with name {0}. \n
Created a parent issue for {1}""".format(summary, group_id))
try:
- parent_issue = self._create_issue(group_id, None)
- print(parent_issue.key)
+ parent_issue = self._create_issue(group_id, None, None)
except:
logging.error("""Failed creating a parent issue for {0}.
Stop handling the JIRA issue for {1}, {2}""".format(group_id,
dep_name, dep_latest_version))
@@ -85,29 +94,37 @@ def run(self, dep_name, dep_latest_version, sdk_type,
group_id=None):
Stop handling the JIRA issue for {1},
{2}""".format(parent_issue.key, dep_name, dep_latest_version))
return
logging.info("Found the parent issue {0}. Continuous to create or
update the sub-task for {1}".format(parent_issue.key, dep_name))
+
# creating a new issue/sub-task or updating on the existing issue of the
dep
- summary = _ISSUE_SUMMARY_PREFIX + dep_name + " " + dep_latest_version
+ summary = _ISSUE_SUMMARY_PREFIX + dep_name
issues = self._search_issues(summary)
issue = None
for i in issues:
if i.fields.summary == summary:
issue = i
break
+ # Create a new JIRA if no existing one.
if not issue:
if sdk_type == 'Java':
- issue = self._create_issue(dep_name, dep_latest_version,
is_subtask=True, parent_key=parent_issue.key)
+ issue = self._create_issue(dep_name, dep_current_version,
dep_latest_version, is_subtask=True, parent_key=parent_issue.key)
else:
- issue = self._create_issue(dep_name, dep_latest_version)
+ issue = self._create_issue(dep_name, dep_current_version,
dep_latest_version)
logging.info('Created a new issue {0} of {1} {2}'.format(issue.key,
dep_name, dep_latest_version))
+ # Add descriptions in to the opening issue.
elif issue.fields.status.name == 'Open' or issue.fields.status.name ==
'Reopened':
- self._append_descriptions(issue, dep_name, dep_latest_version)
+ self._append_descriptions(issue, dep_name, dep_current_version,
dep_latest_version)
logging.info('Updated the existing issue {0} of {1}
{2}'.format(issue.key, dep_name, dep_latest_version))
+ # Check if we need reopen the issue if it was closed. If so, reopen it
then add descriptions.
+ elif self._need_reopen(issue, dep_latest_version):
+ self.jira.reopen_issue(issue)
+ self._append_descriptions(issue, dep_name, dep_current_version,
dep_latest_version)
+ logging.info("Reopened the issue {0} for {1} {2}".format(issue.key,
dep_name, dep_latest_version))
return issue
except:
raise
- def _create_issue(self, dep_name, dep_latest_version, is_subtask=False,
parent_key=None):
+ def _create_issue(self, dep_name, dep_current_version, dep_latest_version,
is_subtask=False, parent_key=None):
"""
Create a new issue or subtask
Args:
@@ -117,24 +134,13 @@ def _create_issue(self, dep_name, dep_latest_version,
is_subtask=False, parent_k
parent_key: only required if the 'is_subtask'is true.
"""
logging.info("Creating a new JIRA issue to track {0} upgrade
process".format(dep_name))
- assignee, owners = self._find_owners(dep_name)
summary = _ISSUE_SUMMARY_PREFIX + dep_name
- if dep_latest_version:
- summary = summary + " " + dep_latest_version
- description = """\n\n{0}\n
- Please review and upgrade the {1} to the latest version {2} \n
- cc: """.format(
- datetime.today(),
- dep_name,
- dep_latest_version
- )
- for owner in owners:
- description += "[~{0}], ".format(owner)
+ description = self._create_descriptions(dep_name, dep_current_version,
dep_latest_version)
try:
if not is_subtask:
- issue = self.jira.create_issue(summary, [_JIRA_COMPONENT],
description, assignee=assignee)
+ issue = self.jira.create_issue(summary, [_JIRA_COMPONENT], description)
else:
- issue = self.jira.create_issue(summary, [_JIRA_COMPONENT],
description, assignee=assignee, parent_key=parent_key)
+ issue = self.jira.create_issue(summary, [_JIRA_COMPONENT],
description, parent_key=parent_key)
except Exception as e:
logging.error("Failed creating issue: "+ str(e))
raise e
@@ -157,7 +163,7 @@ def _search_issues(self, summary):
return issues
- def _append_descriptions(self, issue, dep_name, dep_latest_version):
+ def _append_descriptions(self, issue, dep_name, dep_current_version,
dep_latest_version):
"""
Add descriptions on an existing issue.
Args:
@@ -168,21 +174,42 @@ def _append_descriptions(self, issue, dep_name,
dep_latest_version):
logging.info("Updating JIRA issue {0} to track {1} upgrade process".format(
issue.key,
dep_name))
- description = issue.fields.description + """\n\n{0}\n
- Please review and upgrade the {1} to the latest version {2} \n
+ description = self._create_descriptions(dep_name, dep_current_version,
dep_latest_version, issue=issue)
+ try:
+ self.jira.update_issue(issue, description=description)
+ except Exception as e:
+ traceback.print_exc()
+ logging.error("Failed updating issue: "+ str(e))
+
+
+ def _create_descriptions(self, dep_name, dep_current_version,
dep_latest_version, issue = None):
+ """
+ Create descriptions for JIRA issues.
+ Args:
+ dep_name
+ dep_latest_version
+ issue
+ """
+ description = ""
+ if issue:
+ description = issue.fields.description
+ description += """\n\n ------------------------- {0}
-------------------------\n
+ Please consider upgrading the dependency {1}. \n
+ The current version is {2}. The latest version is {3} \n
cc: """.format(
datetime.today(),
dep_name,
+ dep_current_version,
dep_latest_version
)
- _, owners = self._find_owners(dep_name)
+ owners = self._find_owners(dep_name)
for owner in owners:
description += "[~{0}], ".format(owner)
- try:
- self.jira.update_issue(issue, description=description)
- except Exception as e:
- traceback.print_exc()
- logging.error("Failed updating issue: "+ str(e))
+ description += ("\n Please refer to "
+ "[Beam Dependency Guide
|https://beam.apache.org/contribute/dependencies/]";
+ "for more information. \n"
+ "Do Not Modify The Description Above. \n")
+ return description
def _find_owners(self, dep_name):
@@ -199,20 +226,67 @@ def _find_owners(self, dep_name):
owners = dep_info['owners']
if not owners:
logging.warning("Could not find owners for " + dep_name)
- return None, []
+ return []
except KeyError:
traceback.print_exc()
logging.warning("Could not find the dependency info of {0} in the OWNERS
configurations.".format(dep_name))
- return None, []
+ return []
except Exception as e:
traceback.print_exc()
logging.error("Failed finding dependency owners: "+ str(e))
- return None, None
+ return None
logging.info("Found owners of {0}: {1}".format(dep_name, owners))
owners = owners.split(',')
owners = map(str.strip, owners)
owners = list(filter(None, owners))
- primary = owners[0]
- del owners[0]
- return primary, owners
+ return owners
+
+
+ def _need_reopen(self, issue, dep_latest_version):
+ """
+ Return a boolean that indicates whether reopen the closed issue.
+ """
+ # Check if the issue was closed with a "fix version/s"
+ # Reopen the issue if it hits the next release version.
+ next_release_version = self._get_next_release_version()
+ for fix_version in issue.fields.fixVersions:
+ if fix_version.name in next_release_version:
+ return True
+
+ # Check if there is other new versions released.
+ # Reopen the issue if 3 new versions have been released in 6 month since
closure.
+ try:
+ if issue.fields.resolutiondate:
+ closing_date = datetime.strptime(issue.fields.resolutiondate[:19],
"%Y-%m-%dT%H:%M:%S")
+ if (datetime.today() - closing_date).days >= _ISSUE_REOPEN_DAYS:
+ # Extract the previous version when JIRA closed.
+ descriptions = issue.fields.description.splitlines()
+ descriptions = descriptions[len(descriptions)-5]
+ # The version info has been stored in the JIRA description in a
specific format.
+ # Such as "Please review and upgrade the <dep name> to the latest
version <version>"
+ previous_version = descriptions.split("The latest version is",
1)[1].strip()
+ if version_comparer.compare_dependency_versions(previous_version,
dep_latest_version):
+ return True
+ except Exception as e:
+ traceback.print_exc()
+ logging.error("Failed deciding to reopen the issue." + str(e))
+ return False
+
+ return False
+
+
+ def _get_next_release_version(self):
+ """
+ Return the incoming release version from sdks/python/apache_beam/version.py
+ """
+ global_names = {}
+ exec(
+ open(os.path.join(
+ os.path.dirname(os.path.abspath(__file__)),
+ '../../../sdks/python/',
+ 'apache_beam/version.py')
+ ).read(),
+ global_names
+ )
+ return global_names['__version__']
diff --git a/.test-infra/jenkins/jira_utils/jira_manager_test.py
b/.test-infra/jenkins/jira_utils/jira_manager_test.py
index dcf11a69e5c..0c9afa6399c 100644
--- a/.test-infra/jenkins/jira_utils/jira_manager_test.py
+++ b/.test-infra/jenkins/jira_utils/jira_manager_test.py
@@ -17,10 +17,13 @@
#
import unittest, mock
+import jira_utils
from mock import patch, mock_open, Mock
from jira_manager import JiraManager
from datetime import datetime
+MOCKED_DEP_CURRENT_VERSION = '0.1.0'
+MOCKED_DEP_LATEST_VERSION = '1.0.0'
class MockedJiraIssue:
def __init__(self, key, summary, description, status):
@@ -32,11 +35,18 @@ def __init__(self, summary, description, status):
self.summary = summary
self.description = description
self.status = self.MockedJiraIssueStatus(status)
+ self.fixVersions = [self.MockedJiraIssueFixVersions()]
+ if status == 'Closed':
+ self.resolutiondate = '1999-01-01T00:00:00'
class MockedJiraIssueStatus:
def __init__(self, status):
self.name = status
+ class MockedJiraIssueFixVersions:
+ def __init__(self):
+ self.name = '2.8.0'
+
@patch('jira_utils.jira_manager.JiraClient')
class JiraManagerTest(unittest.TestCase):
@@ -46,76 +56,6 @@ def setUp(self):
print("\n\nTest : " + self._testMethodName)
- def test_find_owners_with_single_owner(self, *args):
- """
- Test on _find_owners with single owner
- Expect: the primary owner is 'owner0', an empty list of other owners.
- """
- owners_yaml = """
- deps:
- dep0:
- owners: owner0,
- """
- with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
- manager = JiraManager('url', 'username', 'password', owners_yaml)
- primary, owners = manager._find_owners('dep0')
- self.assertEqual(primary, 'owner0')
- self.assertEqual(len(owners), 0)
-
-
- def test_find_owners_with_multi_owners(self, *args):
- """
- Test on _find_owners with multiple owners.
- Expect: the primary owner is 'owner0', a list contains 'owner1' and
'owner2'.
- """
- owners_yaml = """
- deps:
- dep0:
- owners: owner0, owner1 , owner2,
- """
- with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
- manager = JiraManager('url', 'username', 'password', owners_yaml)
- primary, owners = manager._find_owners('dep0')
- self.assertEqual(primary, 'owner0')
- self.assertEqual(len(owners), 2)
- self.assertIn('owner1', owners)
- self.assertIn('owner2', owners)
-
-
- def test_find_owners_with_no_owners_defined(self, *args):
- """
- Test on _find_owners without owner.
- Expect: the primary owner is None, an empty list of other owners.
- """
- owners_yaml = """
- deps:
- dep0:
- owners:
- """
- with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
- manager = JiraManager('url', 'username', 'password', owners_yaml)
- primary, owners = manager._find_owners('dep0')
- self.assertIsNone(primary)
- self.assertEqual(len(owners), 0)
-
-
- def test_find_owners_with_no_dep_defined(self, *args):
- """
- Test on _find_owners with non-defined dep.
- Expect: through out KeyErrors. The primary owner is None, an empty list of
other owners.
- """
- owners_yaml = """
- deps:
- dep0:
- owners:
- """
- with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
- manager = JiraManager('url', 'username', 'password', owners_yaml)
- primary, owners = manager._find_owners('dep1')
- self.assertIsNone(primary)
- self.assertEqual(len(owners), 0)
-
-
@patch('jira_utils.jira_manager.datetime',
Mock(today=Mock(return_value=datetime.strptime('2000-01-01', '%Y-%m-%d'))))
def test_run_with_creating_new_issue(self, *args):
"""
@@ -129,11 +69,14 @@ def test_run_with_creating_new_issue(self, *args):
"""
with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
manager = JiraManager('url', 'username', 'password', owners_yaml)
- manager.run('dep0', '1.0', 'Python')
-
manager.jira.create_issue.assert_called_once_with(self._get_experct_summary('dep0',
'1.0'),
+ manager.run('dep0',
+ MOCKED_DEP_CURRENT_VERSION,
+ MOCKED_DEP_LATEST_VERSION,
+ 'Python')
+
manager.jira.create_issue.assert_called_once_with(self._get_experct_summary('dep0'),
['dependencies'],
-
self._get_expected_description('dep0', '1.0', ['owner1', 'owner2']),
- assignee='owner0')
+
self._get_expected_description('dep0', MOCKED_DEP_LATEST_VERSION, ['owner0',
'owner1', 'owner2']),
+ )
@patch('jira_utils.jira_manager.datetime',
Mock(today=Mock(return_value=datetime.strptime('2000-01-01', '%Y-%m-%d'))))
@@ -143,20 +86,22 @@ def test_run_with_updating_existing_task(self, *args):
Expect: jira.update_issue is called once.
"""
dep_name = 'dep0'
- dep_latest_version = '1.0'
owners_yaml = """
deps:
dep0:
owners:
"""
- summary = self._get_experct_summary(dep_name, dep_latest_version)
- description = self._get_expected_description(dep_name, dep_latest_version,
[])
+ summary = self._get_experct_summary(dep_name)
+ description = self._get_expected_description(dep_name,
MOCKED_DEP_LATEST_VERSION, [])
with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
with patch('jira_utils.jira_manager.JiraManager._search_issues',
return_value=[MockedJiraIssue('BEAM-1000', summary, description,
'Open')]):
manager = JiraManager('url', 'username', 'password', owners_yaml)
- manager.run(dep_name, dep_latest_version, 'Python')
+ manager.run(dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ MOCKED_DEP_LATEST_VERSION,
+ 'Python')
manager.jira.update_issue.assert_called_once()
@@ -167,7 +112,6 @@ def test_run_with_creating_new_subtask(self, *args):
Expect: jira.create_issue is called once with certain parameters.
"""
dep_name = 'group0:artifact0'
- dep_latest_version = '1.0'
owners_yaml = """
deps:
group0:artifact0:
@@ -175,19 +119,22 @@ def test_run_with_creating_new_subtask(self, *args):
artifact: artifact0
owners: owner0
"""
- summary = self._get_experct_summary('group0', None)
- description = self._get_expected_description(dep_name, dep_latest_version,
[])
+ summary = self._get_experct_summary('group0')
+ description = self._get_expected_description(dep_name,
MOCKED_DEP_LATEST_VERSION, [])
with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
with patch('jira_utils.jira_manager.JiraManager._search_issues',
side_effect = [[MockedJiraIssue('BEAM-1000', summary, description,
'Open')],
[]]):
manager = JiraManager('url', 'username', 'password', owners_yaml)
- manager.run(dep_name, dep_latest_version, 'Java', group_id='group0')
-
manager.jira.create_issue.assert_called_once_with(self._get_experct_summary(dep_name,
dep_latest_version),
+ manager.run(dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ MOCKED_DEP_LATEST_VERSION,
+ 'Java',
+ group_id='group0')
+
manager.jira.create_issue.assert_called_once_with(self._get_experct_summary(dep_name),
['dependencies'],
-
self._get_expected_description(dep_name, dep_latest_version, []),
- assignee='owner0',
+
self._get_expected_description(dep_name, MOCKED_DEP_LATEST_VERSION, ['owner0']),
parent_key='BEAM-1000',
)
@@ -200,7 +147,6 @@ def test_run_with_reopening_existing_parent_issue(self,
*args):
Expect: jira.reopen_issue is called once.
"""
dep_name = 'group0:artifact0'
- dep_latest_version = '1.0'
owners_yaml = """
deps:
group0:artifact0:
@@ -208,34 +154,99 @@ def test_run_with_reopening_existing_parent_issue(self,
*args):
artifact: artifact0
owners: owner0
"""
- summary = self._get_experct_summary('group0', None)
- description = self._get_expected_description(dep_name, dep_latest_version,
[])
+ summary = self._get_experct_summary('group0')
+ description = self._get_expected_description(dep_name,
MOCKED_DEP_LATEST_VERSION, [])
with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
with patch('jira_utils.jira_manager.JiraManager._search_issues',
side_effect = [[MockedJiraIssue('BEAM-1000', summary, description,
'Closed')],
[]]):
manager = JiraManager('url', 'username', 'password', owners_yaml)
- manager.run(dep_name, dep_latest_version, sdk_type='Java',
group_id='group0')
+ manager.run(dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ MOCKED_DEP_LATEST_VERSION,
+ sdk_type='Java',
+ group_id='group0')
+ manager.jira.reopen_issue.assert_called_once()
+
+
+ @patch('jira_utils.jira_manager.datetime',
Mock(today=Mock(return_value=datetime.strptime('2000-01-01', '%Y-%m-%d'))))
+ @patch.object(jira_utils.jira_manager.JiraManager,
+ '_get_next_release_version', side_effect=['2.8.0.dev'])
+ def test_run_with_reopening_issue_with_fixversions(self, *args):
+ """
+ Test JiraManager.run on reopening an issue when JIRA fixVersions hits the
release version.
+ Expect: jira.reopen_issue is called once.
+ """
+ dep_name = 'dep0'
+ owners_yaml = """
+ deps:
+ dep0:
+ owners:
+ """
+ summary = self._get_experct_summary(dep_name)
+ description = self._get_expected_description(dep_name,
MOCKED_DEP_LATEST_VERSION, [])
+ with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
+ with patch('jira_utils.jira_manager.JiraManager._search_issues',
+ side_effect = [[MockedJiraIssue('BEAM-1000', summary,
description, 'Closed')],
+ []]):
+ manager = JiraManager('url', 'username', 'password', owners_yaml)
+ manager.run(dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ MOCKED_DEP_LATEST_VERSION,
+ 'Python')
+ manager.jira.reopen_issue.assert_called_once()
+
+
+ @patch.object(jira_utils.jira_manager.JiraManager,
+ '_get_next_release_version', side_effect=['2.9.0.dev'])
+ def test_run_with_reopening_issue_with_new_release_available(self, *args):
+ """
+ Test JiraManager.run that reopens an issue once 3 versions releases after 6
+ months since previous closure.
+ Expect: jira.reopen_issue is called once.
+ """
+ dep_name = 'dep0'
+ issue_closed_version = '1.0'
+ dep_latest_version = '1.3'
+ owners_yaml = """
+ deps:
+ dep0:
+ owners:
+ """
+ summary = self._get_experct_summary(dep_name)
+ description = self._get_expected_description(dep_name,
issue_closed_version, [])
+ with patch('__builtin__.open', mock_open(read_data=owners_yaml)):
+ with patch('jira_utils.jira_manager.JiraManager._search_issues',
+ side_effect = [[MockedJiraIssue('BEAM-1000', summary,
description, 'Closed')],
+ []]):
+ manager = JiraManager('url', 'username', 'password', owners_yaml)
+ manager.run(dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ dep_latest_version,
+ 'Python')
manager.jira.reopen_issue.assert_called_once()
- def _get_experct_summary(self, dep_name, dep_latest_version):
- summary = 'Beam Dependency Update Request: ' + dep_name
- if dep_latest_version:
- summary = summary + " " + dep_latest_version
- return summary
+ def _get_experct_summary(self, dep_name):
+ return 'Beam Dependency Update Request: ' + dep_name
def _get_expected_description(self, dep_name, dep_latest_version, owners):
- description = """\n\n{0}\n
- Please review and upgrade the {1} to the latest version {2} \n
+ description = """\n\n ------------------------- {0}
-------------------------\n
+ Please consider upgrading the dependency {1}. \n
+ The current version is {2}. The latest version is {3} \n
cc: """.format(
- datetime.strptime('2000-01-01', '%Y-%m-%d'),
- dep_name,
- dep_latest_version
+ "2000-01-01 00:00:00",
+ dep_name,
+ MOCKED_DEP_CURRENT_VERSION,
+ dep_latest_version
)
for owner in owners:
description += "[~{0}], ".format(owner)
+ description += ("\n Please refer to "
+ "[Beam Dependency Guide
|https://beam.apache.org/contribute/dependencies/]";
+ "for more information. \n"
+ "Do Not Modify The Description Above. \n")
return description
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 148573)
Time Spent: 4h 10m (was: 4h)
> Implement new policy on Beam dependency tooling
> -----------------------------------------------
>
> Key: BEAM-5339
> URL: https://issues.apache.org/jira/browse/BEAM-5339
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: yifan zou
> Assignee: yifan zou
> Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> (1) Instead of a dependency "owners" list we will be maintaining an
> "interested parties" list. When we create a JIRA for a dependency we will not
> assign it to an owner but rather we will CC all the folks that mentioned that
> they will be interested in receiving updates related to that dependency. Hope
> is that some of the interested parties will also put forward the effort to
> upgrade dependencies they are interested in but the responsibility of
> upgrading dependencies lie with the community as a whole.
> (2) We will be creating JIRAs for upgrading individual dependencies, not for
> upgrading to specific versions of those dependencies. For example, if a given
> dependency X is three minor versions or an year behind we will create a JIRA
> for upgrading that. But the specific version to upgrade to has to be
> determined by the Beam community. Beam community might choose to close a JIRA
> if there are known issues with available recent releases. Tool may reopen
> such a closed JIRA in the future if new information becomes available (for
> example, 3 new versions have been released since JIRA was closed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
