Hi guys,
some of you were on the Monday's meeting, where discussion raised a
request for automatic pull request checker and next generator. I took a
look on it and implemented very basic version just to see, whether it's
worth it. Please note that some parts are hardcoded to my setup.
Currently it only:
1) checks all open pull requests for changes from previous run
2) runs check_patch and boot test on each changed pull request. (changed
commits, not comments)
3) if there are pull requests, which were not used in the previous
next-branch or were used and shouldn't be used now, next-branch is
freshly recreated. If it passes the boot test, can be pushed to the
original repo (currently commented out).
4) update the github pull request comments (currently only prints the
informations to the cmdline but should be simple to push these messages
to each pull request.
5) stores the information to github_checker.sav for the next run
Output of the first run:
--< UPDATE >--
Checking changes of PR1438
Checking PR1438...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1438
git checkout master --force
git branch -D github-1438
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
PASS
Checking changes of PR1437
Checking PR1437...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1437
git checkout master --force
git branch -D github-1437
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
PASS
Checking changes of PR1436
Checking PR1436...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1436
git checkout master --force
git branch -D github-1436
FAIL (checkpatch)
git am --abort
Checking changes of PR1435
Checking PR1435...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1435
git checkout master --force
git branch -D github-1435
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
PASS
Checking changes of PR1434
Checking PR1434...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1434
git checkout master --force
git branch -D github-1434
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
PASS
Recreating next
git fetch upstream
git checkout next
git reset --hard upstream/master
Merging PR1434...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1434.patch -O
/var/tmp/tmpL2zCmD
git am -3 /var/tmp/tmpL2zCmD
PASS
Merging PR1435...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1435.patch -O
/var/tmp/tmpe2388C
git am -3 /var/tmp/tmpe2388C
PASS
Merging PR1437...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1437.patch -O
/var/tmp/tmpFKVbkQ
git am -3 /var/tmp/tmpFKVbkQ
PASS
Merging PR1438...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1438.patch -O
/var/tmp/tmpSkQBs4
git am -3 /var/tmp/tmpSkQBs4
PASS
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
Next is ready
git checkout master
--< MESSAGES >--
PR1434
check_patch :+1:
boot_test :+1:
PR1435
check_patch :+1:
boot_test :+1:
PR1436
check_patch :-1:
PR1437
check_patch :+1:
boot_test :+1:
PR1438
check_patch :+1:
boot_test :+1:
Output of the second run:
--< UPDATE >--
Checking changes of PR1438
Checking changes of PR1437
Checking changes of PR1436
Checking changes of PR1435
Checking changes of PR1434
--< MESSAGES >--
As you can see there were no changes, no new messages in pull request
and no next recreation in the second run. Now output of another run,
when two new pull requests are added:
--< UPDATE >--
Checking changes of PR1438
Checking changes of PR1437
Checking changes of PR1436
Checking changes of PR1435
Checking changes of PR1434
Checking changes of PR1433
Checking PR1433...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1433
git checkout master --force
git branch -D github-1433
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
PASS
Checking changes of PR1431
Checking PR1431...
AUTOTEST_PATH=../../../ tools/check_patch.py -g 1431
git checkout master --force
git branch -D github-1431
FAIL (checkpatch)
git am --abort
Recreating next
git fetch upstream
git checkout next
git reset --hard upstream/master
Merging PR1433...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1433.patch -O
/var/tmp/tmpTixM1R
git am -3 /var/tmp/tmpTixM1R
PASS
Merging PR1434...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1434.patch -O
/var/tmp/tmpDzglAn
git am -3 /var/tmp/tmpDzglAn
PASS
Merging PR1435...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1435.patch -O
/var/tmp/tmpESs4ja
git am -3 /var/tmp/tmpESs4ja
PASS
Merging PR1437...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1437.patch -O
/var/tmp/tmps6Uki9
git am -3 /var/tmp/tmps6Uki9
PASS
Merging PR1438...
git rev-parse HEAD
wget https://github.com/autotest/virt-test/pull/1438.patch -O
/var/tmp/tmp7tlW4Q
git am -3 /var/tmp/tmp7tlW4Q
PASS
AUTOTEST_PATH=../../../ ./run -t qemu --tests='boot' --no-downloads
-k --keep-image-between-tests
Next is ready
git checkout master
--< MESSAGES >--
PR1431
check_patch :-1:
PR1433
check_patch :+1:
boot_test :+1:
...
there is one additional message not shown here:
Potential conflict with other pull request, not applying to next. :-1:
which means that patch which is applicable on master has conflicts when
applying on next (some previous pull requests collides with it)
As you can see it found them successfully, checked only the new ones,
found that the 1431 fails to apply on current master so ignored it while
recreating the next.
Honestly I don't see very big income. Users should run check_patch
themselves to see, whether their pull request is ready. Similarly no
maintainer should apply test unless he tests, that "boot" test is not
screwed up. Anyway, consider this as a discussion starter, maybe I just
didn't get the point correctly.
Kind regards,
Lukáš
#!/usr/bin/env python
import sys
import os
import getpass
import datetime
import github
import tempfile
import subprocess
from astroid.node_classes import Exec
import pickle
def execute(cmd, ignore_status=False):
"""
Executes command using subprocess
:return: tuple($status, $output)
"""
print " " + cmd
process = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
out = process.communicate()
ret = process.poll()
if ret and not ignore_status:
raise OSError("Fail to execute %s (%s)\nSTDOUT:\n%s\nSTDERR:\n%s"
% (cmd, ret, out[0], out[1]))
return (ret, "\n".join(out))
def test_boot():
"""
Test virt-test using 'boot' test
:return: True on error
"""
ret, out = execute("AUTOTEST_PATH=../../../ "
"./run -t qemu --tests='boot' --no-downloads -k "
"--keep-image-between-tests", False)
if not ret and 'boot: PASS' in out:
return False # Correctly executed
else:
# Remove the image and try it again
execute("/var/lib/virt_test/images/jeos-19-64.qcow2", False)
ret, out = execute("AUTOTEST_PATH=../../../ "
"./run -t qemu --tests='boot'", False)
if not ret and 'boot: PASS' in out:
return False
return True
class GITTree(list):
@staticmethod
def _merge_pull_request(pr):
"""
Try to merge pull request
"""
print "Merging PR%s..." % pr.pr.number
old_head = execute("git rev-parse HEAD")[1].strip()
patch_file = tempfile.NamedTemporaryFile('w', dir="/var/tmp/",
delete=True)
execute("wget %s -O %s" % (pr.pr.patch_url, patch_file.name))
ret = execute("git am -3 %s" % patch_file.name, True)[0]
patch_file.close()
if ret:
execute("git am --abort", True)
execute("git reset --hard %s" % old_head)
pr.set_valid(False)
pr.add_message("Potential conflict with other pull "
"request, not applying to next. :-1:\n")
print "FAIL"
return
print "PASS"
def recreate(self, prs):
"""
Recreate next using prs pull requests
"""
print "Recreating next"
execute("git fetch origin")
execute("git checkout next")
execute("git reset --hard origin/master")
for pr in prs:
if pr.is_valid():
self._merge_pull_request(pr)
if test_boot():
print "Next is not executable"
execute("git checkout master")
return
print "Next is ready"
# TODO: Uncomment this if you want next to be automatically updated
# execute("git push origin next --force")
execute("git checkout master")
class PullRequest(object):
"""
Pull request with additional data
"""
def __init__(self, pull_request):
self.__commits = []
self.__valid = True
self.__message = ""
self.pr = pull_request
def is_valid(self):
return self.__valid
def set_valid(self, status):
self.__valid = status
def check(self):
"""
Check this pull request
"""
i = 0
old_len = len(self.__commits)
for commit in self.pr.get_commits():
if i >= old_len or commit.sha != self.__commits[i]:
return self._run_check_patch()
i += 1
if i != old_len:
return self._run_check_patch()
def _run_check_patch(self):
"""
Runs checkpatch
:return: True when __valid status changes
"""
print "Checking PR%s..." % self.pr.number
ret = execute("AUTOTEST_PATH=../../../ tools/check_patch.py -g %s"
% self.pr.number, True)[0]
# Some pull request might add files, which might be removed by checkout
execute("git checkout master --force", True)
execute("git branch -D github-%s" % self.pr.number, True)
self.__commits = [commit.sha for commit in self.pr.get_commits()]
if ret: # Checkpatch failed
print "FAIL (checkpatch)"
execute("git am --abort", True)
self.add_message("check_patch :-1:")
if self.is_valid:
self.set_valid(False)
return True
else:
return False
# Checkpatch passed, run additional tests
self.add_message("check_patch :+1:")
# Put additional tests here
if test_boot():
self.add_message("boot_test :-1:")
if self.is_valid():
self.set_valid(False)
print "FAIL (boot)"
return True
else:
self.add_message("boot_test :+1:")
print "PASS"
if not self.is_valid():
self.set_valid(True)
return True
return False
def sync_message(self):
"""
Sends messages as pull request comments
"""
if self.__message:
# TODO: Send to github
print "PR%s\n %s" % (self.pr.number,
self.__message[:-1].replace("\n", "\n "))
self.__message = ""
def add_message(self, msg):
"""
Add message
"""
self.__message += "%s\n" % msg
class PRCache(object):
def __init__(self):
self.__cache = {}
self.__numbers = set()
self.__idx = 0
def __iter__(self):
for number in self.__numbers:
yield self.__cache[number]
def update(self, prs):
"""
Check for changes, add new ones, removes old ones.
:param prs: pull requests
:return: True if new or updated acceptable pull request is present
"""
changed = False
_prs = set()
for pr in prs:
if self.check(pr):
changed = True
_prs.add(pr.number)
for pr in self.__numbers.difference(_prs):
if self.remove(pr):
changed = True
return changed
def check(self, pr):
"""
Check pull request for changes
:return: True when pull request which is ready to next changes
"""
print "Checking changes of PR%s" % pr.number
if pr.number not in self.__numbers:
self.add(pr)
return self.__cache[pr.number].check()
def add(self, pr):
"""
Adds pull request to this cache
"""
self.__cache[pr.number] = PullRequest(pr)
self.__numbers.add(pr.number)
def remove(self, number):
"""
Removes pull request from cache
"""
if number not in self.__cache:
raise IndexError(number)
changed = False
if self.__cache[number].is_valid():
changed = True
del(self.__cache[number])
self.__numbers.remove(number)
return changed
class GHChecker(object):
def __init__(self, gh, repo):
self.__gh = gh
self.__repo = gh.get_repo(repo)
# Cache of open pull request and their status
self.__pr_cache = PRCache()
# Branch with all good open pull request applied
self.__next_tree = GITTree()
def update(self):
print "--< UPDATE >--"
if self.__pr_cache.update(self.__repo.get_pulls(state="open")):
self.__next_tree.recreate(self.__pr_cache)
print "--< MESSAGES >--"
for pr in self.__pr_cache:
pr.sync_message()
if __name__ == "__main__":
try:
checker = pickle.load(open('github_checker.sav', 'rb'))
except Exception:
print "Previous state load failed"
#gh = github.Github()
gh = github.Github(login_or_token=raw_input("Enter github username: "),
password=getpass.getpass('Enter github password: '))
repo = 'autotest/virt-test'
checker = GHChecker(gh, repo)
checker.update()
pickle.dump(checker, open('github_checker.sav', 'wb'))
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
