Hi Martin,

I have started to correct quite a lot of issues I have with Python on AIX, and since I had to test quite a lot of patchs, I though it would be more convenient to setup a buildbot for that platform.

So I now have a buildbot environment with 2 slaves (AIX 5.3 and 6.1) that builds and tests Python (branch py3k) with both gcc and xlc (the native AIX compiler) (I have 4 builders ("py3k-aix6-xlc", "py3k-aix5-xlc", "py3k-aix6-gcc", "py3k-aix5-gcc").

I expect to add 4 more builders for branch 2.7 in coming days.

I would like to share the results of this buildbot to the Python community so that issues with AIX could be addressed more easily.

R. David Murray pointed me to the page on the python wiki concerning buildbot. It is stated there that is is possible to connect some slaves to some official Python buildbot master.

Unfortunately, I don't think this solution is possible for me: I don't think the security team in my company would appreciate that a server inside our network runs some arbitrary shell commands provided by some external source. Neither can I expose the buildbot master web interface.

Also I had to customize the buildbot rules in order to work with some specificities of AIX (see attached master.cfg), and I can't guarantee that this buildbot will run 24 hours a day; I may have to schedule it only once at night for example if it consumes too much resources.

(And the results are very unstable at the moment, mostly because of issue 9862).

On the other hand, I could upload the build results with rsync or scp somewhere or setup some MailNotifier if that can help.

How do you think I could share those results?

regards

--
Sébastien Sablé



Le 15/09/2010 23:28, R. David Murray a écrit :

R. David Murray  added the comment:

Sébastien, you could email Martin (tracker id loewis) about adding your 
buildbot to our unstable fleet (or even to stable if it is stable; that is, the 
tests normally pass and don't randomly fail).  As long as you are around to 
help fix bugs it would be great to have an aix buildbot in our buildbot fleet.

(NB: see also http://wiki.python.org/moin/BuildBot, which unfortunately is a 
bit out of date...)

----------
nosy: +r.david.murray

_______________________________________
Python tracker<rep...@bugs.python.org>
<http://bugs.python.org/issue1633863>
_______________________________________

# -*- python -*-
# ex: set syntax=python:

# This is a sample buildmaster config file. It must be installed as
# 'master.cfg' in your buildmaster's base directory (although the filename
# can be changed with the --basedir option to 'mktap buildbot master').

# It has one job: define a dictionary named BuildmasterConfig. This
# dictionary has a variety of keys to control different aspects of the
# buildmaster. They are documented in docs/config.xhtml .

# This is the dictionary that the buildmaster pays attention to. We also use
# a shorter alias to save typing.
c = BuildmasterConfig = {}

####### BUILDSLAVES

# the 'slaves' list defines the set of allowable buildslaves. Each element is
# a BuildSlave object, which is created with bot-name, bot-password.  These
# correspond to values given to the buildslave's mktap invocation.
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave("phenix", "bot1passwd", max_builds=1),
               BuildSlave("sirius", "bot2passwd", max_builds=1)]

# to limit to two concurrent builds on a slave, use
#  c['slaves'] = [BuildSlave("bot1name", "bot1passwd", max_builds=2)]


# 'slavePortnum' defines the TCP port to listen on. This must match the value
# configured into the buildslaves (with their --master option)

c['slavePortnum'] = 9989

####### CHANGESOURCES

# the 'change_source' setting tells the buildmaster how it should find out
# about source code changes. Any class which implements IChangeSource can be
# put here: there are several in buildbot/changes/*.py to choose from.

from buildbot.changes.pb import PBChangeSource
c['change_source'] = PBChangeSource()

# For example, if you had CVSToys installed on your repository, and your
# CVSROOT/freshcfg file had an entry like this:
#pb = ConfigurationSet([
#    (None, None, None, PBService(userpass=('foo', 'bar'), port=4519)),
#    ])

# then you could use the following buildmaster Change Source to subscribe to
# the FreshCVS daemon and be notified on every commit:
#
#from buildbot.changes.freshcvs import FreshCVSSource
#fc_source = FreshCVSSource("cvs.example.com", 4519, "foo", "bar")
#c['change_source'] = fc_source

# or, use a PBChangeSource, and then have your repository's commit script run
# 'buildbot sendchange', or use contrib/svn_buildbot.py, or
# contrib/arch_buildbot.py :
#
#from buildbot.changes.pb import PBChangeSource
#c['change_source'] = PBChangeSource()

# If you wat to use SVNPoller, it might look something like
#  # Where to get source code changes
# from buildbot.changes.svnpoller import SVNPoller
# source_code_svn_url='https://svn.myproject.org/bluejay/trunk'
# svn_poller = SVNPoller(
#                    svnurl=source_code_svn_url,
#                    pollinterval=60*60, # seconds
#                    histmax=10,
#                    svnbin='/usr/bin/svn',
## )
# c['sources'] = [ svn_poller ]

from buildbot.changes.svnpoller import SVNPoller
source_code_svn_url='http://svn.python.org/projects/python/branches/py3k'
svn_poller = SVNPoller(
                   svnurl=source_code_svn_url,
                   pollinterval=60*60, # seconds
                   histmax=10,
                   svnbin='/usr/bin/svn',
 )
c['sources'] = [ svn_poller ]

####### SCHEDULERS

## configure the Schedulers

from buildbot.scheduler import Scheduler, Try_Jobdir
c['schedulers'] = []
c['schedulers'].append(Scheduler(name="all", branch=None,
                                 treeStableTimer=2*60,
                                 builderNames=["py3k-aix6-xlc", 
"py3k-aix5-xlc", "py3k-aix6-gcc", "py3k-aix5-gcc",]))
c['schedulers'].append(Try_Jobdir(name="try1",
                                  jobdir="jobdir",
                                  builderNames=["py3k-aix6-xlc", 
"py3k-aix5-xlc", "py3k-aix6-gcc", "py3k-aix5-gcc",]))


####### BUILDERS

# the 'builders' list defines the Builders. Each one is configured with a
# dictionary, using the following keys:
#  name (required): the name used to describe this builder
#  slavename (required): which slave to use (must appear in c['bots'])
#  builddir (required): which subdirectory to run the builder in
#  factory (required): a BuildFactory to define how the build is run
#  periodicBuildTime (optional): if set, force a build every N seconds

# buildbot/process/factory.py provides several BuildFactory classes you can
# start with, which implement build processes for common targets (GNU
# autoconf projects, CPAN perl modules, etc). The factory.BuildFactory is the
# base class, and is configured with a series of BuildSteps. When the build
# is run, the appropriate buildslave is told to execute each Step in turn.

# the first BuildStep is typically responsible for obtaining a copy of the
# sources. There are source-obtaining Steps in buildbot/steps/source.py for
# CVS, SVN, and others.

source_code_svn_url='http://svn.python.org/projects/python/branches/py3k'

from buildbot.process import factory
from buildbot.steps.source import SVN
from buildbot.steps.shell import Configure, Compile, Test, ShellCommand
from buildbot.steps.transfer import FileDownload
from buildbot.steps.python_twisted import Trial


def build_factory(branch, platform, compiler):
    f = factory.BuildFactory()
    f.addStep(SVN(mode='update', 
baseURL='http://svn.python.org/projects/python/', defaultBranch='branches/%s' % 
branch))
    
    f.addStep(FileDownload, mastersrc="scripts/bootstrap.sh", 
slavedest="bootstrap.sh")
    
    # f.addStep(FileDownload, mastersrc="scripts/cleanup.sh", 
slavedest="cleanup.sh")
    env = {"CC": compiler}
    # with buildbot 0.8: 'PYTHONHOME': None    

    f.addStep(ShellCommand(command="chmod u+x bootstrap.sh; ./bootstrap.sh",
               description=['bootstraping',], descriptionDone=['bootstrap',], 
env=env))
               
    configure_extra = ""
    if platform == "aix5":
        configure_extra += " --disable-ipv6"
    if compiler == "xlc":
        configure_extra += " --without-computed-gotos"

    f.addStep(Configure(command="unset PYTHONHOME; ./configure %s" % 
configure_extra, env=env))
    
    f.addStep(Compile(command="unset PYTHONHOME; ${MAKE} all", env=env))
    
    f.addStep(Test(command="unset PYTHONHOME; ${MAKE} test", env=env))
    
    f.addStep(ShellCommand(command="{MAKE} distclean",
               description=['cleaning',], descriptionDone=['cleanup',], 
alwaysRun=True, env=env))
    # chmod u+x cleanup.sh; ./cleanup.sh
    
    # f.addStep(shell.Test, command="python%s tests/runtests.py --noinput 
--verbosity=1 --settings=testproject.settings" % (version)) # env={'PYTHONPATH' 
: '.:/var/lib/buildbot'})
    return f


b1 = {'name': "py3k-aix6-xlc",
      'slavename': "phenix",
      'builddir': "py3k-aix6-xlc",
      'factory': build_factory('py3k', 'aix6', 'xlc'),
      }
b2 = {'name': "py3k-aix5-xlc",
      'slavename': "sirius",
      'builddir': "py3k-aix5-xlc",
      'factory': build_factory('py3k', 'aix5', 'xlc'),
      }
b3 = {'name': "py3k-aix6-gcc",
      'slavename': "phenix",
      'builddir': "py3k-aix6-gcc",
      'factory': build_factory('py3k', 'aix6', 'gcc'),
      }
b4 = {'name': "py3k-aix5-gcc",
      'slavename': "sirius",
      'builddir': "py3k-aix5-gcc",
      'factory': build_factory('py3k', 'aix5', 'gcc'),
      }
c['builders'] = [b1, b2, b3, b4]


####### STATUS TARGETS

# 'status' is a list of Status Targets. The results of each build will be
# pushed to these targets. buildbot/status/*.py has a variety to choose from,
# including web pages, email senders, and IRC bots.

c['status'] = []

# Use allowForce=True (boolean, not a string. ie: not 'True') to allow
# Forcing Builds in the Web User Interface. The default is False.
# from buildbot.status import html
# c['status'].append(html.WebStatus(http_port=8010,allowForce=True))

from buildbot.status import html
# from buildbot.status.web.authz import Authz
# authz = Authz(
#     forceBuild=True,
#     stopBuild=True)
c['status'].append(html.WebStatus(http_port=8010))

# from buildbot.status import mail
# c['status'].append(mail.MailNotifier(fromaddr="build...@localhost",
#                                      extraRecipients=["bui...@example.com"],
#                                      sendToInterestedUsers=False))
#
# from buildbot.status import words
# c['status'].append(words.IRC(host="irc.example.com", nick="bb",
#                              channels=["#example"]))
#
# from buildbot.status import client
# c['status'].append(client.PBListener(9988))


####### DEBUGGING OPTIONS

# if you set 'debugPassword', then you can connect to the buildmaster with
# the diagnostic tool in contrib/debugclient.py . From this tool, you can
# manually force builds and inject changes, which may be useful for testing
# your buildmaster without actually committing changes to your repository (or
# before you have a functioning 'sources' set up). The debug tool uses the
# same port number as the slaves do: 'slavePortnum'.

c['debugPassword'] = "debugpassword"

# if you set 'manhole', you can ssh into the buildmaster and get an
# interactive python shell, which may be useful for debugging buildbot
# internals. It is probably only useful for buildbot developers. You can also
# use an authorized_keys file, or plain telnet.
#from buildbot import manhole
#c['manhole'] = manhole.PasswordManhole("tcp:9999:interface=127.0.0.1",
#                                       "admin", "password")


####### PROJECT IDENTITY

# the 'projectName' string will be used to describe the project that this
# buildbot is working on. For example, it is used as the title of the
# waterfall HTML page. The 'projectURL' string will be used to provide a link
# from buildbot HTML pages to your project's home page.

c['projectName'] = "Python on AIX"
c['projectURL'] = "http://python.org/";

# the 'buildbotURL' string should point to the location where the buildbot's
# internal web server (usually the html.Waterfall page) is visible. This
# typically uses the port number set in the Waterfall 'status' entry, but
# with an externally-visible host name which the buildbot cannot figure out
# without some help.

c['buildbotURL'] = "http://rtsandbox:8010/";
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Reply via email to