It dawns on me that the problem may not be that the buildbot somehow retains a memory of the earlier location of a buildslave. It could instead be that the entire log that displays is actually from an earlier (and failed) build attempt.

Is there a way to have buildbot insert a datetime stamp into the log for a given step?



On 6/14/2016 3:01 PM, Greg Bullock wrote:
I'm using version 0.8.12.

I have indeed rebooted the master. I've also deleted the CentOS virtual machine (the slave) and created a new one from a fresh install of the system. No luck.

I have not tried both of these measures in the same test, I wouldn't think that could make the difference.

I will look into purging the sun cache, from the log, it seems that the buildslave is already reporting using the obsolete (and nonexistent) working directory before it calls svn.

Greg Bullock
NorthWest Research Associates
301 Webster St. <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
Monterey, CA 93940 <geo:0,0?q=301%20Webster%20St.%20Monterey%20CA%2093940>
(831) 582-4907 <tel:%28831%29%20582-4907>
[email protected]

-------- Original message --------
From: Pierre Tardy <[email protected]>
Date: 6/14/16 1:12 PM (GMT-08:00)
To: Greg Bullock <[email protected]>, [email protected]
Subject: Re: [[email protected]] buildbot retains memory of an earlier location of a buildslave?

Hi,
Looks indeed like an interresting problem. Did you try to reboot the master?

The master might indeed store the workdir, but I would expect it to be reset when the work is reconnecting.

Which version of buildbot is that?

Also thing to check is svn. could svn cache something?

Regards
Pierre

Le mar. 14 juin 2016 à 21:04, Greg Bullock <[email protected] <mailto:[email protected]>> a écrit :

    Where might the buildbot retain its memory of an earlier location of a
    buildslave?  I've moved a buildslave to a different drive, but the log
    from the new build tests shows the steps (e.g., the SVN, ShellCommand,
    and Compile steps) all running from the original drive, not the
    new drive.

    My buildbot runs on Windows 7 64-bit, and this particular buildslave
    (called "centos7-worker") runs on a virtual machine on CentOS 7
    (linux)
    hosted on the same Windows machine with the buildbot master.

    Originally, the buildslave's files were on a shared drive (which
    is also
    a Windows share shared from the Windows host), in
    /media/sf_G_DRIVE/buildbot/centos7-worker, but the SVN checkout step
    would successfully checkout only about half the project, then give
    an error

    ------------------------------------------------------------
       ... (previous lines omitted)
       svn: E200030: disk I/O error, executing statement 'RELEASE s10246'
       svn: E200030: sqlite: disk I/O error
       svn: E200030: sqlite: disk I/O error
       svn: E200030: no such savepoint: s10247, executing statement
    'RELEASE   s10247'
       svn: E200030: no such savepoint: s10247, executing statement
    'ROLLBACK TO s10247'
       program finished with exit code 1
    ------------------------------------------------------------

    After a Google search turned up several mentions of conflicts between
    Windows shared drive and sqlite (see, for example,
    
http://stackoverflow.com/questions/18782941/tortoisesvn-checkout-sqlite-disk-i-o-error-s10),
    I moved the buildslave files to a local drive within the virtual
    machine, with the path /home/buildbot/Documents/centos7-worker.

    At this point, I expect the log to have references only to the new
    /home/buildbot location, with no more references to the older
    /media/sf_G_DRIVE location.  But instead, the log still shows steps
running "in dir /media/sf_G_DRIVE/buildbot/centos7-worker/...". The top
    of the log from the SVN checkout step shows an example:

    ------------------------------------------------------------
       svn --version
        in dir
    /media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
    (timeout 1200 secs)
        watching logfiles {}
        argv: ['svn', '--version']
        environment:
    
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-ebvEqaI7D1,guid=b74561f6a34d14afe5191e515759b435
         DESKTOP_SESSION=gnome-classic
         DISPLAY=:0
         GDMSESSION=gnome-classic
         GDM_LANG=en_US.UTF-8
         GJS_DEBUG_OUTPUT=stderr
         GJS_DEBUG_TOPICS=JS ERROR;JS LOG
         GNOME_DESKTOP_SESSION_ID=this-is-deprecated
         GNOME_SHELL_SESSION_MODE=classic
         GPG_AGENT_INFO=/run/user/1000/keyring/gpg:0:1
         HISTCONTROL=ignoredups
         HISTSIZE=1000
         HOME=/home/buildbot
         HOSTNAME=localhost.localdomain
         IMSETTINGS_INTEGRATE_DESKTOP=yes
         IMSETTINGS_MODULE=none
         LANG=en_US.UTF-8
         LESSOPEN=||/usr/bin/lesspipe.sh %s
         LOGNAME=buildbot
    
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;
     
5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
         MAIL=/var/spool/mail/buildbot
         OLDPWD=/home/buildbot
    
PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/buildbot/.local/bin:/home/buildbot/bin
    PWD=/media/sf_G_DRIVE/buildbot/centos7-worker/gfortran_centos7_build/build
         QTDIR=/usr/lib64/qt-3.3
         QTINC=/usr/lib64/qt-3.3/include
         QTLIB=/usr/lib64/qt-3.3/lib
         QT_GRAPHICSSYSTEM_CHECKED=1
         QT_IM_MODULE=ibus
    
SESSION_MANAGER=local/unix:@/tmp/.ICE-unix/2119,unix/unix:/tmp/.ICE-unix/2119
         SHELL=/bin/bash
         SHLVL=2
         SSH_AGENT_PID=2306
         SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
         TERM=xterm-256color
         USER=buildbot
         USERNAME=buildbot
         VTE_VERSION=3803
         WINDOWID=44049837
         WINDOWPATH=1
         XAUTHORITY=/run/gdm/auth-for-buildbot-xH6kQ3/database
         XDG_CURRENT_DESKTOP=GNOME-Classic:GNOME
         XDG_MENU_PREFIX=gnome-
         XDG_RUNTIME_DIR=/run/user/1000
         XDG_SEAT=seat0
         XDG_SESSION_DESKTOP=gnome-classic
         XDG_SESSION_ID=1
         XDG_VTNR=1
         XMODIFIERS=@im=ibus
         _=/usr/bin/buildslave
        using PTY: False
       svn, version 1.7.14 (r1542130)
          compiled Nov 20 2015, 19:25:09
       (subsequent lines omitted)...
    ------------------------------------------------------------

    I've tried recreating the buildbot master and both of its buildslaves
    (deleting the master folder and the buildslaves folders, creating the
    buildbot and buildslaves, then restoring the buildbot.tacs and
    master.cfg from backup).  And I also unmounted the /media/sf_G_DRIVE
    altogether.  All of this seemed to purge the log as expected, but the
    next build attempt still makes reference to the old location of the
    CentOS buildslave in /media/sf_G_DRIVE drive.

    Where might the reference to this obsolete location of the
    buildslave be
    stored?  The centos7-worker's buildbot.tac file contains no explicit
    reference to the obsolete location:
    ------------------------------------------------------------

       import os

       from buildslave.bot import BuildSlave
       from twisted.application import service

       basedir = '.'
       rotateLength = 10000000
       maxRotatedFiles = 10

       # if this is a relocatable tac file, get the directory
    containing the TAC
       if basedir == '.':
           import os.path
           basedir = os.path.abspath(os.path.dirname(__file__))

       # note: this line is matched against to check that this is a
    buildslave
       # directory; do not edit it.
       application = service.Application('buildslave')

       try:
         from twisted.python.logfile import LogFile
         from twisted.python.log import ILogObserver, FileLogObserver
         logfile = LogFile.fromFullPath(os.path.join(basedir,
    "twistd.log"),
    rotateLength=rotateLength,
    maxRotatedFiles=maxRotatedFiles)
         application.setComponent(ILogObserver,
    FileLogObserver(logfile).emit)
       except ImportError:
         # probably not yet twisted 8.2.0 and beyond, can't set log yet
         pass

       buildmaster_host = 'kraken'
       port = 9989
       slavename = 'centos7-worker'
       passwd = 'pass'
       keepalive = 600
       usepty = 0
       umask = None
       maxdelay = 300
       allow_shutdown = None

       s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
                      keepalive, usepty, umask=umask, maxdelay=maxdelay,
                      allow_shutdown=allow_shutdown)
       s.setServiceParent(application)


    Likewise, the master.cfg file creates a single SVN checkout step as

       checkout = SVN(repourl  = 'svn://kraken/trunk/sf_code',
                      username = "buildbot",
                      password = "buildbot",
                      haltOnFailure = True,
                      )
    ------------------------------------------------------------

    and this step is used in all builders on both buildslaves (the
    "centos7-worker" and the "win64-worker"), so I don't see how it could
    convey the obsolete location to the buildslave.


    --
    Greg Bullock
    NorthWest Research Associates
    301 Webster St.
    Monterey, CA  93940
    (831) 582-4907
    [email protected] <mailto:[email protected]>


    _______________________________________________
    users mailing list
    [email protected] <mailto:[email protected]>
    https://lists.buildbot.net/mailman/listinfo/users



_______________________________________________
users mailing list
[email protected]
https://lists.buildbot.net/mailman/listinfo/users

--
Regards.
Greg Bullock
NorthWest Research Associates
301 Webster St.
Monterey CA 93940
(831) 582-4907
[email protected]

_______________________________________________
users mailing list
[email protected]
https://lists.buildbot.net/mailman/listinfo/users

Reply via email to