On Thu, 21 Nov 2013, Kohsuke Kawaguchi wrote: > While looking at > https://wiki.jenkins-ci.org/display/JENKINS/LTS+1.532.x+RC+Testing and > tinkering with our acceptance tests (aka selenium-tests), I noticed that > more tests can be automated if we have means to set up test fixtures more > easily. > > For example, one of the tests listed there is the scp plugin, and if we can > just run an ssh server that accepts a known user name & password, we can > turn this into just another cucumber test! > > So I put together a PoC in > https://github.com/jenkinsci/selenium-tests/tree/docker > > You can run this with "bundle exec cucumber features/docker_test.feature" > > The idea is to accumulate test fixtures into containers and put them under > directories under "fixtures", then allow a test case to run them.
This is a wonderful idea Kohsuke, I'm glad you're still in pursuit of good
end-to-end tests for Jenkins :)
That said, I wasn't able to get this working locally, I'm not sure if it's due
to my version of Chromium, Ruby, or what:
-> % cuke features/docker_test.feature
WARNING: Nokogiri was built against LibXML version 2.8.0, but has
dynamically loaded 2.9.0
Feature: Fixtures via docker
To test Jenkins features that require non-trivial fixtures,
this test uses Docker to launch a fixture without affecting the host
environment.
@docker
Scenario: Run an SSH server # features/docker_test.feature:6
Bringing up a temporary Jenkins instance...... Jenkins completed
initialization
Unknown command 'WaitForAllTabsToStopLoading'. Options:
AcceptOrDismissAppModalDialog, ActionOnSSLBlockingPage, ActivateTab,
AddBookmark, AddDomEventObserver, AppendTab, ApplyAccelerator,
BringBrowserToFront, ClearEventQueue, CloseBrowserWindow, CloseTab,
CreateNewAutomationProvider, DeleteCookie, DeleteCookieInBrowserContext,
DoesAutomationObjectExist, DragAndDropFilePaths, ExecuteJavascript,
ExecuteJavascriptInRenderView, GetActiveTabIndex, GetAppModalDialogMessage,
GetBookmarkBarStatus, GetBookmarksAsJSON, GetBrowserInfo,
GetBrowserWindowCount, GetChromeDriverAutomationVersion, GetCookies,
GetCookiesInBrowserContext, GetDownloadDirectory, GetExtensionsInfo,
GetIndicesFromTab, GetLocalStatePrefsInfo, GetMultiProfileInfo, GetNextEvent,
GetPrefsInfo, GetProcessInfo, GetSecurityState, GetTabCount, GetTabIds,
GetTabInfo, GetViews, GoBack, GoForward, InstallExtension,
IsDownloadShelfVisible, IsFindInPageVisible, IsMenuCommandEnabled,
IsPageActionVisible, IsTabIdValid, MaximizeView, NavigateToURL, OpenFindInPage,
OpenNewBrowserWindow, OpenNewBrowserWindowWithNewProfile, OpenProfileWindow,
OverrideGeoposition, RefreshPolicies, Reload, RemoveBookmark,
RemoveEventObserver, ReparentBookmark, RunCommand, SendWebkitKeyEvent,
SetBookmarkTitle, SetBookmarkURL, SetCookie, SetCookieInBrowserContext,
SetDownloadShelfVisible, SetExtensionStateById, SetLocalStatePrefs, SetPrefs,
SetViewBounds, SimulateAsanMemoryBug, TriggerBrowserActionById,
TriggerPageActionById, UninstallExtensionById, UpdateExtensionsNow,
WaitForBookmarkModelToLoad, WaitUntilNavigationCompletes,
WebkitMouseButtonDown, WebkitMouseButtonUp, WebkitMouseClick,
WebkitMouseDoubleClick, WebkitMouseDrag, WebkitMouseMove,
AcceptCurrentFullscreenOrMouseLockRequest, AddOrEditSearchEngine,
AddSavedPassword, CloseNotification, DenyCurrentFullscreenOrMouseLockRequest,
DisablePlugin, EnablePlugin, FindInPage, GetAllNotifications, GetDownloadsInfo,
GetFPS, GetHistoryInfo, GetInitialLoadTimes, GetNTPInfo, GetNavigationInfo,
GetOmniboxInfo, GetPluginsInfo, GetSavedPasswords, GetSearchEngineInfo,
GetV8HeapStats, IsFullscreenBubbleDisplayed,
IsFullscreenBubbleDisplayingButtons, IsFullscreenForBrowser,
IsFullscreenForTab, IsFullscreenPermissionRequested,
IsMouseLockPermissionRequested, IsMouseLocked, KillRendererProcess, LaunchApp,
LoadSearchEngineInfo, OmniboxAcceptInput, OmniboxMovePopupSelection,
PerformActionOnDownload, PerformActionOnInfobar, PerformActionOnSearchEngine,
RemoveNTPMostVisitedThumbnail, RemoveSavedPassword,
RestoreAllNTPMostVisitedThumbnails, SaveTabContents, SetAppLaunchType,
SetOmniboxText, SetWindowDimensions, WaitForAllDownloadsToComplete,
WaitForNotificationCount, (Selenium::WebDriver::Error::UnknownError)
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/response.rb:51:in
`assert_ok'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/response.rb:15:in
`initialize'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:59:in
`new'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:59:in
`create_response'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/default.rb:66:in
`request'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:40:in
`call'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:634:in
`raw_execute'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:612:in
`execute'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:110:in
`get'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/common/navigation.rb:14:in
`to'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/selenium/driver.rb:43:in
`visit'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/session.rb:193:in
`visit'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/dsl.rb:51:in
`block (2 levels) in <module:DSL>'
/home/tyler/source/github/jenkins/selenium-tests/lib/pageobject.rb:87:in
`jenkins_version'
/home/tyler/source/github/jenkins/selenium-tests/lib/controller/jenkins_controller.rb:63:in
`start'
/home/tyler/source/github/jenkins/selenium-tests/features/support/hooks.rb:25:in
`Before'
Given a docker fixture "sshd" #
features/step_definitions/docker_steps.rb:1
Then I can login via ssh #
features/step_definitions/docker_steps.rb:9
It looks like the test failed/errored, so here's the console from Jenkins:
--------------------------------------------------------------------------
Running from: /home/tyler/source/github/jenkins/selenium-tests/jenkins.war
webroot: System.getProperty("JENKINS_HOME")
Nov 22, 2013 1:38:00 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Jenkins home directory:
/home/tyler/source/github/jenkins/selenium-tests/temp_dir_19933_6668_imswz
found at: System.getProperty("JENKINS_HOME")
Nov 22, 2013 1:38:03 PM winstone.Logger logInternal
INFO: HTTP Listener started: port=57404
Nov 22, 2013 1:38:03 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v0.9.10 running: controlPort=51690
Nov 22, 2013 1:38:04 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Nov 22, 2013 1:38:05 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Nov 22, 2013 1:38:05 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Nov 22, 2013 1:38:05 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Nov 22, 2013 1:38:05 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Nov 22, 2013 1:38:05 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Nov 22, 2013 1:38:10 PM
org.apache.sshd.common.util.SecurityUtils$BouncyCastleRegistration run
INFO: Trying to register BouncyCastle as a JCE provider
Nov 22, 2013 1:38:10 PM
org.apache.sshd.common.util.SecurityUtils$BouncyCastleRegistration run
INFO: Registration succeeded
Nov 22, 2013 1:38:10 PM org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 45707
Nov 22, 2013 1:38:10 PM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Nov 22, 2013 1:38:10 PM hudson.TcpSlaveAgentListener <init>
INFO: JNLP slave agent listener started on TCP port 46429
Nov 22, 2013 1:38:11 PM hudson.WebAppMain$2 run
INFO: Jenkins is fully up and running
Unknown command 'WaitForAllTabsToStopLoading'. Options:
AcceptOrDismissAppModalDialog, ActionOnSSLBlockingPage, ActivateTab,
AddBookmark, AddDomEventObserver, AppendTab, ApplyAccelerator,
BringBrowserToFront, ClearEventQueue, CloseBrowserWindow, CloseTab,
CreateNewAutomationProvider, DeleteCookie, DeleteCookieInBrowserContext,
DoesAutomationObjectExist, DragAndDropFilePaths, ExecuteJavascript,
ExecuteJavascriptInRenderView, GetActiveTabIndex, GetAppModalDialogMessage,
GetBookmarkBarStatus, GetBookmarksAsJSON, GetBrowserInfo,
GetBrowserWindowCount, GetChromeDriverAutomationVersion, GetCookies,
GetCookiesInBrowserContext, GetDownloadDirectory, GetExtensionsInfo,
GetIndicesFromTab, GetLocalStatePrefsInfo, GetMultiProfileInfo, GetNextEvent,
GetPrefsInfo, GetProcessInfo, GetSecurityState, GetTabCount, GetTabIds,
GetTabInfo, GetViews, GoBack, GoForward, InstallExtension,
IsDownloadShelfVisible, IsFindInPageVisible, IsMenuCommandEnabled,
IsPageActionVisible, IsTabIdValid, MaximizeView, NavigateToURL, OpenFindInPage,
OpenNewBrowserWindow, OpenNewBrowserWindowWithNewProfile, OpenProfileWindow,
OverrideGeoposition, RefreshPolicies, Reload, RemoveBookmark,
RemoveEventObserver, ReparentBookmark, RunCommand, SendWebkitKeyEvent,
SetBookmarkTitle, SetBookmarkURL, SetCookie, SetCookieInBrowserContext,
SetDownloadShelfVisible, SetExtensionStateById, SetLocalStatePrefs, SetPrefs,
SetViewBounds, SimulateAsanMemoryBug, TriggerBrowserActionById,
TriggerPageActionById, UninstallExtensionById, UpdateExtensionsNow,
WaitForBookmarkModelToLoad, WaitUntilNavigationCompletes,
WebkitMouseButtonDown, WebkitMouseButtonUp, WebkitMouseClick,
WebkitMouseDoubleClick, WebkitMouseDrag, WebkitMouseMove,
AcceptCurrentFullscreenOrMouseLockRequest, AddOrEditSearchEngine,
AddSavedPassword, CloseNotification, DenyCurrentFullscreenOrMouseLockRequest,
DisablePlugin, EnablePlugin, FindInPage, GetAllNotifications, GetDownloadsInfo,
GetFPS, GetHistoryInfo, GetInitialLoadTimes, GetNTPInfo, GetNavigationInfo,
GetOmniboxInfo, GetPluginsInfo, GetSavedPasswords, GetSearchEngineInfo,
GetV8HeapStats, IsFullscreenBubbleDisplayed,
IsFullscreenBubbleDisplayingButtons, IsFullscreenForBrowser,
IsFullscreenForTab, IsFullscreenPermissionRequested,
IsMouseLockPermissionRequested, IsMouseLocked, KillRendererProcess, LaunchApp,
LoadSearchEngineInfo, OmniboxAcceptInput, OmniboxMovePopupSelection,
PerformActionOnDownload, PerformActionOnInfobar, PerformActionOnSearchEngine,
RemoveNTPMostVisitedThumbnail, RemoveSavedPassword,
RestoreAllNTPMostVisitedThumbnails, SaveTabContents, SetAppLaunchType,
SetOmniboxText, SetWindowDimensions, WaitForAllDownloadsToComplete,
WaitForNotificationCount, (Selenium::WebDriver::Error::UnknownError)
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/response.rb:51:in
`assert_ok'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/response.rb:15:in
`initialize'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:59:in
`new'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:59:in
`create_response'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/default.rb:66:in
`request'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/http/common.rb:40:in
`call'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:634:in
`raw_execute'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:612:in
`execute'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/remote/bridge.rb:110:in
`get'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/selenium-webdriver-2.37.0/lib/selenium/webdriver/common/navigation.rb:14:in
`to'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/selenium/driver.rb:90:in
`reset!'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/session.rb:77:in
`reset!'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara.rb:245:in
`block in reset_sessions!'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara.rb:245:in
`each'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara.rb:245:in
`reset_sessions!'
/home/tyler/.rvm/gems/ruby-1.9.3-p385@jenkins-selenium-tests/gems/capybara-2.1.0/lib/capybara/cucumber.rb:10:in
`After'
Failing Scenarios:
cucumber features/docker_test.feature:6 # Scenario: Run an SSH server
1 scenario (1 failed)
2 steps (2 skipped)
0m18.810s
Just a data point for you :)
- R. Tyler Croy
--------------------------------------
Code: https://github.com/rtyler
Chatter: https://twitter.com/agentdero
[email protected]
signature.asc
Description: Digital signature
