Re: Jenkins running unit tests MSTest 10.0

[jerome]

Many thing can cause this

   - unreliable tests, where concurrency can occur or uninitialized values
   - Windows permission
   - invalid path inside the machine
   - the user under which the salve run the tests (service under windows 
   don't have access to the GUI part and you need to run the salve under a 
   normal user account, OpenGL, DirectX, Canvas, Qt, etc)
   - Make sure the number you get is valid with MsBuild command line and 
   not Visual Studio (they aren't totally the same when it come to project 
   behavior).
   - Make sure you use the right visual studio/MsBuild version along the 
   right architecture (x86 vs x64).
   - Try to run your tests multiple times into random order if the test 
   suite allow it (help catch some problems).
   - Make sure you development env and jenkins slave have the Fault 
   Tolerent Heap disabled (world worst idea ever was born on the day that damn 
   thing was created)
   https://msdn.microsoft.com/en-us/library/dd744764(VS.85).aspx

A failed tests even if not always happening (once in a blue moon also 
apply) is always a bug somewhere.

On Tuesday, July 18, 2017 at 9:05:34 AM UTC-4, Simon Whale wrote:
>
> Hi, 
>
> I am wondering if someone can help.  I have setup Jenkins to run Unit 
> Tests.  Our application currently has 289 tests.   
>
> If I run it through Jenkins only 89 pass.  But If I run it through VS 2010 
> 247 pass. 
>
> Is there something that I should be doing in Jenkins? 
>
> This is how I have configured the Execute Windows batch command in the 
> build settings for the project. 
>
> "C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe" 
> /resultsfile:"%WORKSPACE%\Brian\Results.trx" /testcontainer:"C:\Program 
> Files\Jenkins\workspace\Brian - 
> Freestyle\Brian\trunk\BrianTestProject\bin\x86\Debug\BrianTestProject.dll" 
> /nologo
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0ea27d79-eef9-4cff-9277-f8ec9d9e1c45%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Jenkins running unit tests MSTest 10.0

[Mark Waite]

If your Jenkins agent is running as a service, then that may cause
different behaviors if your tests depend on interacting with a user
interface (like Selenium tests, or web browser driven tests).

If your Jenkins agent is running as a different user, and your tests depend
on a specific user, that could cause different behavior in Jenkins than in
the user environment.

If your Jenkins agent is running on a different Windows version and your
tests have Windows version requirements, that could cause failures.

Mark Waite

On Tue, Jul 18, 2017 at 5:05 AM Simon Whale  wrote:

> Hi,
>
> I am wondering if someone can help.  I have setup Jenkins to run Unit
> Tests.  Our application currently has 289 tests.
>
> If I run it through Jenkins only 89 pass.  But If I run it through VS 2010
> 247 pass.
>
> Is there something that I should be doing in Jenkins?
>
> This is how I have configured the Execute Windows batch command in the
> build settings for the project.
>
> "C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe"
> /resultsfile:"%WORKSPACE%\Brian\Results.trx" /testcontainer:"C:\Program
> Files\Jenkins\workspace\Brian -
> Freestyle\Brian\trunk\BrianTestProject\bin\x86\Debug\BrianTestProject.dll"
> /nologo
>
> --
> You received this message because you are subscribed to the Google Groups
> "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jenkinsci-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jenkinsci-users/c5fc26e9-a2d7-48d2-a8e3-b561e2524547%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtG0cUfRgOO%2BBOYN2XZ9z6AMtWQdtp_cbi2Bi-KJk5h%3D0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Jenkins running unit tests MSTest 10.0

[Simon Whale]

Hi, 

I am wondering if someone can help.  I have setup Jenkins to run Unit Tests. 
 Our application currently has 289 tests.   

If I run it through Jenkins only 89 pass.  But If I run it through VS 2010 
247 pass. 

Is there something that I should be doing in Jenkins? 

This is how I have configured the Execute Windows batch command in the 
build settings for the project. 

"C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe" 
/resultsfile:"%WORKSPACE%\Brian\Results.trx" /testcontainer:"C:\Program 
Files\Jenkins\workspace\Brian - 
Freestyle\Brian\trunk\BrianTestProject\bin\x86\Debug\BrianTestProject.dll" 
/nologo

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/c5fc26e9-a2d7-48d2-a8e3-b561e2524547%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.