Search This Blog

Friday, 16 September 2011

Running SOAP UI Tests in Teamcity using MSBuild

One of the web services we have has a bunch of SOAP UI Tests. I wanted to make sure when this was run on the build server we have good feedback for each test case rather than a whole build running a pack of tests and telling you if it failed or passed. Kind of a reason i was moving this build away to Teamcity from cruise control

The MSBuild script is as below. I had to define the variables I need and the Directory where all project files for Soap UI can be found.

   1: <Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
   2: <PropertyGroup>
   3:     <SoapUIDir>$(MSBuildProjectDirectory)\SoapUI</SoapUIDir>
   4:     <SoapTestTool>C:\Program Files\eviware\soapUI-4.0.0\bin\testrunner.bat</SoapTestTool>
   5: </PropertyGroup>

The next thing I had to do was run each project file using the SOAP Ui testrunner.bat file.

The options I have used –j will ensure JUnit style reports are pushed out of the Soap UI test runner, for local builds you could just make it push out Html reports

-h allows you to specify the host header to use for your Urls, The host header you supply here will override what is stored in the SOAP UI projects , so you run these tests on different sites based on your environment.

I have done a FileUpdate because i wanted to change Urls of the service from the old server to the ones on the new servers, you wont need this and it can be ignored

   1:  
   2: <Target Name="ConfigureSoapUITests">
   3:    <MakeDir Directories="$(SoapUIDir)\Report"/>    
   4:    <ItemGroup>
   5:      <SoapUIProjectFiles Include="$(SoapUIDir)\*-soapui-project.xml"/>      
   6:    </ItemGroup>
   7:    <!-- Replace old server urls with new ones -->
   8:    <FileUpdate Files="@(SoapUIProjectFiles)" Regex="$(someurl)" ReplacementText="$(newurl)" IgnoreCase="true"/>
   9: </Target>
  10:  
  11: <Target Name="RunFunctionalSoapUITests" DependsOnTargets="ConfigureSoapUITests">
  12:    <!--Run all the soap ui functional tests-->
  13:    <Exec Command="&quot;$(SoapTestTool)&quot; &quot;%(SoapUIProjectFiles.Identity)&quot; -h &quot;$(HostHeader)&quot; -I -r -a -j -f &quot;$(SoapUIDir)\Report&quot;" />
  14: </Target>

At this point I hooked up the MSBuild target to run on Teamcity using the MSBuild runner, for the target “RunFunctionalSoapUITests”. The feedback wasn't good

The final bit you need to do is configure Teamcity to read the xml styled junit reports that the test runner is spitting out. You can do this using the Build feature option for XmlReport processing as shown in the screen shot.

Now trigger the build if you have sorted out all the other variables required you can run the build and see the output on Teamcity, pretty good really on test case by test case basis.

buildfeature

For failed tests I added a configuration for artifacts as follows, Teamcity will show you the response but to see the entire request response file , you need to setup artifacts as follows. All failed test cases create a .txt file which ends with FAILED.txt, so I push these as artifacts and i can see this for failed tests,

SoapUI/Report/*FAILED.txt => FailedTests

I find this useful that with little effort so much could be done and feedback is really good

5 comments:

Dhiraj Kumar said...

Hi,

I wanted to implement this process. Please guide me . Please provide me your contact details so i can contact you for more details upon Teamcity and SOAPUI integration process.

Query :- Need put the SOAPUI project on a server (Teamcity). Once on the server, Need to know how to call the ‘job/SOAPUI’ test run and figure out how to distribute the results from the run. We want to see this update ALM.

Dave Creer said...

Hi all,

I am trying to integrate SoapUI into TeamCity using this guide.

I have a working MSBuild file, but when running the build in TeamCity it hangs, and the thread dump for Java returns the a message stating that it is waiting on a condition.

"VM Periodic Task Thread" prio=10 tid=0x0000000006f3e800 nid=0x2e28 waiting on condition"

When running the MSBuild file outside of teamcity there are no errors that would seem to be problematic. There are some permissions errors for the logs, but that does not stop the execution or fail any of the tests and if I add a failed test the 'FAIL' Log file is generated perfectly.

Many thanks in advance for any help you can give and for a great guide!

Алексей Крекотнев said...

Hi!
Thanks for your post!
What informs the string "$(SoapTestTool)" and the string "%(SoapUIProjectFiles.Identity)?

Thanks for your answer!

Anonymous said...

The topic of this blog is very popular as more .NET companies are wanting to automate API testing using MSBuild. Now several years after your post could you follow up with a more in depth guide including some of the obstacles you came across when integrating SOAP UI / Ready API with Team City using MSBuild? Thank you!

Anonymous said...

I am also very interested in running Ready API from TeamCity.

Is there any way to get an updated and/or complete walkthrough on how to set up the CI process using GIT/Ready API/TeamCity?