Post on 27-Dec-2015
Software Testing with Microsoft Test Manager 2012 and Lab ManagementBrian Keller
DEV-B327
Who is this session for?• For manual testers who want to improve
their productivity and collaborate more effectively.
• For developers who want to start automating functional UI tests.
• For teams who want to improve software quality by producing better bugs and/or adopt agile testing.
• For teams who want to automate the provisioning of dev/test labs and build-deploy-test workflows.
This session is not for….• …unit testing.• …fakes / mocking.• …web / load testing.
http://tinyurl.com/ALM2012 http://tinyurl.com/ProTFS2012
USS Yorktown, SmartShip
• crew member entered 0 in a data entry field, caused a “divide by 0” error that shut down propulsion
• dead in the water for 2hrs 45mins
Ariane 5 Flight 501
• re-used code from Ariane 4, but took a different flight path because of different booster design
• conversation from 64bit float to 16bit signed int caused overflow (exception handler was disabled for perf reasons)
• cost: > $370 million @ 1996 prices
Vasa
• sank after sailing less than 1 nautical mile out of Stockholm
• specifications were faulty, failing to take into account enough ballast to keep the ship upright
• test teams revealed that it was unstable, but launch was rushed, and QA was ignored
Training
Conditioning
Climbing
Training Phase
Cost of Failure
Development
Test
Release
Software Phase
Cost of Bugs
Generalist Specialist
Manual Testing Some scripting
Creates scripts to set up lab, create data
Strong scripting skills
Some coding skills
Strong coding
Develops automated testing procedures
Expert coding skills
Tester Segmentation
Black Box Testing
White Box Testing
API Testing
70% of testing happens heremajority of test tools target here
What usually happens when a testerfinds a bug?
Graphic by Jeff Atwood (CodingHorror.com)
Unified ALM Solution
Team Foundation
Server
Planning SCM Work Item Tracking
Testing and Lab
Management
Build Automation
Reporting and
Analytics
Microsoft Test Manager 2012
demo
(1) Get source (2) Compile projects
(3) Copy build to running environment
(4) Run deployment scripts for each machine
(5) Create environment snapshot
(6) Executeautomated tests
(7) Send testresults
(8) Publish results toTeam Foundation Server
Build – Deploy – Test
Team Build
Team Foundation Server
Test Controller
VMM Agent Test Agent Test Agent
VM Host Web Server (VM)
Database Server (VM)
Virtual Environment
VMM Agent Test Agent Test Agent
VM Host Web Server (VM)
Database Server (VM)
SCVMM ENVIRONMENT
Test Architecture
Test Agent Test Agent
Web Server Database Server
STANDARD ENVIRONMENT
Test Agent Test Agent
VM Template 1
VM Template 2
VM TEMPLATE LIBRARY
Team Foundation Server
System Center Virtual Machine Manager
(SCVMM)
Team Build
Test Controller
SCVMM vs. Standard EnvironmentsSCVMM Environment Standard Environment
Build-Deploy-Test
Manual Testing
Snapshot / Restore
Share Bug Snapshots
Network Fencing
3rd Party Virtualization
Physical Machines
Lab Management
demo
Lord Kelvin, 1824-1907
“…you cannot manage what you cannot measure.”
Team Foundation Service details
Included for certain paid MSDN subscribers:
Free Plan for up to 5 users
Additional information at http://tfs.visualstudio.com
* Capability in preview – limits may apply. Authoring load tests requires Visual Studio Ultimate 2013 Preview.
Version control (TFVC or Git) Comment on changesets & commits Work item tracking and tagging Team rooms Agile planning tools Feedback Management Agile Portfolio Management* Build* Web-based test case management* Load testing*
New!
New!
New!
New!
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.