http:// tinyurl.com/ProTFS2012.

32
Software Testing with Microsoft Test Manager 2012 and Lab Management Brian Keller DEV-B327

Transcript of http:// tinyurl.com/ProTFS2012.

Page 1: http:// tinyurl.com/ProTFS2012.

Software Testing with Microsoft Test Manager 2012 and Lab ManagementBrian Keller

DEV-B327

Page 2: http:// tinyurl.com/ProTFS2012.

Who is this session for?• For manual testers who want to improve

their productivity and collaborate more effectively.

• For developers who want to start automating functional UI tests.

• For teams who want to improve software quality by producing better bugs and/or adopt agile testing.

• For teams who want to automate the provisioning of dev/test labs and build-deploy-test workflows.

Page 3: http:// tinyurl.com/ProTFS2012.

This session is not for….• …unit testing.• …fakes / mocking.• …web / load testing.

Page 4: http:// tinyurl.com/ProTFS2012.

ALM Virtual Machines:http://aka.ms/ALMVMs

Page 5: http:// tinyurl.com/ProTFS2012.

http://tinyurl.com/ALM2012 http://tinyurl.com/ProTFS2012

Page 6: http:// tinyurl.com/ProTFS2012.

USS Yorktown, SmartShip

• crew member entered 0 in a data entry field, caused a “divide by 0” error that shut down propulsion

• dead in the water for 2hrs 45mins

Page 7: http:// tinyurl.com/ProTFS2012.

Ariane 5 Flight 501

• re-used code from Ariane 4, but took a different flight path because of different booster design

• conversation from 64bit float to 16bit signed int caused overflow (exception handler was disabled for perf reasons)

• cost: > $370 million @ 1996 prices

Page 8: http:// tinyurl.com/ProTFS2012.

Vasa

• sank after sailing less than 1 nautical mile out of Stockholm

• specifications were faulty, failing to take into account enough ballast to keep the ship upright

• test teams revealed that it was unstable, but launch was rushed, and QA was ignored

Page 9: http:// tinyurl.com/ProTFS2012.
Page 10: http:// tinyurl.com/ProTFS2012.
Page 11: http:// tinyurl.com/ProTFS2012.
Page 12: http:// tinyurl.com/ProTFS2012.
Page 13: http:// tinyurl.com/ProTFS2012.

Training

Conditioning

Climbing

Training Phase

Cost of Failure

Page 14: http:// tinyurl.com/ProTFS2012.
Page 15: http:// tinyurl.com/ProTFS2012.

Development

Test

Release

Software Phase

Cost of Bugs

Page 16: http:// tinyurl.com/ProTFS2012.

Generalist Specialist

Manual Testing Some scripting

Creates scripts to set up lab, create data

Strong scripting skills

Some coding skills

Strong coding

Develops automated testing procedures

Expert coding skills

Tester Segmentation

Black Box Testing

White Box Testing

API Testing

70% of testing happens heremajority of test tools target here

Page 17: http:// tinyurl.com/ProTFS2012.

What usually happens when a testerfinds a bug?

Graphic by Jeff Atwood (CodingHorror.com)

Page 18: http:// tinyurl.com/ProTFS2012.

Unified ALM Solution

Team Foundation

Server

Planning SCM Work Item Tracking

Testing and Lab

Management

Build Automation

Reporting and

Analytics

Page 19: http:// tinyurl.com/ProTFS2012.

Microsoft Test Manager 2012

demo

Page 20: http:// tinyurl.com/ProTFS2012.

(1) Get source (2) Compile projects

(3) Copy build to running environment

(4) Run deployment scripts for each machine

(5) Create environment snapshot

(6) Executeautomated tests

(7) Send testresults

(8) Publish results toTeam Foundation Server

Build – Deploy – Test

Team Build

Team Foundation Server

Test Controller

VMM Agent Test Agent Test Agent

VM Host Web Server (VM)

Database Server (VM)

Virtual Environment

Page 21: http:// tinyurl.com/ProTFS2012.

VMM Agent Test Agent Test Agent

VM Host Web Server (VM)

Database Server (VM)

SCVMM ENVIRONMENT

Test Architecture

Test Agent Test Agent

Web Server Database Server

STANDARD ENVIRONMENT

Test Agent Test Agent

VM Template 1

VM Template 2

VM TEMPLATE LIBRARY

Team Foundation Server

System Center Virtual Machine Manager

(SCVMM)

Team Build

Test Controller

Page 22: http:// tinyurl.com/ProTFS2012.

SCVMM vs. Standard EnvironmentsSCVMM Environment Standard Environment

Build-Deploy-Test

Manual Testing

Snapshot / Restore

Share Bug Snapshots

Network Fencing

3rd Party Virtualization

Physical Machines

Page 23: http:// tinyurl.com/ProTFS2012.

Lab Management

demo

Page 24: http:// tinyurl.com/ProTFS2012.

Lord Kelvin, 1824-1907

“…you cannot manage what you cannot measure.”

Page 25: http:// tinyurl.com/ProTFS2012.
Page 26: http:// tinyurl.com/ProTFS2012.
Page 27: http:// tinyurl.com/ProTFS2012.
Page 28: http:// tinyurl.com/ProTFS2012.
Page 29: http:// tinyurl.com/ProTFS2012.
Page 30: http:// tinyurl.com/ProTFS2012.
Page 31: http:// tinyurl.com/ProTFS2012.

Team Foundation Service details

Included for certain paid MSDN subscribers:

Free Plan for up to 5 users

Additional information at http://tfs.visualstudio.com

* Capability in preview – limits may apply. Authoring load tests requires Visual Studio Ultimate 2013 Preview.

Version control (TFVC or Git) Comment on changesets & commits Work item tracking and tagging Team rooms Agile planning tools Feedback Management Agile Portfolio Management* Build* Web-based test case management* Load testing*

New!

New!

New!

New!

Page 32: http:// tinyurl.com/ProTFS2012.

© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.