Ok. So if you want to do QA testing — “manual testing” in Microsoftspeak — with Team Foundation Server (TFS) or Visual Studio Team Services (VSTS), you’re going to spend a lot of time in the Test Hub. The vast majority of QA/manual testing features have moved to the web interface as of TFS2015 and the Test Hub is the section of the TFS web interface that has all those features.
Going into the Test Hub for the first time can be a little bit daunting because there are a lot of terms that might not make a whole lot of sense. Plus, you’ve probably just been greeted by a message that says “You can’t create test cases with a test plan.” Uhhh…what’s a test case? What’s a test plan? What’s the difference between a test case and a test plan? Those sound pretty much the same, right? My goal with this post is to clear some of that stuff up.
Test Plans
A TFS Test Plan is a container for a testing effort. When you create one, you’ll be asked to choose an Area Path and an Iteration.
Area Path is an idea that’s scattered around lots of places in TFS that allows you to keep things relevant to specific parts of your Team Project. If you leave the Area Path blank (default value), it means that this Test Plan is available for the whole Team Project. If your Team Project is relatively small and/or already pretty concentrated, this is probably your best choice. Other choices in Area Path get to some combination of Team and/or functional area.
Iteration Path is another idea that’s scattered around TFS and it helps you to scope things to certain time periods and/or to certain releases or Sprints. If you’re using Scrum, this Iteration Path thing makes a LOT of sense because it helps you to connect your testing effort to the work that you’re doing in the current sprint. In fact, I think that thinking of your Test Plan as always being attached to your current iteration is the best way to do this and I always name my Test Plan so that it includes the name of the sprint.
If you’re in the Test Hub and it shows you the “You can’t create test cases without a test plan” message then the easiest way to create a new test plan is just to click the “Click here to create” button.
Test Suites
Once you’ve created your new Test Plan, you have two options: 1) start creating Test Cases or 2) create one or more Test Suites. A Test Case describes the steps that a QA/manual tester should run (and the expected results) when testing a functional path through the application. The Test Hub is perfectly happy to have you create all your Test Cases directly off of your Test Plan…but this’ll get really messy really fast and you’ll start having trouble tracking what’s going on because your test are just going to be in a big pile. What you’ll want is some kind of organization and that’s where Test Suites come in. There are three types of Test Suite: Requirement-based, Query-based, and Static.
Requirement-based Test Suites
A Requirement-based Test Suite organizes your Test Cases by Product Backlog Item or User Story. (Nerdy fact: under the hood of TFS, there’s a thing called a Requirement Category and it defines which work item type(s) counts as a “requirement” in your Team Project. Hence, “Requirement-based” Test Suite rather than PBI-based or User Story-based.) If you’re doing Scrum, you’ll probably have a Requirement-based Test Suite for each of the Product Backlog Items (PBIs) that your team is taking on in this Sprint.
So, in Scrum, why have a Test Suite separating the Test Plan from the Test Case for a PBI? Well, chance are very high that you’re going to have more than one Test Case that’s related to a PBI. For example, if you’ve got a PBI called “Order a Pizza” and you can order a pizza via a browser or via a mobile app. That’ll probably turn into two test cases “Order pizza via the web” and “Order pizza via mobile app”. There might be test cases related to payment type, too. “Order pizza via mobile app and pay with Apple Pay” and “Order pizza on the web and pay with PayPal”. These Test Cases can multiply like tribbles and you can get overwhelmed if you don’t plan ahead so definitely create a Requirement-based Test Suite so you’re prepared.
Query-based Test Suites
A Query-based Test Suite is driven by to a TFS Work Item Query. A Work Item Query is a filter over the work items in your Team Project Collection and helps you to locate Test Cases based on the data value of that Test Case and/or the data values of work items that are linked to a Test Case. What’s super helpful is that this query doesn’t have to be constrained to just the current Team Project — the query can reference any Work Item that is stored in the current Team Project Collection. Why’s that helpful? Well, because that let’s you run test cases that might not be directly related to your development effort. For example, maybe you’re working on a set of features that some other set of features depends on. Something like the “Shipping” system depends on the code you’re creating in the “Order Management” system. It might be helpful to run a few basic tests from the Shipping system’s test plans in order to validate that your Order Management changes haven’t broken their code.
Static Test Suites
Static Suites are underloved and misunderstood. They’re the basically just folders for your test cases. Kinda boring and useless, right? Wrong. These are fantastic for all the miscellaneous testing you need to do — especially for managing stuff like regression test suites for work that’s not related to new development work in this sprint. A Static Suite groups Test Cases the way that you want to group them rather than by any other TFS-governed method. Why a regression test suite? Well, when you’re developing new functionality in your Sprint/iteration, you’ll probably have a bunch of functionality inside of the app that always need to be checked. This is kind of like that case that I described for using Query-based Test Suites where you might grab test cases from another team using a work item query. Well, in this case, the test cases are more random or arbitrary or unrelated or just plain harder to identify using a work item query — so you’ll just add them in to the Static Suite manually and that’ll be it.
Summary
Well, that’s a quick tour of Test Plans, Test Suites, and Test Cases in the Team Foundation Server Test Hub. I hope that cleared things up.
-Ben
— Need some help doing QA Testing with TFS? Not sure how to get your Test Cases into TFS? Want some help getting QA Testing working with Scrum & Agile? We can help. Drop us a line at info@benday.com.
Leave a Reply