WebApplicationFactory<T> is one of those awesome features in ASP.NET Core that practically nobody knows about. WebApplicationFactory<T> lets you quickly and easily run integration tests against your ASP.NET Core MVC or Web API application without having to deploy the app to an actual server like Kestrel or IIS. Instead of running in a “real” server, your app gets run in a test server and that lets you do some handy testing tricks that makes testing your app easier.
(TL;DR — here’s the source code on GitHub)
Testing Tricks with WebApplicationFactory<T>
What do I mean by “handy testing tricks”? The handiest of handy testing tricks is simply just turning off security during your tests. You’d never do this in production but when you’re writing automated integration tests against your ASP.NET Core apps, you usually just want to focus on feature functionality in your application. So if you have to worry about coding all the “log into the app” test code in addition to testing the feature functionality — well, if you’re like most devs, you’ll say “that feels like a whole lot of work” and then not write the test. With integration tests that are written with WebApplicationFactory<T>, you can swap our your production security with some fake security and be on your way to Productivity Nirvana with just a few lines of code.
Here’s the basic idea. If you’re using dependency injection to configure your dependencies in Startup.cs for production, what WebApplicationFactory<T> does is it lets you inject some custom code into Startup.cs for the sake of your tests. Want to turn off security? Sure. Change your database connection strings? No problem. Replace some type registrations with mock/fake type registrations that return canned answers? Simple.
Selenium & WebApplicationFactory<T>
What about running Selenium tests using WebApplicationFactory<T>? Uhhhhh…that’s a little bit of a problem. You’ll almost definitely get an error that says something like ERR_CONNECTION_REFUSED.
OpenQA.Selenium.WebDriverException: unknown error: net::ERR_CONNECTION_REFUSED (Session info: headless MicrosoftEdge=91.0.864.70) Stack Trace: at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse) at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters) at OpenQA.Selenium.Remote.RemoteWebDriver.set_Url(String value) at OpenQA.Selenium.Remote.RemoteNavigator.GoToUrl(String url)
I worked for HOURS and HOURS on this error and didn’t make a whole lot of progress. But then I found a couple of blog posts that started to clear things up for me:
- “Real Browser Integration Testing with Selenium Standalone, Chrome, and ASP.NET Core 2.1” by Scott Hanselman
- “Quick fix for integration testing with Selenium in ASP.NET Core 3.1” by Bertrand Thomas
What’s Going Wrong?
The core of the problem is related to how integration tests work with WebApplicationFactory<T>. If you’re running a simple, non-selenium integration test, you call WebApplicationFactory<T>.GetClient() and it gives you an instance of HttpClient to work with. You use it and everything’s happy. But then you try to access your app at the same URL but instead using a different instance of HttpClient or a browser or a Selenium test, and then you get ERR_CONNECTION_REFUSED.
WHY!?!?!?!?!?!?!?!
Well, under the surface, that call to WebApplicationFactory<T>.GetClient() is doing some magic tricks. It’s running your app in memory and it’s basically behaving like an HTTP application…but it’s not actually exposing the application to your network. So when you come along with your Selenium tests and try to hit that same URL, it’s not really actually there and listening for HTTP requests and therefore ERR_CONNECTION_REFUSED.
How to Fix It?
This is where Bertrand Thomas’s blog post helped out immensely. He figured out that you needed to make a separate call to spin up an instance of TestServer in order to make your application respond to HTTP requests coming over the network. (High five, Bertrand! 🙌)
Bertrand’s solution is to use a custom version of WebApplicationFactory<T> that adds the missing initialization calls.
Where Stuff Starts Getting Weird
Like I mentioned earlier in this article, when I do integration tests with WebApplicationFactory<T>, I want to do type replacements during startup so that I can make my test cases easier to write and maintain. When I took Bertrand’s sample code and started making changes to support type replacements, I broke everything. Tests didn’t work. Type replacements didn’t work. Nothing behaved like I expected. Basically, everything worked when I was NOT using selenium but then when I tried doing type replacements and testing them with Selenium-based tests, the type replacements disappeared.
Turns out that when you make the calls to create an instance of TestServer, that what you’re doing is starting up a *SECOND* instance of your application! Ahhhhhhh. So all my type replacements were happening in one instance of my app but not the other instance.
The Solution
In the end, I ended up creating my own version of WebApplicationFactory<T>. In order to make type replacements easy, the constructor for CustomWebApplicationFactory<T> takes an optional Action<IWebHostBuilder> parameter. This parameter value gets stored and then later gets called during the initialization of the application code and web servers.
The CustomWebApplicationFactory<T> is going to be what’s known as the System Under Test (SUT). The SUT is the code that we’re trying to verify using our test code. In our integration test code, we’ll initialize the SUT and optionally pass in an instance of Action<IWebHostBuilder> that contains our runtime adjustments to the application configuration.
Summary
Initialize the System Under Test using our custom WebApplicationFactory<T> and our calls to IWebHostBuilder and now our Selenium tests run happily.
The source code for this is available at Github. There’s a very simple ASP.NET Core MVC web app that I use to test again. Then there are a handful of tests in IntegrationTestFixtures.cs. The logic for loading WebApplicationFactory<T> is in CustomWebApplicationFactory.cs.
I hope this helps.
-Ben
Leave a Reply