Manual Testing Code performs by a human sitting in front of a computer carefully executing the test steps. Automation Testing means using an automation tool to execute your test case suite.
The problems of Manual Testing
- Prone to human errors: because the QA engineers are humans and they will mistake and may miss some bugs
- Time-Consuming: it takes a very long time to test
- Regression Test: every time we add new feature previous features should be tested too because something may break in development mode
The automation software can also enter test data into the System Under Test, compare expected and actual results and generate detailed test reports. Test Automation demands considerable investments of money and resources.
Automated testing is the practice of writing code to test our code, and then run those tests in an automated fashion. So, with automated testing, our source code consists of the application code, which we also call production codes and test code.
Why do we need to test our code?
because each application certainly has bugs in it so and developers should find these bugs before got to production, Fixing bug in production is ten time expensive than findings bugs manually by QA engineers and Fixing bugs in production is a hundred times expensive than development mode so we need to catch these bugs before the production and best way to do this is writing the tests for decreasing our costs.
imagine you have this code:
It’s a function that takes the time from the input and goes to some condition based on this input and returns different values as a time of day. If you want to test this function manually, you have to launch your application in the browser, perhaps you have to login, or maybe you have to do a few clicks to go to the page that this function exists and fill your time in the date picker and click on the submit button and see the result of this function on the screen then you have to repeat all these steps each time using different values in your form.
As you can see, this is very time-consuming. This workflow to test this function may take several minutes every time. Now to make matters worse, this is not the only function in your application. In a real application, you have tens or hundreds of functions like this. As your application grows, in size and complexity, the time required to manually test all the different bits and pieces increases exponentially.
With automated testing for testing .net code, you write code and directly call this function with different inputs and verify that this function returns the right output. Now, you can re-run these tests every time you change your code, every time you commit your code to a repository and before deploying your application. With this approach, you can test all the execution paths in this function in less than a second! You can write several hundred or thousands of automated tests for various parts of your application, and run them all in just a few seconds.
The Benefits of Automated Testing
- Test Application in less time as described in above
- Catch the bugs before deploying the application
- deploy our application with more confidence
- Refactor Code With Confidence
- Improve the Quality of code with different inputs
- Free to Run over and over
- Run Any Time
- Quicker to run compare to human manual testing
- Reduce Cost: Because we find bugs before production
- Fast: Automation testing is so faster than manual testing
- Less Prone to human errors
- Easier Regression Test: because these test runs multiple times and this makes regression testing more easier and we can check side effect new feature to previous feature multiple times
Types of Automated Test
We have three major types of tests:
- Unit test
- Integration Test or Service Test
- End to End Test (E2E) or Acceptance Test or UI Test
A Unit Test for testing .net code is a process that tests behavior within an application in isolation from any of its dependencies such as databases, message queue, file system and etc. The behavior of a component being tested can be isolated from the behavior of its dependencies using substitute dependencies known as
Test Doubles such as mock and stub (maybe I explain test doubles with detail in another post in future).
In unit testing for testing .net code, we only test behaviors that expose through public APIs, not private ones. when we call an external component from unit test it becomes an integration test in the unit test we must mock them.
The unit stands for “unit of work” or a “use case” inside the system. This idea of a unit of work means, to me, that a unit can span as little as a single method and up to multiple classes and functions to achieve its purpose.
A unit test is a piece of code (usually a method) that invokes a unit of work and checks one specific end result of that unit of work and checks the correctness of some assumptions. If the assumptions on the end result turn out to be wrong, the unit test has failed. A unit test’s scope can span as little as a method or as much as multiple classes.
A unit test should be
Cheep to write and
fast in execution. So it shouldn’t cost a lot of time to write one. And in fact, setting up a unit test is much cheaper than setting up an integration test. in this test because we don’t test external dependency and we only test isolate unit (with using mock) so this test doesn’t give us a lot of confidence and we need an integration test also. Usually, we write a unit test before or immediately after implementing the logic. In the best case, you’ll write a unit test before implementing the logic. This would say the TDD approach.
Properties of a Good Unit Test
- automated and repeatable.
- easy to implement.
- relevant tomorrow.
- Anyone should be able to run it at the push of a button.
- It should run quickly.
- consistent in its results (it always returns the same result if you don’t change anything between runs).
- should have full control of the unit under test.
- fully isolated (runs independently of other tests).
- When it fails, it should be easy to detect what was expected and determine how to pinpoint the problem
An Example for Unit Test
Here I create a simple example to show you the structure of a unit test. Assume you have a CalculationParser class that you’d like to test. It has a method named ParseAndSum that takes in a string of zero or more comma-separated numbers. If there are no numbers, it returns 0. If there’s a single number, it returns that number as an int and if there are multiple numbers, it adds them all up and returns the sum.
you can write a CalculationParserTests class as shown in the following code and add your test scenarios with some methods to this test class. our test method (Test Scenario) invokes a method on SUT (system under test) class, here it’s CalculationParser and then checks the returned value. If it’s not what’s expected, the test method writes
to the console. It also catches any exception and writes it to the console.
Another example of a Unit Test:
Test application with external dependencies, It tests the composition of all units. It ensures that all units are working together in the right way. This means it may need a lot more effort to setup a test because you need to setup the dependencies. Tests that verify that components behave as expected with all or a subset of their real dependencies are often categorized as Integration Tests. Of particular interest are interactions with third-party libraries and external resources such as file systems, databases, and network services.
So an integration test for testing .net code uses real dependencies; unit tests isolate the unit of work from its dependencies so that they’re easily consistent in their results and can easily control and simulate any aspect of the unit’s behavior
Integration testing is testing a unit of work without having full control over all of it and using one or more of its real dependencies, such as time, network, database, threads, random number generators, and so on.
These tests take long running time to execute and are slow because they collaborate with external dependencies but give users more confidence in the test of the application.
An Example of Integration Test
In this example, I show you a simple example of an integration test. I don’t go too deep in this example I show you in detail about the integration test another post, I use XUnit in this sample.
End to End Test
that tests an application through its user-interface. There are specific tools built for creating end-to-end tests.
One popular tool that you might have heard of is Selenium, which allows us to record the interaction of a user with our application and then play it back and check if the application is returning the right result or not. These tests for testing .net code give you the greatest amount of confidence about the health of your application.
End to End Tests have Two big problems:
- they are very slow. Because they require launching the application and testing it through the UI. So, every test is going to launch the application, potentially login, navigate to an internal page, submit a form and inspect the result.
- they’re very brittle because a small enhancement to the application or a small change in the user-interface can easily break these tests.
For each project in your solution, you’ll have a unit testing project. So, if there is a project called HotelBooking, we should have a separate project called HotelBooking.UnitTests and a separate project for integration tests (HotelBooking.IntegrationTests) because unit test executes fast, integration tests take longer. to run unit tests frequently as we’re writing code, and run integration tests just before committing our code to the repository to make sure everything works. for each project in the solution, you’re going to have a separate unit testing project.
In this project, you often have a test class for each class in your production code. So if you have a class called reservation, you should have a class called reservation tests. Note the plural name here. This indicates that this class has multiple tests. Now, for each method in the reservation class, you should have one or more test methods.
How many tests do you need? It depends on what you’re testing. Often, the number of tests is equal to or greater than the number of execution paths.
The name of your test methods should clearly specify the business rule that you’re testing. there are many naming conventions for the test, we use popular naming convention introduced by Roy Osherove.
- For example ReserveRoom_Room_RoomIsReserved, in this sample first part of the test name is the name of the method being tested(ReserveRoom), the second part is scenario or state under which the method is being tested in this example Room as that’s whats you’re passing through to add it and last part is expected behavior when the scenario invoked in other words when room is reserved with ReserveRoom method.
With this convention, we can look at the name of a test method, and tell what business rule is being tested.
Now, sometimes you’re dealing with a large complex method with so many execution paths and edge cases. In that case, it may be better to dedicate a separate test class for that method. Because otherwise, the test for this method may pollute your test class. Let’s say in our example, ReserveRoom method requires several tests. You can extract all the tests for this method, into a separate class and call it Reservation_ReserveRoomTests. This way you can easily locate the tests for this method.
Arrange, Act and Assert (AAA) Pattern
The AAA (Arrange-Act-Assert) pattern has become almost a standard across the industry. It suggests that you should divide your test method into three sections: arrange, act and assert. Each one of them only responsible for the part in which they are named after.
- Arrange section: used to set up the test like setup mock or initialize objects or create test data
- Act Section: as the name implies this means invoking the method or properties that we want to test, some times parameters that set in arrange section use here
- Asset Section: verify and checking results from the Act section, if the result of the Act section matches the expected result in our test then the test will pass otherwise the test will fail.
This pattern has several significant benefits. It creates a clear separation between a test’s setup, operations, and results. This structure makes the code easier to read and understand and makes it easier to find out why a test fails. If you place the steps in order and format your code to separate them, you can scan a test and quickly comprehend what it does.
If you want to get serious about automated tests for your software there is one key concept you should know about: the test pyramid. Mike Cohn came up with this concept in his book Succeeding with Agile. It’s a great visual metaphor telling you to think about different layers of testing. It also tells you how much testing to do on each layer.
Mike Cohn’s original test pyramid consists of three layers that your test suite should consist of (bottom to top):
- Unit Tests
- Service Tests or Integration Test
- User Interface Tests or End to End Test
This pyramid argues that most of your tests should be in the category of unit tests because these tests are easy to write, and they execute quickly. But since they don’t give you much confidence about the health of your application, you should have a bunch of integration tests that test the integration of your application code with its external dependencies. These tests provide many advantages of end-to-end tests, but without the complexities of dealing with the user interface. we should write very few end-to-end tests for the key functions of the application, but you should not test the edge cases with these end-to-end tests. You only test the happy path and leave the edge cases to unit tests.
The pyramid says that tests on the lower levels are cheaper to write and maintain, and quicker to run. Tests on the upper levels are more expensive to write and maintain, and slower to run. Therefore you should have lots of unit tests, some service tests, and very few UI tests.
Unit tests are great for quickly testing methods with complex logic and calculation, you should test them with your unit tests. the example for this scenario is
"TimeOfDay" function in earlier of this post because we can quickly test all execution paths in this function with different test scenarios in less than a second. manually test this function through the user interface takes more amount of time and prone to errors.
Also, we may need to read some data or store some data in a database or file system or calling another service in these scenarios we need an integration test for testing collaboration with external resources.
This test pyramid gives us three recommendations:
- Favor unit test over UI or End to End Test: because a unit test is fastest to run and cheapest to write and it is Precise and gives us rapid feedback.
- Cover unit tests gaps with integration tests
- Use the End to End Test rarely: we use the End to End test only for key functions of our application.
This is a measure to describe the degree to which the source code of an application is executed when tests are run.
If you have a high code coverage (so a lot of unit tests that span across the lines of code of your application), you have a lower chance of running into bugs, compared to applications with lower code coverage.
A good deal of different metrics can be used to calculate code coverage, which is measured as a percentage. The most basic is the calculation of the percentage of application statements called during the execution of your tests.
We can use some tools like Visual Studio Enterprise or JetBrains dotcover tools for code coverage feature.
Test-Driven Development (TDD)
Test-driven development, or TDD, also called test first, is an approach to build software. With TDD, you write your test before writing the application or production code. The first time I heard this I asked how on earth is that even possible. So, let’s see how TDD works.
- You start by writing a failing test: This test should fail because you don’t have any application code that would make it pass, right?
- Write the simplest application code that will make this test pass: we don’t want to over-engineer here, you don’t want to design a class diagram with many classes and methods. Use the simplest implementation that would make the test pass.
- Refactor your code if necessary
These three steps are the foundation of TDD. You repeat these three steps over and over until you build a complete feature. Now, what is so special about TDD?
- Source code will be testable right from the get-go: You don’t have to make any changes to your code to make it testable.
- Every line of your production code is fully covered by tests: This means you can refactor and deploy with confidence
- often results in a simpler implementation: When you start with a big class diagram. Chances are you are over-engineering and making the solution more complex. If you write enough code to make all the tests pass, and that solution works, there is no reason to write more code.
The fact that all your tests passed, means you have fulfilled all the business requirements. So, unless there is a new requirement, you don’t need to write new code. And if there is a new requirement, you start with failing tests. So this is TDD. In TDD, we write our tests first, and that’s why we call this approach test-driven development.
So our development is driven by our tests. In contrast to TDD, or test first, we have code first, which is what you have been doing so far. You start with your application code, and then you write tests. Which approach is better? In theory, TDD is more promising because of the benefits I told you. But in practice, sometimes it can get really complex and it may slow you down. If that’s the case, it’s better to switch to the code-first approach and write your tests after. Once you master the testing fundamentals, then you will be ready to start your test first journey.
I will explain testing .net code with xunit in this post series and also I explain unit test .net code and integration testing in detail in separate posts in future posts.