Smoke Testing and Everything You Need to Know About it

Testers and developers face several types of testing software challenges. There are unit tests, integration tests, end-to-end tests, and smoke tests, to name just a few. Each of these has a distinct purpose and use cases, all with the goal of making sure that software works as it should as well as catching any bugs or software malfunctions before your users do. How does smoke testing fit into the mix? Read on to find out what it is and why you should be smoke testing your applications.

What is Smoke Testing? 

Smoke testing is very much akin to what it sounds like: imagine filling an airtight container with smoke and then checking for any leakages. If you can see smoke escaping, then you know something is wrong. It may not tell you much about what is wrong or why, but that’s not its purpose. Its purpose is simply to alert you that something fundamental is wrong. Smoke testing in software is much the same: it’s a relatively quick and imprecise way to check that your core functionality still works after making changes to your codebase. It’s sometimes referred to as ‘surface level testing’ or ‘build verification testing’, as it is intended to simply demonstrate that an application still works as expected, rather than getting very deep into testing specific modules or functions.

Smoke testing is similar to regression testing, in that it is applied after new changes are made in order to check that your core functionality still works as expected. However, regression testing is generally more in-depth and tests against specific modules. It generally takes longer to run (by several orders of magnitude for sufficiently mature projects,) which makes it unsuitable for a quick ‘first pass’ round of testing. Smoke testing is intended to quickly indicate whether something is wrong at a surface level, whereas regression testing is much more in-depth.

Why is Smoke Testing Important?

As mentioned, smoke testing is performed to give an early indication as to whether something has gone wrong or not. If an application doesn’t pass smoke testing, you know that it needs to be sent back to the developers. However, just because an application passes smoke testing, that doesn’t necessarily indicate that there are no issues, but rather that nothing major appears out of the ordinary at first glance. Smoke testing is favored because it is much faster to run than other types of testing. It’s an ‘early warning’ system intended to catch breaking changes and major issues, without having to wait potentially hours for larger test suites to return information about obvious issues related to the core functionality.

To put it succinctly, smoke testing is important because it catches major issues and showstoppers quickly. This allows developers to receive feedback quickly, allowing them to correct any issues as soon as possible.

When Should You Perform Smoke Testing 

Smoke testing is generally run as the first round of testing after new functionality has been added. Smoke testing is generally only applied when adding new functions to existing software that is already established, as opposed to being performed during initial development. Smoke testing can be considered the first line of defense; if problems are found during smoke testing, it can immediately be sent back to developers. If software passes smoke testing, it can proceed on to further testing.

As an example, a login feature is a core mechanic for most websites that allow users to register an account. When new changes are made, smoke testing should be run to ensure that this functionality still works. Smoke tests can be as simple as attempting to login. If this functionality no longer works and users cannot login, then this issue can be quickly highlighted and passed back to developers to fix.

Benefits of Smoke Testing 

Smoke testing brings a number of benefits over other types of testing, some of which have already been mentioned, such as the speed of smoke tests compared to more formal tests. There are a number of other benefits to performing smoke testing, such as:

They’re cheap and easy to run

Smoke tests are not very comprehensive but they are quick and cheap to run. When the goal of smoke testing is to make sure that the big picture functionality works as expected, they’re a very cost-effective way of highlighting major issues in code changes. They don’t require much in the way of time or resources to run.

They help detect bugs early on

Smoke tests are generally used as an initial first-pass before more comprehensive testing is performed. Bugs caught during smoke testing will typically be show-stoppers that prevent your application from working correctly. These are the kind of major issues that you want to be found as soon as possible, which is exactly what smoke testing does.

They improve QA efficiency

Smoke tests don’t require you to spend hours waiting for full test suites to run in order to highlight immediate problems. By running smoke tests ahead of other types of tests, your QA team does not need to spend hours waiting for a test suite to finish before detecting a major bug.

Important Tips For Performing Smoke Tests 

For teams looking to implement smoke testing for the first time, smoke testing can be performed manually. Since the tests involved are generally simple and surface level, they can easily be performed by non-engineers. Smoke testing is not an extensive type of testing, so no guarantees are expected if an application passes smoke testing, but it does free up your developers and testers to focus on more complicated or comprehensive test types.

However, many types of smoke tests are repetitive and, therefore, easily automated. Once you have a well-defined suite of smoke tests, you’ll probably want to consider automation. However, the problem is that automating your smoke tests is yet another engineering project, so it’s important to consider the role this type of testing will play.

Smoke testing is meant to be fast and cheap to run. If your smoke test suite is taking hours to run, then you should consider cutting down on the amount of tests you run. You need to find a happy medium between sufficient test coverage of the fundamentals and not too many that your test suite takes hours or more to complete. As a very rough guide, you should aim for somewhere between 20-50 tests. Too few and you’re probably missing some vital areas, too many and your test suite will be too slow to quickly and easily run against new code changes.

Another consideration to be given to automating smoke tests is the type of tests likely to feature in a smoke test suite. Since smoke tests are a surface level type of testing intended to highlight major functionality issues, it’s likely that your smoke tests will be composed mainly of functional UI tests. These types of tests are generally known to be flaky: UI components constantly get shuffled around, which often results in breaking automated tests. Your engineers should not be spending time fixing broken smoke tests before every build or it defeats the purpose of having a quick and cheap set of tests in the first place.

Using a popular front-end framework, such as Selenium, can help to alleviate some of those issues, but there is often a lot going on under the hood, even when simply testing the UI. Javascript events getting fired, asynchronous APIs returning data, animations being played out with CSS or SVG, etc. All of these must be accounted for in order to provide reliable, automated smoke tests.

Deploy Your Smoke Test

Smoke testing is useful for developing and running a quick set of tests to determine whether new code changes contain any fundamental flaws or breaking changes that affect the core functionality of an application. They’re intended to be quick and cheap to run, but they can be difficult to automate, due to the nature of smoke tests being typically front-end functional UI tests.

If smoke tests are not part of your testing strategy today and you want to get started, then get in touch with a member of staff at Guide Rails. Our team can share details for how to put your overall testing strategy into action by using our platform for software development, increasing both speed and quality of software delivered.