Ad-hoc Testing – Complete Guide

Finding problems or weaknesses that cannot be detected by conventional or formal techniques used during the testing cycle is the primary goal of this.

As previously mentioned, the fundamental aspect of this testing is the absence of a formal or structured testing methodology. It is evident when such random testing methods are used that the testers do so without any specific use case in mind to disrupt the system.

This makes it increasingly clearer that the tester must be highly competent, skilled, and well-versed in the system in order to use such an intuitive or creative testing methodology. Ad hoc testing is highly helpful in assessing the efficacy of the test bucket and guarantees that the testing is comprehensive.

Recommended reading => Exploratory Testing – How to Think Beyond Traditional Testing Boundaries?

An Ad-hoc Testing Example

Here is an example of how we can perform this testing for UI Wizard.

Let’s say you need to create a plan or a template for some kind of task to be performed using this UI wizard. The wizard is a series of panes that have user input, like name, description, etc.

As the wizard progresses: say on one pane, user data is to be entered which involves the UI wizard throwing a context pop-up box that adds the associated data to complete the wizard and deploy/activate the wizard.

To test this tester does his regular testing, such as:

• Complete the wizard successfully with all the valid data and create the plan.
• Cancel the wizard mid-way.
• Edit a created plan through the wizard.
• Delete the created plan and see that there is no residue of it.
• Enter a negative value in the wizard and see the error messages are seen.

Now, for the above example, here are some test cases for ad hoc tests that could be performed to uncover as many defects as possible:

• While trying to add negative data, we should add certain special characters that are not restricted to see if the system handles them properly. For Example, sometimes wizards don’t restrict {or [ braces, but in certain situations, this can conflict with code based on the language it is written in, and cause very unreliable behavior.
• Another test is specifically regarding pop-ups. A user can cause the pop-up to launch and then try to press the backspace button on the keyboard. Many times I have observed that doing so makes the background wizard completely disappear and the entire user data that was entered until the point the pop-up was launched is lost!

Characteristics of Ad-hoc testing:

If you note the scenarios above, you will see some very distinct characteristics of this type of testing.

They are:

• They are always in line with the test objective. However, there are certain drastic tests performed with the intent of breaking the system.
• The tester needs to have complete knowledge and awareness about the system being tested. The result finds bugs that attempt to highlight the loopholes in the testing process.
• When considering the two tests above, the logical response is that this type of test should only be done once unless there is a related defect that requires a re-test.

When Do We Do Ad-hoc Testing

A million-dollar question indeed!

Most of the time, test teams are always burdened with too many features to test within limited timelines. In that limited time, there are lots of testing activities that are derived from the formal process that must also be completed. In such situations, ad hoc testing finding its way into the testing is slim.

However, from my experience, one round of ad hoc testing can do wonders for product quality and raise many design questions. 

Since ad hoc testing is more of a “wild-child” testing technique that doesn’t have to be structured, the general recommendation is that it must be performed after the execution of the current test bucket is done. Another point of view is that this could be done when detailed testing cannot be performed because of less time.

I would say that ad hoc testing can be done almost any time – in the beginning, towards the middle, and at the end! It just finds its place at any time. However, when ad hoc testing must be done to bring out maximum value is best judged by an experienced tester who has in-depth knowledge of the system being tested. 

When not to execute?

If the previous question was worth a million dollars, this should be worth a billion!

While we’ve established how effective and fruitful ad hoc testing can be, as a skilled and capable tester, we also need to decipher when not to invest in this type of testing. Although it is at the discretion of the tester, here are some recommendations/examples when it might not be necessary. 

• Avoid this testing when there is a test case for which a defect exists. In such a situation, there is a need to document the test case failure point and then verify/re-test the defect when it is fixed. Hence, it won’t be applicable here.
• There may also be certain scenarios where customers or clients may be invited to test the beta version of the software. In such cases, this testing should not be conducted.
• Another scenario is when there is a very simple UI screen that is added. Traditional positive and negative testing should suffice here to bring out maximum defects.

Types Of Ad-hoc Testing

There are three categories below in which we can categorize ad-hoc testing:

#1) Buddy Testing

A test person and a development person will be selected to work on the same module in this type of testing. The developer and tester sit down to work on the module shortly after the developer finishes the unit testing. Both parties can view the feature in a wider context thanks to this type of testing.

By gaining insight into the various tests the tester does, the developer will be able to better understand the intrinsic design, which will help the tester avoid creating invalid situations and, ultimately, invalid faults. It will enable one to think as the other does.

#2) Pair Testing

In this testing, two testers work together on a module with the same test setup shared between them. The idea behind this form of testing is to have the two testers brainstorm ideas and methods to have several defects. Both can share the work of testing and make necessary documentation of all observations made.

#3) Monkey Testing

Testers mainly perform this testing at the unit testing level. The tester parses data or tests completely randomly to ensure that the system can withstand any crashes. Testers further classify this testing into two categories.

Ad-hoc Testing Benefits

Testing warrants the tester with a lot of power to be as creative as necessary.

This increases the testing quality and efficiency as below:

• The biggest advantage that stands out is that a tester can find the number of defects than in traditional testing because of the various innovative methods they can apply to test the software.
• This form of testing can apply to any stage of the SDLC. It’s not only restricted to the testing team. The developers can also conduct this testing, which would help them code better and also predict what problems might occur.
• It can be coupled with other testing to get the best results, which can sometimes cut short the time needed for regular testing. This would enable better quality test cases to be generated and better quality of the product on the whole.
• It doesn’t mandate any documentation to be done which prevents the extra burden on the tester. A tester can concentrate on actually understanding the underlying architecture.
• When there is not much time available to test, this can prove very valuable in terms of test coverage and quality.

Ad-hoc Testing Drawbacks

There are also a few disadvantages to ad hoc testing. Now, let’s examine some of the notable drawbacks.

Since it’s not very organized and there is no documentation mandated, the most evident problem is that the tester has to remember and recollect all the details of the ad hoc scenarios in memory. This can be even more challenging, especially in scenarios where there is a lot of interaction between different components.

• Following from the first point, this would also result in not being able to recreate defects in the subsequent attempts if asked for information.
• Another very important question this brings to light is the effort of accountability. Since there is no planning or structure, it becomes impossible to account for the time and effort invested in this kind of testing.
• A very knowledgeable and skilled tester in the team must be the one to perform ad-hoc testing, as it requires being proactive and intuitive in terms of foreseeing potential defect-ridden areas.

Tips For Maximizing the Effectiveness of This Testing

We’ve discussed at length the strengths and weaknesses associated with this testing.

Ideally, ad hoc testing should find its place in the SDLC, however, if not approached appropriately, it can prove costly and a waste of valuable testing time. So, given below are a few pointers to make ad hoc testing effective:

#1) Identify Defect prone areas

When you have a good hold over testing a particular piece of software, you agree that there will be certain features that are more prone to errors than others. If you’re new to the system, then check the features v/s defects opened against them.

The number of defects in a particular feature will show you it’s sensitive and you should precisely choose that very area to perform ad hoc testing. This proves to be a very time-efficient way of exposing some serious defects.

#2) Building expertise

Undoubtedly, a tester who has more experience is more intuitive and can guess where the errors might be when compared to someone who has not much experience. I would say, experienced or not, it’s up to the individual to take the plunge and build expertise on the system that is being tested.

Yes, experienced testers have an edge as their skills built up over the years come in handy, but the new testers should use this as a platform to gain as much knowledge as possible to design better ad hoc scenarios.

#3) Create test categories

Once you are aware of the list of features to be tested, set aside a few minutes to decide how you would categorize those features and test them. For example, decide to test features that are most visible and most commonly used by the user before anything else, as these would seem critical to the software’s success.

Then you could categorize them functionality/priority-wise and test them segment by segment.

Another example where this is particularly very important is if there is integration between components or modules. In these cases, there can be a lot of abnormalities that can occur. Using categorization would help touch upon this kind of test at least once or twice.

#4) Have a rough plan

Yes, this point might confuse you a bit as we described ad hoc testing as testing which should have no planning or documentation. The idea here is to stick to the essence of ad hoc testing, but still have some rough pointers jotted down on how you plan to test.

A very basic example is that sometimes you may just not be able to remember all the tests you intend to perform. So jotting them down would ensure you don’t miss out on anything.

#5) Tools

Let’s take an example faced commonly by all of us. A lot of times if you observe, the testing of the functionality in itself is successful with no discrepancy reported in its behavior. However, the testers could miss some exceptions reported in the logs behind the scenes, as it doesn’t affect the test objective.

These could be even higher in severity. Hence, we need to learn tools that will help pinpoint this immediately.

#6) Document for more defects

Furthermore, this could raise some doubts. A brief note outlining all the various scenarios covered, any deviations in the methods involved, and the flaws for the specific test feature category can do as documentation; it need not be extensive.

This will assist you in enhancing the entire test bucket as well, allowing you to choose how to enhance current test cases or add new as needed.

Conclusion

We’ve talked about the advantages and disadvantages of ad hoc testing methods as well as the circumstances in which they could be useful.

This is one testing method that ensures a tester’s creativity will be fully accommodated and satisfied. Ad hoc testing has given me the most happiness in my whole testing career because it allows for endless innovation and only leads to increased understanding.

YOU MAY BE INTERESTED IN

The Art of Software Testing: Beyond the Basics

Automation testing course in Pune

Automation testing in selenium

Mastering Software Testing: A Comprehensive Syllabus