On a general level, defect tracking is one of the key components of the Defect Life cycle. This is crucial because, while testing software, test teams uncover multiple flaws that only become more noticeable when the system being tested is intricate. It can be difficult to manage and analyze these flaws in order to promote closure in such a situation.
When a tester reports a defect, he must include some categorical information that would help with an incorrect classification of the fault in addition to the method or description to reproduce the issue observed. This is in accordance with defect maintenance procedures. This would serve as the foundation for faster defect turnaround times and aid in effective defect monitoring and maintenance procedures.
The two main parameters that form the basis for effective Defect Tracking and Resolution are:
- Defect Priority in Testing
- Defect Severity in Testing
These are often a confusing concept and are almost used interchangeably amongst not only test teams but also development teams. There’s a fine line between the two and it’s important to understand that there are indeed differences between the two.
Let’s understand briefly the theoretical definitions of the two parameters in the next section.
What Is Defect Severity And Priority
In the English definition, priority is used to compare two things or conditions, where one needs to be given more importance than the other(s) and needs to be tackled/resolved first before moving on to the next one(s). Therefore, in the context of defects, the priority of a defect would indicate the urgency with which it would need to be fixed.
Severity, by the English definition, is used to describe the gravity of an undesirable occurrence. Hence, with bugs, the severity of a bug would indicate the effect it has on the system in terms of its impact.
Who defines these?
Based on the defect’s complexity and criticality, QA assigns a severity rating.
Project managers, business analysts, and product owners are among the business stakeholders who determine the flaws’ priority.
The role of who owns and categorizes the criticality and severity of the flaws is shown in the figure below.
How to choose these levels?
As previously said, the tester evaluates the severity parameter, while the triage team or the product manager primarily evaluates the priority parameter. Despite this, one of the determining and impacting elements for defect prioritization is the defect’s severity. To prevent misunderstandings with development teams, a tester must choose the appropriate severity.
Difference Between Severity And Priority
Priority is associated with scheduling, and “severity” is associated with standards.
“Priority” means something is afforded or deserves prior attention; order of importance (or urgency establishes precedence).
The state or quality of being severe is known as “severity.” It is characterized by or necessitates tight adherence to high standards or rigorous standards, and it frequently connotes harshness.For instance, a strict code of conduct.
The words priority and severity come up in bug tracking.
A variety of commercial, problem-tracking/management software tools is available. These tools, with the detailed input of software test engineers, give the team complete information so developers can understand the bug, get an idea of its ‘Severity’, reproduce it, and fix it.
The fixes are based on the project ‘Priorities’ and ‘Severity’ of bugs.
The ‘Severity’ of a problem is defined following the customer’s risk assessment and recorded in their selected tracking tool.
Buggy software can ‘severely’ affect schedules, which can lead to a reassessment and renegotiation of ‘priorities’.
What Is Priority
As the term implies, priority refers to ranking a problem according to its severity and business requirements. Priority denotes how urgent or significant a flaw has to be fixed.
As he looks at the product from the end-user’s point of view, the tester first determines the priority when opening a fault. Accordingly, there are various levels:
Broadly, the Priority of the defects can be classified:
Priority #1: Immediate/Critical (P1)
This needs to be resolved right away, within 24 hours. This happens when all functionality is disabled, making it impossible to test. In some other situations, if there are notable memory leaks, the fault is rated as a priority -1, which indicates that the application or feature is currently unusable.
Any fault that impacts the testing process and requires quick care will be placed in the immediate category.
This category includes all critical severity faults (until stakeholders or the company reprioritize them).
Priority #2: High (P2)
In order for any test activity to meet the “exit” criterion, a defect with this priority must be rectified after the critical problems have been fixed.
Priority 2 defects typically occur when a feature is not useful as intended because of a program flaw, the need to write new code, or occasionally even because the code must address an environmental issue.
This is the flaw or problem that needs to be fixed prior to the release. After we take care of the critical issues, let’s fix these flaws.
All the Major severity defects fall into this category.
Priority #3: Medium (P3)
A defect with this priority must be in contention to be fixed, as it could also deal with functionality issues that are not as per expectation. Sometimes even cosmetic errors, such as expecting the right error message during the failure, could qualify to be a priority 3 defect.
This defect should be resolved after all the serious bugs are fixed.
Once the Critical and the High-priority bugs are done, we can go for the medium-priority bugs.
All the Minor severity defects fall into this category.
Priority #4: Low (P4)
Low priority defects show that there is a problem, but they do not need to be corrected in order to meet the “exit” requirements. But before the GA is finished, this needs to be resolved. Generally speaking, this category could include some typographical problems or even cosmetic faults, as was previously said.
Low priority defects are occasionally also created to propose improvements to the current design or to request the addition of a minor feature to improve the user experience.
This problem, which is classified as low severity, is fixable and does not require immediate care.
Priority dictates the required defect turnaround time, as was previously said. When there are several flaws, the priority determines which one needs to be corrected first.verified immediately versus which defect can be fixed a bit later.
What Is Severity
Severity defines the extent to which a particular defect could create an impact on the application or system.
Severity is a parameter to denote the implication of defect on the system–how critical defect is and what is the impact of the defect on the whole system’s functionality? The severity is a parameter set by the tester while he opens a defect and is mainly in control of the tester. Again, different organizations have different tools to use for defects, but on a generic level, these are the following severity levels:
For Example, consider the following scenarios:
- If the user tries to do online shopping and the application does not load or the server is unavailable message pops up.
- The user performs adding an item to the cart, number of quantities added is incorrect/wrong product gets added.
- The user makes the payment and after the payment; the order stays in the cart as reserved instead of confirmed.
- The system accepts the order but finally cancels the order after half an hour due to any issues.
- The system accepts the “Add to Cart” on a double click only instead of on a single click.
- The Add To Cart button is spelled as Add To Cart.
What would be the user experience, if any of the above scenarios could happen?
Broadly, the defects can be classified:
#1) Critical (S1)
A defect that completely hampers or blocks testing of the product/feature is a critical defect. An example would be with UI testing where, after going through a wizard, the UI just hangs in one pane or doesn’t go further to trigger the function. Or in some other cases, when the feature developed itself is missing from the build.
For any reason, if the application crashes or becomes unusable/not able to proceed further, the defect could be classified under critical severity.
Any catastrophic system failures could lead the user to non-usability of the applications could be classified under the Critical severity
For example, In an email service provider like Yahoo or Gmail, after typing the correct username and password, instead of logging in, the system crashes or throws an error message, this defect is classified as critical as this defect makes the whole application unusable.
The scenario in point 1 discussed above could be classified as a Critical Defect, as the online application becomes completely unusable.
#2) Major (S2)
If a major feature doesn’t meet its requirements or use cases and behaves differently than expected, it falls under Major Severity.
When functionality deviates significantly from expectations or fails to perform its intended function, it is considered a serious defect. One illustration might be: Let’s say you are utilizing a UI template that initiates the deployment of a VLAN on the switch. A major functionality limitation occurs when the switch’s VLAN configuration template fails.
For example, in an email service provider like Yahoo or Gmail, when you are not allowed to add more than one recipient in the CC section, this defect is classified as a Major defect as the major functionality of the application is not working properly.
What is expected of the behavior of the CC section in the mail, it should allow the user to add multiple users. So when the major functionality of the application is not working properly or when it behaves differently than expected, it is a major defect.
The scenarios on points 2 & 3 discussed above could be classified as Major Defects, as the order is expected to move smoothly to the next phase of the order life cycle, but in reality, it varies in behavior.
Any defect that could lead to incorrect data persistence, data issues, or wrong application behaviors could be broadly classified under the Major severity.
#3) Minor/Moderate (S3)
Any implemented feature that operates differently than intended and does not match its requirements or use case(s) can be categorized as minor severity if its impact on the application is minimal or non-significant.
When a product or application fails to meet specific requirements or continues to display some abnormal behavior, it is considered to have a significant fault. The functionality is unaffected, though. For instance, when the VLAN template is correctly installed on the switch, a moderate or usual flaw will arise. Nevertheless, the user is not receiving any notification.
For example, in an email services provider like Yahoo or Gmail, there is an option called “Terms and Conditions” and in that option, there will be multiple links regarding the terms and conditions of the website.
When one among the multiple links is not working fine, it is called a Minor severity as it only affects minor functionality of the application and it doesn’t have a big impact on the Usability of the application.
The scenario on point 5 discussed above could be classified as Minor Defect, as there is no data loss or failure in system flow order but a slight inconvenience for user experience.
These types of defects result in minimal loss of functionality or user experience.
#4) Low (S4)
Low Severity refers to any aesthetic flaws, such as misspellings, alignment problems, or font casing.
Even if a tiny low-severity problem seldom affects functioning, it is still a legitimate flaw that needs to be fixed. Examples of this could include spelling mistakes in error messages printed to users or defects to enhance the look and feel of a feature.
For example, in an email services provider like Yahoo or Gmail, you would have noticed the “License page”. If there are any spelling mistakes or misalignment on the page, this defect is classified as Low.
The scenario on point 6 discussed above could be classified as a Low Defect, as the Add button is displayed in the wrong Casing. This kind of defect will not impact system behavior or data presentation or data loss or data flow or even user experience, but will be very cosmetic.
To summarize, the following figure depicts the broad Defect classification based on Severity and Priority:
Examples
As was previously said, different organizations use different technologies for defect tracking and related processes, thus different technical staff and management levels use the same tracking system.
The Test Engineer determines the defect’s severity because it falls more inside the functionality’s scope.Although the developers have some influence over the severity of the problem, the tester ultimately determines how much a certain feature can affect overall functionality.
On the other hand, when it comes to setting defect priority, although initially, the defect originator sets the priority, it is defined by the Product Manager as having an overall view of the product and how quickly a particular defect has to be addressed. A tester is not an ideal person to set the defect priority.
Shocking as this may seem, there are two distinct examples as to why:
Example #1: Consider that there is a situation where the user finds a mistake in the product’s naming itself or some problem with the UI documentation. A tester would normally open a minor/cosmetic defect and may be very simple to fix, but when it comes to the product’s look and feel/user experience, it could cause a serious impact.
Example #2: There could be certain conditions under which a particular defect occurs which may be an extremely rare or no possibility to hit in the customer environment. Even though functionality-wise this may seem like a high-priority defect to a tester, considering its rarity of occurrence and high cost to fix – this would be classified as a low-priority defect.
Hence, in effect, the defect priority is generally set by the product manager in a “defect triage” meeting.
Different Levels
Priority and Severity have some classifications that aid in determining how the defect must be handled. A lot of different organizations have different defect logging tools, so the levels might vary.
Let’s look at the different levels for both Priority and Severity.
- High Priority, High Severity
- High Priority, Low Severity
- High Severity, Low Priority
- Low Severity, Low Priority
The following figure depicts the classification of the categories in a single snippet.
#1) High Severity and High Priority
Any Critical/major business case failure automatically gets promoted to this category.
Any defects due to which the testing cannot continue at any cost or cause a severe system failure fall into this category. For example, clicking on a particular button doesn’t load the feature itself. Or performing a particular function brings down the server consistently and causes data loss. The red lines in the above figure indicate these kinds of defects.
For example, the system crashes after you make the payment or when you cannot add the items to the cart. This defect is marked as a High Severity and High Priority defect.
Another example would be the ATM vending currency feature wherein after entering the correct username and the password, the machine does not dispense money but deducts the transferred from your account.
#2) High Priority and Low Severity
Any minor severity defects that could directly impact the user experience automatically get promoted to this category.
Defects that have to be fixed but do not affect the application come under this category.
For example, the feature is expected to display a particular error to the user concerning its return code. In this case, functionally the code will throw an error, but the message will need to be more relevant to the return code generated. The blue lines in the figure indicate these kinds of defects.
For example, the logo of the company on the front page is wrong, it is considered to be High Priority and Low Severity defect.
Example #1: In the Online shopping website when the FrontPage logo is spelled wrong. For example, instead of Flipkart it is spelled as Flipkart.
Example #2: In the bank logo, instead of ICICI, it is written as ICCCI.
In terms of functionality, it is not affecting anything so we can mark as Low Severity, but it has an impact on user experience. This kind of defect needs to be fixed on high priority even though they have very little impact on the application side.
#3) High Severity and Low Priority
Any flaw that has functional ramifications for the system or fails to fulfill requirements but is pushed to the back burner by stakeholders due to business criticality is automatically elevated to this category.
flaws that need to be corrected, but not right away. In particular, this can happen during ad hoc testing. It indicates that although the functioning is significantly impacted, it is only noticeable when specific unusual input parameters are utilized.
For instance, a specific feature may only be utilized with a later firmware version. To confirm this, the tester downgrades his system, runs the test, and finds a significant and legitimate functionality fault.
In such a case, the defects will be classified in this category denoted by pink lines, as normally end users will be expected to have a higher version of the firmware.
For example, in a social networking site, if a beta version of a new feature is released with not many active users using that facility as of today. Any defect found on this feature can be classified as a low priority as the feature takes back seat due to business classification as not important.
Though this feature has a functional defect, as it does not impact the end customers directly, a business stakeholder can classify the defect under low priority, though it has a severe functional impact on the application.
This is a high severity fault but can be prioritized to low priority as it can be fixed with the next release as a change request. Business stakeholders also prioritize this feature as a rarely used feature and do not impact any other features that have a direct impact on user experience. This kind of defect can be classified under the High Severity but Low Priority category.
#4) Low Severity and Low Priority
Any spelling mistakes /font casing/ misalignment in the paragraph of the 3rd or 4th page of the application and not in the main or front page/ title.
These defects are classified in the green lines as shown in the figure and occur when there is no functionality impact, but still not meeting the standards to a small degree. Generally, cosmetic errors or say dimensions of a cell in a table on UI are classified here.
For example, if the privacy policy of the website has a spelling mistake, this defect is set as Low Severity and Low Priority.
Guidelines
Below are certain guidelines that every tester must try to follow:
- Firstly, understand the concepts of priority and severity well. Avoid confusing one with the other and using them interchangeably. In line with this, follow the severity guidelines published by your organization/team so that everyone agrees.
- Always choose the severity level based on the issue type, as this will affect its priority. Some examples are:
- For a critical issue, such as the entire system goes down and nothing can be done, this severity should not be used to address program defects.
- For a major issue, such as where the function is not working as expected, this severity could address new functions or improvements in the current working.
Remember that choosing the right severity level will give the defect. It’s the due priority.
- As a tester, understand how a particular functionality, rather than drilling down further understand how a particular scenario or test case would affect the end-user. This involves a lot of collaboration and interaction with the development team, Business Analysts, architects, Test lead, and Development lead. In your discussions, you also need to factor in how much time it would take to fix the defect based on its complexity and time to verify this defect.
- Finally, it’s always the product owner who possesses the veto power of the release of the defect that should be fixed. However, since the defect triage sessions contain varied members to present their perspectives on the defect on a case basis, at such a time, if the developers and testers are in sync, it surely helps in influencing the decision.
Conclusion
It is the tester’s duty to appropriately assign the flaws’ severity when they are being opened. The entire STLC process and the final output may be severely impacted by improper severity and, consequently, priority mapping. Priority and severity are topics that are frequently covered in employment interviews to make sure that, as a tester, you are well aware of them.
Additionally, we saw real-time demonstrations of how to categorize defects into different Severity/Priority buckets. I wish you had provided sufficient details on the severity and priority buckets for defect classification by now.
YOU MAY BE INTERESTED IN
The Art of Software Testing: Beyond the Basics
Automation testing course in Pune