Technical Debt Derails Organizational Goals and Impedes Progress and Innovation
According to Philip Stanhope, the 4th Earl of Chesterfield, “Whatever is worth doing at all is worth doing well.” However, anyone who has worked in software development knows that projects have to meet time and budget constraints. Shortcuts and tradeoffs are par for the course, although it means more total work in the end when you have to do it the “right” way later.
Technical debt is the cost, risk, and liability associated with taking technology shortcuts. While some technical debt in software engineering might be unavoidable, it should be minimized and managed.
How? By understanding what types of technical debt you can and should avoid, what types you may have to live with, and how you should properly classify, prioritize, and manage the associated costs, risks, and liabilities.
What is Technical Debt?
Technical debt is the collection of liabilities and risks that accumulate when shortcuts or less-than-optimal paths are taken in the software development process.
Suppose a software development team adopts an inflexible framework to meet an early launch deadline. This framework allows the team to develop their application quickly, enabling them to meet their deadline. The framework also has performance issues that the team will need to resolve later. Once early launch success is achieved, the team must refactor the application using a better framework.
If the development team used the better framework initially, they would have invested less total time in the project. They chose to prioritize the launch date instead of the quality of the application.The team’s technical debt accrued in the form of the extra time and effort they used for refactoring because they didn’t build on an appropriate framework in the beginning.
This is deliberate technical debt. But technical debt can also be inadvertent, such as when a team produces low-quality code because they lack experience. The low-quality code will require a rewrite in the future–resulting in a higher total cost in terms of money and time–but may also introduce risk and liability into the product.
Technical debt can also accumulate over time in the form of bit rot. Wired describes bit rot as “A component or system [that] slowly devolves into unnecessary complexity through lots of incremental changes, often exacerbated when worked upon by several people who might not fully understand the original design.”
Types of Technical Debt
People have classified technical debt in a number of different ways. The larger categories of deliberate, inadvertent, and bit rot technical debt give insights into how this debt arises. Software developer and Agile Manifesto co-author Martin Fowler added some nuance by classifying deliberate and inadvertent technical debt as either reckless or prudent.
- Deliberate and reckless technical debt happens when the development team prioritizes speed over quality even though they don’t need to—and they could avoid the consequences if desired.
- Deliberate and prudent technical debt happens when the development team chooses to accumulate technical debt to avoid delays or other problems, or after having weighed the cost/benefits of doing so.
- Inadvertent and reckless technical debt can happen when inexperienced developers create applications without understanding and knowledge.
- Inadvertent and prudent technical debt consists of lessons learned and corrections made after accidental technical debt accrues.
You can avoid reckless technical debt with better planning, but knowingly carrying technical debt is not uncommon.
Researchers continue to refine their descriptions of technical debt. These categories have emerged:
- Architecture debt: Problems resulting from architecture defects which may affect performance, and which usually requires extensive intervention to correct.
- Build debt: Build-related problems that waste time such as the inclusion of unnecessary code.
- Code debt: Issues with the source code such as legibility and maintenance challenges.
- Defect debt: Defects in the source code identified through testing or user reports that cannot be fixed right away because of competing priorities. Accumulated defects become more challenging to correct the longer they are delayed.
- Design debt: Violation of the principles of good object-oriented design that may be discovered through source code analysis.
- Documentation debt: Missing or incomplete information. Those who need to reference the documentation later must reconstruct or guess at what should have been written initially.
- Infrastructure debt: Issues within an organization’s infrastructure such as delayed upgrades that can slow or impede development.
- People debt: Delays because of lack of expertise or too few people with the right expertise.
- Process debt: Inefficiencies in processes.
- Requirement debt: Trade-offs made to meet requirements.
- Service debt: Use of web services or substitution of services in order to meet business objectives can create technical debt during the transformation.
- Test automation debt: The work associated with automating tests.
- Test debt: Issues discovered during testing, problems resulting from testing process deficiencies, or the accumulation of problems resulting from failure to test.
These categories address the specific nature of each type of debt and link each of them to associated indicators.
The Cost of Technical Debt
The costs and liabilities associated with technical debt can be classified as direct or indirect. Direct technical debt can be quantified in terms of dollars or hours lost. Indirect technical debt produces liabilities that are more difficult to quantify, such as reduced functionality or lack of customer satisfaction.
Direct Technical Debt
Recent reports suggest that 69 percent of IT leaders identify technical debt as a major threat to the ability of their company to innovate. Moreover, companies spend an average of 28 percent of their budgets addressing technical debt, which is almost as much as the 33 percent they spend on driving innovation.
Contributors to the direct, measurable cost of technical debt include:
- The cost of hiring additional personnel to manage and maintain existing systems or replacing personnel who leave because they’re frustrated with technical debt
- The cost of resources and technologies needed to resolve issues resulting from technical debt
- The cost of data breaches or cyberattacks that occur because of security gaps related to technical debt
- Opportunity costs associated with development cycle slowdowns caused by technical debt
Indirect Technical Debt
Technical debt includes many costs and liabilities that can’t be easily quantified. These include:
- Diminished ability to innovate or remain competitive
- Limited ability to attract and retain qualified developers
- Increased employee frustration and low morale
- Decreased shareholder satisfaction
- Decreased customer satisfaction and retention
- Reduced decision-making ability due to lack of reliable data
- Decreased productivity
- Increased system outages
- Increased security vulnerabilities
Identifying Technical Debt
The first step in managing technical debt properly is to assess the full scope of your current technical debt. Technical debt has metrics, signs, and drivers that identify it.Once you know what they are, you can take a systematic approach to identify and assess your technical debt.
Drivers and Signs
Common technical debt drivers include:
- Financial constraints
- Time pressure
- Underqualified developers
- Inadequate training
- Low-quality platform
- Fragmented and hasty decision-making
- Irregular hardware and software patching
- Underinvestment in automated workflows
- Insufficient documentation practices
- Inconsistent standards or inconsistent application of existing standards
Drivers provide an indication of where you need to look, but you also need to know what to look for. Signs of technical debt include:
- Code smells, which are any characteristics of the code that indicate an underlying issue
- Design smells, which are any characteristics of the design or architecture that indicate an underlying issue
- Overlapping technology, which happens when you have multiple technologies that perform the same function
- Testing absence
- Excessive bugs in your code
- Inconsistent coding styles, which often results in inefficiencies
- Slowed production velocity
- Instability of the production environment
A Systematic Approach to Identification and Assessment
You should begin identification and assessment by taking inventory and documenting all applications and infrastructure assets.
Within each application, identify any redundant or poorly designed code. Determine what activities or investments are required to bring each application up to corporate standards. Additionally, determine the costs and risks associated with not bringing each application up to standard.
Include both hard (direct) and soft (indirect) dollar costs of updating–and failing to update–each asset. This thorough assessment will give you the information you need to choose how to prioritize and manage your technical debt.
Reducing Technical Debt
If you had infinite resources, you could easily reduce or eliminate your technical debt. Your company doesn’t have infinite resources, so you need a strategic plan to reduce your technical debt within the limits of your resources.
A complete inventory of current technical debt will enable you to create a prioritized list of technical debt you need to reduce. This prioritization should align with overall business goals (including risk reduction plans).
Paying down technical debt reduces risks associated with unplanned outages or other adverse outcomes. Other factors in prioritization include the amount of debt repayment, and where the debt is in the application or infrastructure lifecycle.
Funding ongoing technical debt reduction should be a standard budget item. You may also be able to find additional funding from other departments or projects with a budget surplus. This can help with paying off large and infrequent sources of technical debt.
Educate product owners and others who may oversee processes that are prone to the accrual of technical debt. When decision-makers know about technical debt and its associated risks and liabilities, they can facilitate practices that minimize debt accrual and prioritize debt elimination strategically.
Avoiding Future Technical Debt
As your company moves forward, you should establish a practice of technical debt management. Sometimes technical debt is unavoidable. But you can plan for this type of technical debt so that you minimize its negative impacts. All new projects should include details about any technical debt you are incurring and its associated hard and soft costs.
Reckless technical debt and other forms of debt can and should be avoided. You can do this by regularly inventorying applications and other other assets for technical debt and eliminating it.
Your company’s budget should allocate funding for technical debt elimination and management. You should inform stakeholders about the business benefits of this budget item. When stakeholders understand the liability of technical debt, the company can proactively work to minimize future technical work.
If you need help assessing and mitigating your current technical debt, AVI can help.