programming and human factors
by Jeff Atwood
Every software project I've ever worked on has accrued technical debt over time:
Technical Debt is a wonderful metaphor developed by Ward Cunningham to help us think about this problem. In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.The metaphor also explains why it may be sensible to do the quick and dirty approach. Just as a business incurs some debt to take advantage of a market opportunity developers may incur technical debt to hit an important deadline. The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments.
No matter how talented and smart the software developers, all these tiny deferments begin to add up and cumulatively weigh on the project, dragging it down. My latest project is no different. After six solid months working on the Stack Overflow codebase, this is exactly where we are. We're digging in our heels and retrenching for a major refactoring of our database. We have to stop working on new features for a while and pay down some of our technical debt.
I believe that accruing technical debt is unavoidable on any real software project. Sure, you refactor as you go, and incorporate improvements when you can -- but it's impossible to predict exactly how those key decisions you made early on in the project are going to play out. All you can do is roll with the punches, and budget some time into the schedule to periodically pay down your technical debt.
The time you take out of the schedule to make technical debt payments typically doesn't result in anything the customers or users will see. This can sometimes be hard to justify. In fact, I had to defend our decision with Joel, my business partner. He'd prefer we work on some crazy thing he calls revenue generation, whatever that is.
Steve McConnell has a lengthy blog entry examining technical debt. The perils of not ackowledging your debt are clear:
One of the important implications of technical debt is that it must be serviced, i.e., once you incur a debt there will be interest charges. If the debt grows large enough, eventually the company will spend more on servicing its debt than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (i.e., "servicing the debt") that there is little time left over to add new capabilities to the system. With financial debt, analysts talk about the "debt ratio," which is equal to total debt divided by total assets. Higher debt ratios are seen as more risky, which seems true for technical debt, too.
Beyond what Steve describes here, I'd also argue that accumulated technical debt becomes a major disincentive to work on a project. It's a collection of small but annoying things that you have to deal with every time you sit down to write code. But it's exactly these small annoyances, this sand grinding away in the gears of your workday, that eventually causes you to stop enjoying the project. These small things matter.
It can be scary to go in and rebuild a lot of working code that has become crufty over time. But don't succumb to fear.
I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain.
When it comes time to pay down your technical debt, don't be afraid to break stuff. It's liberating, even energizing to tear down code in order to build it up stronger and better than it was before. Be brave, and realize that paying your technical debt every so often is a normal, necessary part of the software development cycle to avert massive interest payments later. After all, who wants to live forever?
ra ra ra ..
oh wait .. it would have been more interesting to get insight into refactoring SO rather than just handwaving. What are some lessons learnt?
thx
frist on February 27, 2009 5:07 AMSo what architectural design ideas would help to minimize the accrual of technical debt? A modular, add future functionality like snapping on Lego blocks approach?
Sea Cat on February 27, 2009 5:17 AMI feel the technical debt painfully and I will rewrite a large module of a survey site that I'm working on soon. However I will have to prepare a good speech for the project manager and other managers...
WIsh me luck.
Andrei Rinea on February 27, 2009 5:20 AMI mean things like HTML response construction in code-behind by string concatenations and other coding horrors...
Andrei Rinea on February 27, 2009 5:20 AMWhen I have unit tests, I don't fear. I am pretty confident that if i broke something, I'll discover it automatically. When I work with a system that does not have unit tests, I do fear. I'm afraid to make things better because I may have inadvertently broke something else.
tieTYT on February 27, 2009 5:22 AMThese are the times when having a large battery of unit tests can be immensely useful. I'm just sayin'. :P
Kevin Dente on February 27, 2009 5:23 AMI agree that sometimes it make sense to accumulate technical debt in order to meet some important deadline.
However debt accumulation shouldn't last longer than couple of month.
With time developers forget how code functions and what business logic
that code implements.
It very hard to refactor old code, so repaying technical debt later in
time is significantly more expensive than repaying it immediately after
deadline is met.
I was always wary of fixing things back before I used source control. Now, I plow right in without thinking twice.
It took me years to realize how fundamentally and profoundly that one small change had unconsciously changed my behavior.
Dave Ward on February 27, 2009 5:38 AMI'd also argue that accumulated technical debt becomes a major disincentive to work on a project
I have to totally agree with you on this one. I'm a Computer Science master student, and worked with 2 fellow students on a pretty big project in the Netherlands. We only had our university knowledge, and at that moment thought we made a very good (object oriented) MVC implementation that would surely be flexible enough for later changes.
After a year, and a lot of more experience for myself and my fellow students, the request for changes came. I had to look through all the code we had written to see where the changes should be made, and I was truly disgusted with our own code. To finish before our deadline we wrote a lot of code that was not very good, but it worked and everybody was happy. But now I had to make some changes I could not even find out for myself where to start. I got so frustrated about our own work that I eventually turned down the job, even when it paid a lot of money (for a student that is).
As a side note I have to add, that since that project, me and my fellow students have had a lot more experience (real world experience and not university experience) and that our latest projects are much better than that one. So one question remains: wtf have I paid my college tuition for :)
Resd on February 27, 2009 5:38 AMJust tell Joel you'll have the revenue generation going in six to eight weeks....
;-)
El on February 27, 2009 6:06 AMYou reap what you sow. You have been planting crufty code and avoiding writing unit tests. You have even be bragging about doing it on your podcasts and everyone has been laughing at you.
Funny how it comes around and bites you in the ass a few months later!
Paul Keeble on February 27, 2009 7:04 AMNice Dune quote.
Peter Schwer on February 27, 2009 7:13 AMTechnical debt is an Abomination! I have spilled much water over deferments made early on in project.
JamToday on February 27, 2009 7:36 AMI think version 1.0 of my programs is like a brute force attempt to get things to work, then 2.0 comes and it a little bit less brute force and a little more elegance.
Lucas McCoy on February 27, 2009 7:50 AMYou reap what you sow.
Unit testing, in this context, is irrelevant; you WILL make imperfect architectural decisions early on.
Technical debt accumulates on every project, because we have not yet managed to invent the mind reading, prescient programmer.
Now you could argue that having a lot of unit tests makes it easier to make sweeping architectural changes. That's true. But I also suspect if your changes are deep enough, you're going to break some tests, too -- factor that time in as well.
Jeff Atwood on February 27, 2009 7:54 AMBut I also suspect if your changes are deep enough, you're going to break some tests, too -- factor that time in as well.
Right. It is better to let your users discover what you broke than to find out by tests going red, great point.
omg on February 27, 2009 9:01 AMI love you, Jeff.
Marcus on February 27, 2009 10:31 AM20
Jerko on February 27, 2009 11:49 AMThis is an interesting concept, Jeff. Going back to rewrite stuff
due to architecture is usually a bad sign (doing it because of changing
customer needs is different, or put another way, better understanding of
customer needs). I find most of the time people have to go back and
fix stuff is because of performance, and a lot of the time that's
because they used the wrong tools in an effort to deliver functionality
faster.
Wow. What a truly awesome take on things. This is the best way I've heard it explained. This post not only notes the benefits of well defined code for a large multi-coder project, but justifies the use of quick, dirty code for small projects or one-time-run programs. You reap what you sow on a large project, but on a small project you usually don't make it to the reaping part more than once.
Great article.
Nicholas on February 28, 2009 1:37 AMThis is an interesting topic! I think everyone working for some time as a developer has had to (or ought to) pay off technical debt many times.
@Jeff,
Since Stack Overflow is a relatively open site and many of your readers
are somewhat familiar with it, this seems like a great opportunity for a
series of articles about this in more detail
How did Stack Overflow handle this particular down payment.
What symptoms did you notice to make you want to do it, what did you
conclude it was caused by, and how did you decide to address it?
What was the design like before and after, how did you practically carry it out to avoid downtime, and what was the result?
What you guys have done with Stack Overflow is inspiring and impressive, I'm sure you have material for a whole book by now. Dreaming in code 2 - Stack overflow I'll pre-order! ;-)
Console on February 28, 2009 1:50 AMExcellent post!
On my last major project (18 months, 8 developers) I tried to convince various influential people about the importance, relevance and inevitability of technical debt but kept being told: refactor as you go.
In practice, refactoring as you go is NOT sufficient to change major early decisions. And, even if it was, by constantly refactoring, you're never able to benefit from accruing SOME technical debt. It is a perfect metaphor as far as I can tell.
In the end the dev team forced through major changes to the database and core entity model. This broke just about everything so we had to spend time making the system work again instead of implementing new features. Subsequent releases however almost certainly arrived quicker and more feature rich as a result of vastly improved productivity brought about by these major changes. We discussed the situation at great length and nobody could see how such changes could have been achieved in an incremental, refactor as you go, way. The impact on developer morale should not be underestimated either - the entire team was invigorated by this event.
With a large database, even small structural changes can be difficult to implement (think about complex data migration scripts). But technical debt accrues in the database design just as much as in the code so can't be ignored.
We did not do much unit testing and while I agree they can help with the refactoring as you go approach, they would not have helped us during this major change. Well over 80% of any unit tests would have needed to be substantially rewritten. In fact, looking at the 18 month project as a whole, I believe having a comprehensive unit test suite would have _substantially_ reduced overall productivity without any significant (and entirely unnecessary) increase in quality.
I wonder if any controlled tests have been conducted to compare two equivalently skilled teams making identical changes to a codebase where one uses unit tests and the other doesn't. It would be interesting to see how the productivity/quality metrics compare between these teams as they are asked to make different kinds of changes and where different amounts of technical debt pre-exist.
Daniel on February 28, 2009 2:47 AMVery good post, I've always known about technical debt I just never knew a word for it so thnx for the word.
Is this technical debt caused by a rush to get features out? I think SO is very good which is the irony that maybe you must create this debt to produce something good out of the code? I would hope not but it seems that way sometimes.
Keep all debts manageable.
This is why I'm proposing a new methodology called Revenue Driven Development. What you do is you figure out the simple possible functionality you can charge money for, you write a test to prove out that functionality, you write the code to perform that function, you release (either internally or externally), you move on to the next simplest possible piece of functionality you can charge money for.
There are supporting practices involved... but they all work just like XP. The difference: under Revenue Driven Development they all have monikers that make marketing and financing people drool.
This way you figure out how to make more than the minimum monthly payment on your technical debt before you max out your credibility card. Stopping all new development on your project to do something other than fix actual reported bugs is like taking a second mortgage on your house to pay your credit card bill off. You still have the debt, perhaps on better terms, but if you don't learn to prevent the debt from building up in the first place... you're going to be in even more trouble later.
Michael on February 28, 2009 3:35 AMAs a counterpoint to those who have espoused the ideal of do it right the first time, I still assert that Technical Debt accrues. Languages improve; hardware improves; operating systems improve. Space-Time tradeoffs change. What may have been the provably best solution in 1992 is likely to be sub-optimal given current theory and technology.
For a simple case, suppose a sort algorithm were discovered that ran in O(n loglog n) time. What would it take for you to rewrite everything to take advantage of this theoretical improvement? Would it be worth it?
MSchmahl on February 28, 2009 3:44 AMGot technical debt? Can't pay your software-quality mortgage?
Call Washington NOW for YOUR bailout!
Billions of LOC available!
1-800-CONGRESS
ah yes, but Fowler and Cunningham want you to realise the technical debt when it's staring you in the face, before you've made the decision to suck. e.g. quick but messy versus clean but longer implementation.
The idea is that you're not doing anyone a favor by cheating with the quick and messy, and the analogy is using your credit card versus paying cash. Use the card, you pay interest in addition to the effort, and the net outcome is paying more. You think you're being clever, but you're not. Pay with cash and you don't have to pay again. You're done.
There's also a peculiar notion expressed by some of you that seems to feel that refactoring is only to do with small code changes. Refactoring can be anything. It just means spending the time to make changes that are state-preserving. You change focus from adding features to cleaning code. Refactoring your architecture is still refactoring.
Phil on February 28, 2009 3:54 AMGreat post Jeff. +1 :)
Given the nature of some of the comments here, one could be forgiven for thinking that some of the commenters were forced at gunpoint to read and recite this post whilst their eyelids were stapled open, until it was burned into their psyches.
This experience being in stark contrast to their otherwise utopian existences, where stakeholders and investors do give a shit about unit testing and quality metrics, and never threatened to offshore a project because they just want it to market, like yesterday.
For a great many of us, getting to market fast, cheap and good enough is the driving factor, and what puts food on the table. We will make some bad decisions along the way, if the project is lucky enough to still be running in 12 months time, they may be fixed, and fixing them will quite possibly break some other shit, there was no budget for unit tests, because the investors couldn't see them as a valid deliverable.
Ideal? Hell No. Would we prefer that to be different? Hell Yeah.
But for a lot of developers that is the grim commercial reality of life, if you want unit tests, you can write em on your weekends, but on weekdays, investors want to see dem blinkenlights right now, and they want their new, pretty, and probably imperfect blinkenlights on the interwhatsit before their competitors.
It becomes even more seat of the pants when you put your own assetts on the line for a startup, as I dare say Jeff did to some extent. In that case, willing to take technical debt in return for cashflow, pagerank and less time to market anyday.
If it floats, you may have a chance to fix it, if it sinks, move along. Unless you make an absolute dog of a product, market factors and bling on your UI will be much more a determinant of your initial success than software quality. If it does make money, and no one really notices the imperfections, they may never get fixed, because it's probably not a weapon aiming system.
For those who don't exist in this sometimes dystopian place for lesser mortals, that seemingly offends you so, why are you here?
You're just increasing traffic and ranking for this blog... it's an imperfect blog, but maybe read more than yours.
seanb on February 28, 2009 3:58 AMI feel as you do, Jeff, but I have no problem believing that Joel balked at this.
On the flip side of what you feel is this article by Joel: http://www.joelonsoftware.com/articles/fog0000000069.html -- that's the one titled Things You Should Never Do, Part I about how stupid Netscape was to rewrite their browser.
Nothing is cut and dried, but how scary is it to tear down code that you know _works_ and replace it with all new code? Do you think you might get into trouble at some point in your refactoring?
Matt on February 28, 2009 3:59 AMTechnical debt accumulates on every project, because we have not yet managed to invent the mind reading, prescient programmer.
Don't you mean architect/designer? At least in the case of Stack Overflow I fear you are correct!
fred on February 28, 2009 4:17 AM@Simon - In reality though if the performance gain from the refactoring allowed the website to run efficiently on the current hardware for longer without requiring upgrades as the website grew or if it prevented hitting a wall in later development where more major changes would need to be made it is enitely worth it.
I think it is best to make these changes early, when there identified
rather than waiting for them to become a major performance issue.
I think it's excellent that you're recognizing the technical debt and taking time to pay it down. However, the level beyond where you are at is where you continuously pay down technical debt. Once you reach that level, having to separately schedule it is a sign you've let too much pile up.
Also, unit tests are a big help with managing technical debt. Fear is one of the biggest factors keeping the teams I work with from changing things they know they should change. If you are afraid of breaking something, that means you don't trust your tests to catch you.
William Pietri on February 28, 2009 5:10 AM@kbbartum If Atwood bores you, and you don't have any constructive criticism of his writing, why are you wasting your time commenting? Just don't read his blog any more.
@Atwood Fantastic article. This is something that's relevant to most developers I would expect, but is particularly relevant to me right now as I'm making the first prototype for my first proper software project. Yikes, it's scary to be making a mess. But I'm making a mess knowing that this will not be the actual project. And when I do get going with the actual codebase, I'll know that technical debt is something to treat as I would interest. It's a great analogy.
One more thing - you included a link to a quote from Flash Gordon on YouTube, therefore you rule. I can only aspire to such greatness.
John Gallagher on February 28, 2009 5:54 AMThis post pretty much boils down to we have to refactor/rewrite some stuff on our software project - Big deal, so do I; and probably so does the poster above me, and the one above him.
Details Jeff, is what makes a technical article... technical. Otherwise this could have been written by a 2nd year business major.
Steve-O on February 28, 2009 6:19 AMRespect is due for having built such a useful site as Stack Overflow. (And for having stuck the boot into the much hated Experts Exchange.)
As other commenters have suggested on this post and others, Joel and your position on automatic tests seems a little extreme though. Sure, tests do have a cost. But as a code base matures, interfaces (in the loose sense) to larger units should tend to stabilise. (Unless there is no architecture at all - ie. big ball of mud.) So the ongoing cost of testing at least some of the code should go down, and change the cost-benefit equation. Even if you don't buy into the 100% test-infected stuff, I think you are probably missing out by rejecting automatic testing so absolutely.
Paul Delhanty on February 28, 2009 7:04 AMWay to rip off Martin Fowlers last blog post. At least you added some pretty pictures.
Justin Chase on February 28, 2009 7:47 AMMost of the time, you just need to get the guts to make the hard decision than to fear the troubles and tolerate. Everything else will just follow. Though some will be tough, you are going to make it, because it is an equally hard decision to give up in the middle once you start, especially when the action you are taking now is the result of a prior taking-guts-to-make decision.
Ling on February 28, 2009 9:26 AMIt seems to me that most of the people criticizing the metaphor or Jeff are only looking at the idea from a very narrow perspective.
To pick on just one, the idea that most refactoring would be a waste of time based on 10% being the average amount of technical debt on a project seems ridiculous to me. Even if 10% were the average, it is representative of such a small slice of the projects going on out there that it proves nothing about whether or not refactoring in general is worth while.
In the real world on projects of all sizes, technical debt can be accrued for so many reasons outside of developer control, from customer pressure for design changes to hardware and environment changes that the technical debt can fluctuate dramatically within a single day. Furthermore, it's impossible to predict *all* of these changes and account for them in the architecture from day one or you would be wasting an incredible amount of time not having a working site/application that is bringing in no money.
The biggest take away from the article should not be whether you think you are smart enough or more likely lucky enough that you haven't run into the issue on your projects, but to be aware that it does happen on a large number of real world projects. I'm not saying that getting into too much technical debt is an everyday occurrence or that it always happens fast or that it is never the developers' fault. Whatever the reason, speed of accrual or where the blame lies, it does happen, you should be aware that it happens, and you should be critical enough to recognize when paying down the debt makes sense.
Great article btw =)
Chuck on February 28, 2009 9:38 AMI have a severe back problem that, for almost a year, has left me unemployed except for a little paid work programming. Programming, which I had previously dabbled with as a hobby, has become my new career out of necessity, but mainly love for the craft.
Since I can't go to a University, I am learning all I can from the internet. I do not have the advantage of the give and take discussions with Professors and fellow students.
I wrote all of tht to say this: You and Steve, among others, provide me with invaluable insights into the world of software development. And the comments are also very helpful.
Your battles with the Coding Horrors of programming encourage me since I face the same battles, although on a much smaller scale.
Criticism can be helpful but sometimes it goes too far. To your unreasonably harsh critics, I recommend they try taking on all the burden of responsibilities and work that you do *and* write quality articles about programming 3 or 4 times a week. Let us see how well they do it! When they do all of that and get as many hits on their blog as you do, then they may see things a little differently.
Keep up the good work, Jeff.
Re - changing number of arguments to a function...
(1) The eclipse refactoring for Java change method signature will rewrite all the callers to the function for you, supplying a default value, so that's not a lot a work.
(2) if you have more than (approximately) 10 tests/test-setups calling a particular method, you have too much duplication in your tests and/or the function being tested is too large.
Use mock objects, stubs, fakes, etc., to decouple classes so that a single test only tests one interesting bit of behavior of one method of one class, not testing the whole pyramid of behaviors invoked by a high-level method. You still want _some_ integration and acceptance tests, but not everything has to be tested that way.
With test-driven development, refactoring, and paying attention to code smells [like Large Class, Large Method, Duplicate Logic] and test smells [like Expensive Setup, Duplicate Logic, Large Test], you don't have the problem of one change making lots of tests fail.
Really.
keithray on February 28, 2009 11:50 AMThere is also the technical tax that accumulates when your projects rely on major components and underlying platforms you have no control over.
Vendors usually try to maintain backward compatibility, at least over a short sequence of versions, but changes can creep up. At some point if you skip major releases you can find yourself in unsupported territory and face a major rework that doesn't add any value in itself.
Yet paying this tax as you go can be a drag on resources too. Of course the alternative means letting the taxes accumulate and compound.
This is less of an issue for a product with a 5 year lifespan than one that must live 10 years or more though.
Homer on February 28, 2009 12:03 PMI tend to agree with this metaphore, but it raises a big issue:
The developper have a company check book out of control....
Make this metaphore widely adopted would create a strong movement
coming from finance to control the use of this hidden checkbook. Lot of headache in perspective ....
In the current time, knowing that some group of people can create debt out of control taking advantage that what they do is too technical to be monitored will trigger very painfull memories that are very fresh.
In addition, finding some else to blame, can be very appealing for those sitting on the grill today :)
any idea how to explain to the company that what they expect costs much more than they know ? And how to explain we need them to still fund it because we need money to live :)
or even worst, telling the truth:
We have no idea how much it will cost utlimatly, but for the fragmented
data we have the purposal you have in hand is the most appealing that we
can write.
Of course we can intend to collect enough data to have a more precise
idea of the real cost without technical debt, but that study alone will
likely cost more than the whole project that you have in mind....
so let's be reasonable, let's admit that organizations are too complex and irrational to know what we do. Going agile is not so bad in that perspective, we throw away money in small chunk and add more if we get a return that makes us happy...
Paul, have you even seen the statistics for the site? 500k page views per day. Millions of unique visitors per month. Yeah, sure, everybody is laughing at the lack of unit tests.
steve on February 28, 2009 12:13 PMSorry, have I wondered into YouTube by mistake? I'm looking at some of the comments here, and I can barely tell the difference. I guess you've reached some critical readership mass that tipped you over some kind of wanker-attraction threshold... 'tis the price of useful feedback, I guess, but it's most unwelcome.
PS: Fake! That picture is faked! Real credit cards don't look blurry at the edges like that, it's obviously photo-shopped. I speak from authority because I once opened paint and looked at the toolbar, and my IQ is more than 3 whole points above retarded. Maybe the CIA did it?
Schmoo on February 28, 2009 12:22 PMSee this story from the early mac days:
http://www.folklore.org/StoryView.py?project=Macintoshstory=Mea_Culpa.txt
They used the upper 8 bits of 32, because in the early days only 24 were used. That was a big burden, when the 68020 got reality ...
cwis on February 28, 2009 12:33 PMA couple of interesting posts linked to here, but I feel it would have been more valuable if you could have added some of your own experience to them.
For example, with reference to Steve's article, what sort of debt did StackOverflow aquire? Type I.A.1 or I.A.2? This would be interesting to know as some of your recent posts might suggest you were going for the bad sort of debt - lack of comments, generic variable names, not following coding conventions. The linked article suggests this is never a good strategy, even in the short term.
By the way, I wish you had credited Martin Fowler for the first quote.
Steve W on March 1, 2009 3:50 AMHey,
I love you Jeff, and all, but this continued aversion to testing makes you sound like an idiot.
Testing massively reduces the fear associated with refactoring. Yes, of course you break tests while making changes, and yes you have to fix them. But that is part of the process of finding out and compensating for the effects your changes have had across the system. The alternative - to cover your eyes and hope for the best - is hopeless. You have a responsibility not to lead people so far astray like this.
Big hugs though, I really do love you and your stuff,
Jonathan
Just now I read an article about how FriendFeed use a weird schema
to make their MySQL more responsive. And it looks like you have database
scaling issues as well. Perhaps it will give you ideas on how to handle
it :
http://bret.appspot.com/entry/how-friendfeed-uses-mysql
Even though personally I think they should not have chosen to use MySQL
in the first place, something more robust and scalable like PostgreSQL
would probably not lead them to the problems they are facing. But afaik
you have the M$ DB, so it might not be relevant for you at all.
To all the folks who say stuff like in the real world, Jeff is 100% spot on and anyone who says differently is dreaming... This is either an ad hominem attack or an inept appeal to authority. Jeff is making some pretty vague claims here about both the scope and necessity of an impending refactoring, and deserves to be questioned. Personally I think it sounds like he is promoting having a permanent version 1.0 by taking time off to refactor major parts of his software. I have never seen that work well.
Michael on March 1, 2009 5:40 AMI'd also argue that accumulated technical debt becomes a major disincentive to work on a project
This is more important than it is usually given credit for. Good programmers dislike dirty code. They want to get rid of it, they don't want to be around it and they certainly don't want to be associated with it.
The point is that a technical debt is not just an undocumented hole in the budget, it's an undocumented HR problem. It lowers moral and raises staff turnover.
If your project has a high level of technical debt it makes everyone who works on it look inexplicably inefficient and weak. People procrastinate rather than touch such painful code, they try to avoid the project, their more tempted to make quick and dirty fixes just to get away from it faster and in the worst case they'll leave the company to leave the mess behind. Likewise attracting and retaining quality new staff is going to be hampered by having a high technical debt.
The point is to look at technical debt as both a financial burden and an HR burden. Programmers working in a clean, powerful, elegant development environment are more motivated, more interested in producing more code of a similar standard and less interested in leaving the company.
James on March 1, 2009 6:49 AMInteresting, seems like you would benefit from SOLID in your designs ... ;)
Thomas Hansen on March 1, 2009 9:48 AMMany don't understand that sometimes, code needs to be rewritten
partially or almost completely to be able to go further. It's a shame,
but that's life, but don't blame me, blame my boss for his incompetence!
The idea of technical debt is valid and the other assertions of, Michael on February 28, making business people drool is also important. Enabling software development to be taken seriously by business is important because the majority see it as a cost. Whilst I hope to not indulge in jargon bingo, development needs to create its own that business people can understand and cost from both a productive and efficiency led perspective.
Living in the dark and hoping to achieve validation by not examining your own profession is tantamount to crazy. Technical debt is one in a series of phrases and measurements software development needs hold itself accountable to because business cannot understand its value.
As far as the sniping that has occurred I believe that Jeff has raised an area of debate that can be used to initiate a debate to bring together IT and Business to a common goal; good maintainable software that is inexpensive to develop and easier to maintain. This is balanced with an understanding of the process of development and the degeneration of the product itself.
M Princz on March 1, 2009 11:08 AMI'm with Jeff and Joel on unit testing. I would rather have functional tests to make sure all the end user functionality works.
Have façade interfaces that expose all the end user functionality to the UI (ie. the UI is very thin wrapper around the façade). Then write test against those interfaces.
Anonymous on March 2, 2009 4:08 AMGreat post. I work for a Fortune 100 company. We face technical debt in two ways. The first, we have a large legacy base (all that COBOL stuff the business really runs on). Over 30 years additions, changes, and hardcoding have been added to solve problems but rarely has anyone refactored. A lot of this is because its hard to refactor COBOL into anything but long complex COBOL programs with confusing global variables. Today we have less than 5 or 6 people who would even be qualified to touch the code, everyone else is scared. We've ignored it so long the people best qualified to understand it have retired (or died) and the people left are too busy keeping it working. We are ultimately facing bankruptcy on the code and start over with a new system.
The second is on new projects that are run by non-technical MS Project leaders. They want to stick to the project plan and meet their timeline. This is admirable and I agree they are necessary to make sure we actually deliver on time. But combine this with a large outsourced Indian development work for the focus is on make it work now and ignore the future. Once a feature is written which does basically what it should do and with reasonable performance its marked done and not touched again.
I deal with both legacy and new code. My mantra is don't be afraid to rewrite code. Its scary to do so, but the benefits are huge. Even if you don't do it any better, you at least didn't blindly leave it as a mess and will have a better understanding. I always try to have at least one developer on the team that does code maintenance, they understand the pain later by not being willing to rewrite code.
Alex on March 2, 2009 4:42 AMI both subscribed martinfowler's and your blog. and most of us did so.
ding jiang hao on March 2, 2009 5:38 AMI enjoyed this post (too) :-)
Thanks
@Lept Wikipedia is your friend:
http://en.wikipedia.org/wiki/Refactoring
I personally find metaphors very useful. Not only do they explain a concept quickly (by reusing past experience) but they can also help you pass the things you learn in life across all disciplines.
Just as the lessons you learn in everyday life can be applied to developing software, the lessons you learn in developing software can be applied to everyday life.
This time I *will* keep my filling knife clean...
http://www.mattburns.co.uk/blog/2009/03/02/keep-your-filling-knife-clean/
Matt Burns on March 3, 2009 4:06 AMJeff -
What exactly do you mean by refactoring? That is a buzzword I see on a lot of PP charts. If you mean rewriting code to embrace new architectures, etc. (multi-core, openmp, etc), please say so. If you mean re-designing projects so they will score less on the stinkometer, please say that too.
(I'm chuckling quietly to myself, but I'm wondering:)how do I resolve the focus on refactoring with your previous suggestion about avoiding re-inventing the wheel?
Lept on March 3, 2009 5:17 AMI've not read more than just the first book, but my favourite Dune litany rip-off has always been the one based on the Mentats:
It is by caffeine alone I set my mind in motion.
It is by the Beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.
Technical debt accumulates on every project, because we have not yet managed to invent the mind reading, prescient programmer.
This is no longer strictly true, because after my spice-binge last weekend, I am the...
Kwisatz Haderach on March 3, 2009 9:14 AMLike the real-estate meltdown I forsee a software meltdown.
There will come a time when this stuff becomes so complex that no single
or probably dozen human minds will be able to comprehend it.
That will probably be the time when the machines decide to refactor us...
What does this have to do with technical debt? There is a hidden and very real cost in complexity. I'm not at all sure that I am happier today than when I was a boy.
Sure, we did not have video games, computers, and all the gadgets and gizmos of today. However; I do believe that I was happier then, the world was less complex and less daunting.
I see kids today going from zombies training for a future war in front of an XBox to totally overloaded in certain school situations to sometimes outright disaster with things like facebook.
I've worked in this industry for the past 30 years and most of the time it's been relatively kind to me.
So, what can I do without?
Cell phones: don't need it, don't want it.
GPS: I can get around just fine thank you very much.
DTV: People watch too much TV anyway. I might just go dark on June 12th.
Home Computer: This probably only applies to me; I work with one all day long, why do I want to see it in my house?
Internet: This is blasphemy I know but; I got by just fine before it
became all the rage, I make money on it now... Has it really changed my
life for the better? Yes because I work on it but, if I was not in this
industry would I really need it? Probably not.
What do I not want to do without:
Medical Machinery breakthroughs such as MRI and CT scans, dialysis
machines, pace makers, heart monitors, etc. Life saving applications are
among the greatest applications for technology.
Electronic Ignition: The best thing that ever happend to the internal combustion engine.
Automated alarm systems: Unfortunately, we still have crooks in the world.
And the ONE THING I HATE THE MOST:
ELECTRONIC VOTING: If we thought things got screwed up with the paper
ballot mess down in Fla. electrons take it to a whole new level...
Amazing...I just finished reading Dune for the first time last week. Nice tie-in :)
Joe on March 3, 2009 10:53 AMOk, where is the print view of this blog post? My boss urgently needs to see this before he comes in and asks for another new feature when I'm busy fixing spaghetti.
imissmyjuno on March 4, 2009 11:32 AMAhem!
I will use stack overflow. I will read joelonsoftware. I will read
codinghorror. If I find something nice, I will take it and use it. But
If I don't agree, I'll make sure that the whole wide world knows about
my strong opinion about what I think about Jeff's shortcomings.
I, ladies and gentlemen, am Master Troll! I rule the blogosphere, I am
king, Lord and Master of the Dark Side. I will never allow anything to
go unnoticed if I don't like it. I have the God given right to insult,
attack and defame anyone in the blogosphere. I am The Law. I will bite
and chew off every little nit in this article. I will spread doubt, fear
and poison in the minds of all the readers. I am the One they should
listen to, or face my words of wrath. There shall be no fame or respect
without my permission,
I am the Lord of Trolls!
----------------
How does that sound? Impressive? Or Insane?
----------------
You reap what you sow. You have been planting crufty code and avoiding
writing unit tests. You have even be bragging about doing it on your
podcasts and everyone has been laughing at you.
Funny how it comes around and bites you in the ass a few months later!
Paul Keeble on February 27, 2009 07:04 PM
----------------
Atwood == *YAWN*.
Talk about flogging. You post, and you post, and you post some more, and
it quickly becomes apparent that 1) You talk too much (especially about
yourself), and 2) You're really not very good at this whole 'computer
science' thing.
You -are- pretty good at the self-promotion, though.
KB Bartum on February 28, 2009 04:33 AM
----------------
@ John Gallagher
... because someone should call Atwood out for what he is -- a shill,
selling himself, effectively regurgitating more intelligent individual's
work (eg, in this case, Ward Cunningham Martin Fowler).
He's polluting the technical space with poorly thought out -- if not
just plain wrong -- ideas, and nearly anything good he publishes, he got
somewhere else.
KB Bartum on February 28, 2009 01:11 PM
----------------
I rate him as being a publicity hound rather than a pioneer, and I tend
to assume that people who think he's got something useful to say aren't
familiar with the original sources he strip-mined.
KB Bartum on February 28, 2009 01:20 PM
----------------
What do you get from this display of small mindedness? Is it jealousy? Does Jeff order you around? Plot things in your life? Try to get you fired? Or try to steal your customers and contracts?
Anyone stopping you from starting a blog of your own?
If you're traced anywhere on the blogosphere in the future, you are
going to be remembered by lots of people. People remember others even if
they genuinely protest, having a decent case to present.
And you here, are trying this! Whatever makes you think that this will go unnoticed or be forgotten? Do you have any idea of how many multiple copies of this content exist on the expanded web (public web + private docs on net-connected PCs) ?
One announcement by Joel and Jeff to ask readers to volunteer to point out routine trolls by identifying their posting pattern will finish the whole race of trolls. That they don't exercise that option is a quality of theirs, not a fear of you.
In the case of less scrupled community leaders, even a genuine protest brings a much harder response. And you think that Jeff cannot ask users to finish the entire race of you trolls?
Deleting your posts is NOT the painful thing. Keeping them marked as
Sick F**king Troll for permanent public display is much worse. If you've
posted under influence, I'm sorry, as much as you would be. But if not,
then I think you might need to change your online nickname.
Trolling is counterproductive and cruel. Give it a peaceful thought.
Evgeny,
FriendFeed's problems are just weird.
http://bret.appspot.com/entry/how-friendfeed-uses-mysql says, In particular, making schema changes or adding indexes to a database with more than 10 - 20 million rows completely locks the database for hours at a time.
My jaw just dropped. 10-20 million rows is SMALL. Any database smaller than 50 gigs is small these days.
They want advice on how to deal with performance issues, then they should probably contact a consulting company who understands high-performance MySQL. For instance, Proven Scaling LLC. http://provenscaling.com/
John Z-Bo Zabroski on March 7, 2009 4:06 AMAlex,
@Great post. I work for a Fortune 100 company. We face technical debt in two ways. The first, we have a large legacy base (all that COBOL stuff the business really runs on). Over 30 years additions, changes, and hardcoding have been added to solve problems but rarely has anyone refactored. A lot of this is because its hard to refactor COBOL into anything but long complex COBOL programs with confusing global variables.
Sorry but this isn't true. COBOL refactoring tools do exist. Netron Fusion is an example. It won't solve all your problems and may not even realistically be worth it for you, but there is a book that talks about properly factored COBOL code (if such a thing exists!): Framing Software Reuse by Paul G. Bassett.
John Z-Bo Zabroski on March 7, 2009 4:13 AMThere's a fine line between not incurring technical debt (i.e. coding until the job is done properly), and gold plating.
Andrew Stuart on March 8, 2009 1:10 AMLove the Dune quote!
Mitch Wheat on March 8, 2009 8:46 AMPost the DB schema, and write up the changes.
anon on February 6, 2010 11:12 PMBreaking tests doesn't have to mean your code has a bug.
It could just be for example that function A now takes 3 parameters instead of 2. All those units tests will then need to be modified to take that into account.
Leon Sodhi on February 6, 2010 11:12 PMLet me start by saying I know nothing about your project. However, let's take the technical debt analogy and run with it.
How much are the interest repayments on that technical debt? 25%, 15%%, 10%, 5%, 2%, 1%?
However, for most projects I think that 10% is typical for most projects. Let's say you do two weeks (10 working days) of refactoring to get the technical debt interest rate down to 1%.
Before the refactoring you were working at 90% effectiveness and after the refactoring you're operating 99% effectiveness. A straightforward calculation shows that it would take 124 working days (or 29 weeks) to be ahead where you would have been if you hadn't have refactored at all.
This is why your business partner wasn't convinced. If you'd spent that two weeks adding or improving pre-existing features, it's obviously going to add a lot more value to the project than the time spent refactoring.
I find that the technical debt analogy actually proves the opposite of what the proponents of term what to demonstrate: that refactoring is typically a really bad use of project time!
Simon Johnson on February 6, 2010 11:12 PMUnit testing, in this context, is irrelevant; you WILL make imperfect architectural decisions early on.
Very true Jeff. Having unit tests does nothing to eliminate technical debt. In fact unit tests themselves, being more code, can have their own technical debt too.
Now you could argue that having a lot of unit tests makes it easier to make sweeping architectural changes.
Yes!!!! And this is the key argument for writing tests. It's not just a side benefit of having them.
For any kind of refactoring - including eliminating technical debt - having unit tests, and as importantly, having functional or full stack tests can be a _huge_ help. More than enough to make it worth having taken the time to write the tests in the first place.
Cameron Booth on February 6, 2010 11:12 PMI call this the software quality mortgage:
http://blog.asmartbear.com/blog/software-quality-mortgage.html
See that link for why IT IS GOOD to have accumulated technical debt, and techniques for convincing non-technical management to allocate time to clean it up later.
Jason Cohen on February 6, 2010 11:12 PMI also like the metaphor. Has anyone tried to measure technical debt (incurring debt and paying it off), and if so, what's your approach? I realize that it's difficult to measure in any accurate way; however, understanding trends would be important. A directional indicator of whether the technical loan is being paid off (or getting worse) would be useful.
Thanks!
Peter Roome on February 26, 2011 1:44 PMThere is a big difference between refactoring your own code and refactoring someone else's code. It is too easy to try to rewrite "legacy" code only because a newcomer doesn't understand it. Our own abstractions are more familiar to us than someone else's.
Daniel E Graham on March 25, 2011 9:59 PMFor a major refactoring effort, such as completely rewriting something in another language - it would appear that it could result in adding technical debt during the refactoring. That's something to consider. Also, the refactoring shouldn't result in less features. But, if a new team can clearly identify bad design decisions or lower level code issues, then incremental refactoring is good. But, to just assume that the new people are "smarter" than the old people and just rewrite everything, is not as good.
Daniel E Graham on March 25, 2011 10:48 PMThe comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
Traffic Stats |