Over-Engineering is tempting. But you ain’t gonna need it!
We developers over-engineer for a good reason. We aim to avoid technical debt.
Technical debt is the implied costs of rework caused by choosing a short-cut solution now instead of using a better approach that would take longer.
Let’s consider the following example. You want to output today’s date as a string with its first and last hour (0 and 23).
Pretty simple. Have a look at the following source code.
Alternatively, we could write the code in an (over-?) engineered way. We would put all things into separate functions.
Which solution is better? The easy way is shorter. The (over-?) engineered way is longer and it creates more
Date-objects during run-time. Is it overkill? Both solutions produce the same output (by the way, today is October 15th, 2019):
today is from 2019–9–15–0 to 2019–9–15–23
Wait! This is wrong. We have October! Not September.
The reason is: the
Date-object starts counting at
0. So, we need to add
1 to it.
In the “engineered way”-code, we can correct the result with a single change:
In the “easy way”-code, we need to correct the error in two places.
This is more work. And changing it in two places is a source of errors. We might easily miss changing it in one place. We get a completely different result. Such as
today is from 2019–10–15–0 to 2019–9–15–23 in case we changed only the first occurrence.
This additional work and this source of errors are our technical debt! It is little. Yes. But it is a little example, too.
Technical debt can be compared to monetary debt. If technical debt is not repaid, it makes changes harder to implement later on. When you change something, you have to deal with the imperfect concepts you codified on the first occasion. The debt keeps increasing over time. It accumulates interest.
If you ignore technical debt long enough, you can go technically bankrupt. This is the case when your code does not allow you changing it anymore without breaking it.
In our example, what if we wanted to output the first and the last hour of our local day in another timezone. Let’s assume we are in New York City. It is 7 a.m. (Eastern Standard Time, -5 to UTC). What is our New York City day in UTC?
We could simply replace the called functions with their
UTC-versions, for instance:
The engineered code does not require many changes.
Due to our technical debt, in the easy-way-code, we need to apply the change at more places than in the engineered example.
But if we look at the result, we see that both outputs differ!
today is from 2019–10–15–0 to 2019–10–15–23 today is from 2019–10–15–5 to 2019–10–16–4
The easy-way-code does not output our local day as it is in UTC. Because it simply takes the date and hard-codes the hours. We need to add more code to make it work. We have to pay interest on our technical debt.
Have a look at the following example! … Do you see where I struggled?
What did I do here? (Besides reintroducing the old bug of having missed adding
+1 to the month… funny fact: This is a real mistake I made. I think I copied the code fourth and back too often…)
Since we are in NYC, I can hard-code the hours of the UTC-day (
4). But depending on the current hour of the day, it may already be tomorrow in UTC! In that case, we add
1 to the date.
This code works for the current day — that is October, 15th. But what if it was October, 31st? We would even need to add another month. What about December 31st? We would even need to add another year.
This is too much for me right now! I don’t have the time… I have to implement my user stories… It is not too bad if the result is wrong for 5 hours at 12 days a year, is it?
Did I just go technically bankrupt?
The technical debt did not only produce more work in the first step. It added up in the second step. It even made me introduce a bug! Even further, the “easy” solution only works in NYC. It does not work in any other timezone.
Too much technical debt will prevent features and bug fixes from shipping in a reasonable amount of time.
If technical debt is that bad, how can we call good engineering, that prevents technical debt, be “over”-engineering?
The problem with code is that you’ll never know which parts you may need to change in the future. You would need to have a magic crystal ball. Both, the requirements and the technology change at a speed that simply does not allow you to foresee the future.
When we first wrote the code example above, how could we have predicted that we want to calculate our local NYC days in another timezone? We couldn’t! At this time, our solution was over-engineered. We were lucky it came in handy.
What if that new requirement wouldn’t have been added? We would have spent quite some time (and thus money) on a solution we did not need.
What if you don’t understand the problem good enough yet? While you’re programming, you are learning. It can take some time before you understand what the best solution is. Sometimes, you first have to provide a proof-of-concept. It might become part of a solution later. It might be a bunch of hooey, as well.
Spending time on something you don’t really need is like gambling. You give some time now for the chance of winning some time back later.
Sometimes, you’ll win. Sometimes, you’ll lose the bet. You end up with code you don’t actually need. But this code is part of your app now. You have to live with it. You need to take care of it. It is another form of technical debt. Because the code is unnecessary for the current purpose of your app. And you don’t know whether it serves a future purpose or not.
No matter what you do, you’ll end up with technical debt. There’s no way we can prevent it. But there are means to control it.
First, there’s a best practice in software development: “You ain’t gonna need it” (YAGNI). This practice says that you should not add anything to your code until you actually need it. You just implement the code you need to solve your problem at hand. You don’t implement now what you (might!) need later.
We’ll incur technical debt! But it allows us to deliver software faster. We get our app into the hands of the users.
Second, there’s another best practice: “Don’t repeat yourself” (DRY). This practice aims to reduce the repetition of code. Rather than duplicating code, you replace it with an abstraction (such as a function or a class).
This is your chance to reduce technical debt in a meaningful way. You just learned you need this piece of code. The more you need it, the more time you invest in it. When you reuse it again, take the time it would take you to implement the code from scratch. You have this time because the chance to reuse some existing code just saved you some time. Look at the code. If necessary, improve it or even refactor it. But only invest the time you saved.
By paying down your technical debt that way, you make sure you invest the time where it matters. Further, you can argue that you worked on the feature (or user story or whatever) you promised to work on. The invested work added to that feature.
You don’t have to ask for or claim time for refactoring. Your business will always ask you why refactoring is necessary right now: “It does work right now. Doesn’t it?” If it does, spending time (thus money) on it does not repay. If it doesn’t, you are to blame for writing code that does not work.
“But if we don’t refactor the code now, implementing features in future will require more time,” you say? The answer most likely is: “You were lazy at the time you implemented it. You took a shortcut. And now we need to pay for implementing it again?”. You’re not gonna win this blame game. It is no fun, either.
Invest the time you save by reusing code to pay off technical debt. You’ll improve the code that matters. You’ll be able to deliver your features while keeping your technical debt down. And you’ll prevent needless discussions.