Abstraction is one of the timeless values of the programming profession, and one of the pillars that make the complex systems we create even possible.
Without abstraction, we’d be writing software in machine code, and while that would have its benefits, I believe even those who bemoan contemporary software’s sluggishness would not want to throw away all of their modern tools. After all, both the expectations towards software, as well as the underlying hardware architectures have gotten considerably more challenging over the years.
But, like other important ideas and concepts, abstraction often becomes dogma, and by being applied toughtlessly, it can introduce the very thing it was meant to contain: complexity.
The way complexity can sneak in here, is a confusion between abstraction and indirection. This happens easily because abstraction alwasy implies some degree of indirection. If we interact with an abstract structure or system, we effectively delegate chosing the concrete manifestations to the inside of the abstraction black box. This is great, because it means we don’t have to make that choice ourselves. But when the code runs, when the rubber hits the road, the choice must be made somewhere. As illustrated with this stupid example from Intro to OOP 101:
[new Cat(), new Dog()].forEach(animal => {
animal.makeSound()
})
In our loop, we don’t have to distinguish between the type of animal we are operating on. The choice, which classes makeSound methods is going to be invoked is made by the Javascript runtime for us. Somewhere inside, a piece of code that could look like this is doing the work for us:
if (animal instanceof Dog) {
Dog.prototype.makeSound.apply(animal)
} else if (animal instanceof Cat) {
Cat.prototype.makeSound.apply(animal)
}
By removing this detail and the decision, we make room in our heads to think about more important issues.
Other examples for useful abstractions are
- the
fetchapi can be used to trivially fetch data from a url, without having to wrestle the details of setting up and configuring an HTTP session (and maybe even the underlying TLS/TCP details) - Your operating system’s filesystem lets you write data to disk without having to worry about the physical layout of the bits and bytes.
Indirection
Let’s look at indirection. A useful definition to help distinguish between abstraction and indirection is
- (In)direction is about how many steps there are between two endpoints
- Abstraction is about how many details need to be expressed between two endpoints
Adding abstraction often requires adding indirection. Even if the details of a process are not relevant for the consumer of an API, the implementation will at some point have to deal with them, so an intermediary is needed that fills in the blanks. The fetch API does not make the connection to a remote server simpler. It saves its user from having to specify details and then fills them in before making the call.
Sometimes you get lucky and while making the interface of an API more abstract, you discover that even the implementation doesn’t care about certain details. More often you’ll find yourself in a position where you discover that you just can’t abstract certain details away, since the consumer actually cares about them. The result of this is observation is an interface that is not much narrower than the one it’s supposed to be abstracting away from. What you end up with is only indirection, and no abstraction.
An example of this that I’ve encountered in different places and several forms over the course of my career is a function (or here, a component) that take a gigantic data structure as a parameter:
function ButtonDropDown({
buttons,
containerProps,
dropdownProps,
moreButtonProp
}) {
const [firstButton, otherButtons] = buttons
const [showMore, setShowMore] = useState(false)
if (rest.length === 0) {
return <button {...firstButton.props}/>
} else {
return <div {..containerProps}>
<button {firstButton.props}/>
<MoreButton {...moreButtonProps} onClick={() => setShowMore(s => !s)}>
<DropDown {...dropdownProps}>
{otherButtons.map(button => (
<button
{...button.props}
onClick={(e) => {
setShowMore(false);
button.props.onclick?.(e)
}}
className="dropdownButton"
/>
))}
</DropDown>
</div>
}
}
This one takes an array of buttons, displays the first directly, and stuffs the rest of them into a dropdown menu. The look and feel can be customized by basically passing in the props for every single part of this component. The idea was to encapsulate the concept turning a set of buttons into a dropdown-button-menu into a distinct component. That impulse isn’t wrong but this particular implementation is pointless.
- To use the
ButtonDropDown, I need intimate knowledge of its internal structure, to be able to know which props to pass and in what way those props might interact. - If I, or someone else, later reads that code, they also need to look up the source to understand what it does.
- Working on the
ButtonDropDownitself requires me to know all of its consumers. It is impossible to tell from looking at its code, what might get passed in via all these props objects
A different, smaller example is the following code for converting an internal identifier into an i18n-key:
export const generateElementLabelFromType = (
elementType: DesignElementType // '//my.namespace/shapes/rect'
) => {
const i18nKey = removeDesignElementNamespace(elementType); // 'shapes/rect'
const i18nKeys = cascadingKeys(i18nKey, '/') // ['shapes.rect', 'shapes']
.map((key) => {
return `component.inspector.${key}.label`;
// ['component.inspector.shapes.rect.label', 'component.inspector.shapes.label']
});
return t([i18nKeys, i18nKey]);
// ['component.inspector.shapes.rect.label', 'component.inspector.shapes.label', '//ly.img.ubq/shapes/rect']
};
The idea here is to codify somehow, that the id of a DesignElementType determines the i18n-key used to look it up in a translation file. The problem lies in the logic for determining the key. We strip a prefix from the id, break the result apart, and put it back together in different ways, then wrap each of these alternatives in a prefix and suffix, as a fallback we also try the original key. This is pretty convoluted, yet calling generateElementLabelFromType(type) looks pretty straightforward. The function does not abstract anything away: To know what to call our keys we need to have a close understanding of its inner workings. But even looking at this function’s implementation won’t give us any answers until we also peek into removeDesignElementNameSpace() and cascadingKeys() The only thing it achieves is to obscure the complexity of the logic.
Where abstraction creates clarity, by removing unnecessary details, these forms of indirection create obscurity by hiding necessary details! Obscurity leads to complexity, that makes software maintenance more expensive and error-prone.
Where does indirection come from
Nobody sets out to write obscure, complex software (almost nobody…), and code like the shown is always created with the best of intentions. So, what can lead to unnecessary indirection? I have a few suspects.
Lack of type systems
When Ruby on Rails took the world by storm some 15 years ago, it brought a ton of developers into the world of the web. Rails made web development easy, even pleasant. One core principle of Rails was DRY: Don’t Repeat Yourself. That idea became somewhat of a mantra for impressionable newcomers to the field (myself included). It’s easy to see in hindsight where this came from: if you have absolutely no guardrails in your types and data structures, the only way to sanely ensure a certain level of consistency across your application is to make absolutely sure that for every thing (process, structure), there is only a single place where it’s being specified. So that if the specification needs to be changed, you’re only gonna have to touch that single place.
I believe Rails left an enormous impression on a whole generation of developers, not just on those that directly worked with it, but to others as well, through practices that got carried over into the Javascript ecosystems by large numbers of developers that grew bored with Ruby just around the time NodeJS was introduced.
Javascript’s type system was even less sound than that of Ruby and the language’s relatively simple structure and powerful primitives (lexical scoping, first-class-functions) encouraged a wild decade of unprotected metaprogramming, and again, using DRY became essential in maintaining consistency.
The untyped wild west of Rails and Javascript was not all bad. It let developers basically come up with their own mental type systems even if those were unsound. Remember Duck Typing? “If it looks and sounds like a duck, it is a duck”? This is structural typing, just without any help from the compiler. At the time that was an immense feeling of freedom, compared to Java or C++.
Inflexible type systems
In their corner of the dev community, Java developers wanting to use abstractions had the opposite problem. A type system that was strict but quite limited in its expressive power. The only way to relate two distinct types was to explicitly spell out all alternatives everywhere through switch statements and function overloading, or — more idiomatic — to combine them through a common ancestor (a superclass or interface). This can quickly lead to the creation of a lot of trivial classes/interfaces, to Abstract Factories and a whole lot of convoluted machinery to make things fit together.
In this landscape, indirection is the only way to re-introduce dynamism that was taken away by the strict type system. The result were dependency injection frameworks, tons and tons of interfaces and delegation of all tasks one might want to implement differently at some point.
Maintainability arguments
A system based on dependency injection, with lots of slots into which various implementations can be placed can be seen as more maintainable, if you accept the premise that you should never have to modify any existing code to add functionality. This premise is valid when we are talking about a library that is released to consumers who don’t have the ability to modify it.
Without that constraint, the notion is absurd. The nature of software is that it’s able to change. That’s why we call it software. If you need to change how a part of a system works, and the whole system is under your control, go ahead and change it! Trying to anticipate all the parts that could possibly change, so that you won’t have to modify your core system introduces unknowns, moving parts and overhead that might end up never being used.
Misunderstandings
If you never really thought hard about the difference between abstraction and indirection, and you had any sort of exposure to these patterns and practices, it’s very easy to confuse the two. As explained earlier, the implementation of abstraction almost always involves indirection. Abstraction is (rightly) held in high regard, so with good intentions you try to put abstractions into your code base, but end up with indirection and obscurity.
Libraries vs. Frameworks
Coming back to the ButtonDropDown example, “render a menu from these buttons” is not an abstract task if you still need to take care of all the little details. Combining details into a single structure does not make them go way, it just introduces unnecessary coupling. This pattern (A complex function or component with lots of interrelated details, whose behavior is controlled via a large set of parameters) is one that I like to describe as frameworks. In software, a framework is a predefined structure, in which you fill in the missing pieces. But the core principle is that the framework is the entity that decicdes the overall shape of a structure and sets the rules. The consumer of the framework is not in charge. There’s a strange attraction to frameworks. The idea to create a beautiful one-size-fits all solution to a particular problem space that your team has can be very alluring to a developer. There’s a vague idea or promise lurking in the back of their mind to turn these into a prestigious pet project that they and they alone have authorship and power over. Maybe it’s even gonna be so great that you’ll end up releasing it as open source and bask in fame and glory. But that is almost certainly not going to happen.
In contrast, a library is a focussed tool for a particular, but small purpose, that needs to be combined with others into something useful. The important difference is that you, as the author and user of all these libraries have the power over the shape of the outcome. These are much less glamorous, but often the more pragmatic choice.
For the ButtonDropDown, we can take two approaches for improving the current implementation. Which one we chose will depend on how it’s used in our codebase.
If we discover that most consumers use it in the same way, we should turn the framework into a library by focussing it better and removing all the configurability that we don’t use. In practice this could mean to allow only a simple list of [title, handler] pairs to be passed in, and maybe a few select styling parameters. As a result, we can reason about the behavior of the ButtonDropDown by reading its source code alone, since we know that its consumers can’t meaningfully change how it works.
If instead it turns out that ever consumer supplies a completely different set of configuration options, we’re probably better off by dismantling the whole thing and breaking it down into several libraries, that can then be combined in different ways. Seeing how this is better is a bit tricky, but there are two things going on:
- We remove a layer of indirection and thus make it more obvious in the consumers what is actually going on.
- Assuming that the different consumers are different subsets of our frameworks features (otherwise, why would the configurations be different), we can remove logic that dynamically combines these features and instead hardcode what we need, where we need it. This brings down the complexity, since fewer moving parts are interacting with each other.
Make change easy
There’s one core principle that should be guiding our decisions when writing or refactoring code, that is to make change easy. Requirements can change, environments can change and it’s good to keep that in mind. But it is also important to understand that while anything could change, most things won’t. And premature abstraction is like premature optimisation, the root of all evil.
The best approach, as in most of software development, is to keep things as simple as possbile and examine closely for each indirection that you introduce, if it really serves that goal, while making change easy, or if it’s just sweeping complexity under a rug and obscuring what’s going on. Code that is simple is easier to understand and easier to change.
There are a few smells in particular to look out for:
- Complex configuration objects passed into functions. The fact that a function has a single parameter does not mean the function is simple. You need to consider the total amount of information passed in, not just the number of paramters.
- Providing parameters that nobody needs (yet). Using parameter objects instead of positional parameters. Long lists of parameters tend to grow into giant configuration objects.
- Using one data structure for an unrelated purpose, with some sort of translation logic. Even if a similarity between two separate structures can be constructed, don’t put that similarity into code unless that it is of essential importance. Deriving the i18n-keys from ids in the example creates a coupling that provides no benefit other than saving you some typing. It’s better to specify the keys explicitly for each id.
- Don’t optimize for things that rarely happen. Path aliases in Javascript projects are an example of those. You’re not gonna restructure your file layout all that much. So you’re saving yourself some typing, maybe you find the sight of
..’s unpleasant. The cost is that every tool in your stack must now be configured to support these aliases. If you want simple paths, why don’t you just keep your file system layout simple?
When considering how to make things easy to change, understand the ways that type systems and refactoring tools can support changing the behavior of a system.
- Change the type of something, let the type errors guide you to adapt the rest of the system.
- Use regular expressions, search and replace, vim macros, refactoring tools like JSCodeshift, and the power of your IDE to automate changes.
- Make code easy to understand