Pensando en programar

Why are there so many mechanisms for asynchronism?

Question: Why does the language have so many mechanisms for asynchronism (callbacks, promises, async/await)? And while we're at it, what are the differences or the advantages/reasons behind each one?

(Yes, I'm almost cheating with this question. Nobody ever asked me this in this particular way. But I have received a number of questions on various involved parts and I believe it's a good idea to approach it all together, even though I know that doing so will produce a longer text.)

The nature of JavaScript

We can find a number of explanations around on the the nature of JavaScript as a language. Most of them will offer different descriptions, usually tinted to some extent by the points that explanation wants to reach. And so we can find people talking about the functional nature of the language or its object oriented nature, or maybe about its dynamic nature, or about how it is interpreted or compiled, or even how its nature is that of being an “easy to learn” language, or how it's simple or how it's designed to fail silently. If we reach for a more general resource, such as a book, we may even find some or many of these natures mentioned.

However, there's an aspect of the language that defines it in a fairly fundamental way and that is less frequently mentioned. 1)

It's essential to understanding JavaScript and understanding asynchronism, to know that it is a language designed to live inside a very specific execution model. It's a language designed to live inside a document.

And while later it may have been extracted from there to other environments and given the opportunity to run as a general scripting language or given I/O libraries as did NodeJS, that original idea of living inside a document model has always, very markedly, defined one of the fundamental aspects of its nature: It's a language with an event driven model of execution.

This will not only mark off its design and evolution in some technical aspects, such as being single-threaded, but will also define in a critical way much of the philosophy behind its usage and application design.

Events => Asynchronism

So, somewhat inevitably, derived from that intrinsic aspect of the design of the language, asynchronicity results from the nature of the language itself. Because, for implementing an event driven, all the execution model is better served if designed to be asynchronous. 2)

There are already available on the web a number of nice overviews of JavaScript's model of execution, the message/task queue, the event loop, the influence of HyperTalk, and so on. It doesn't make much sense for me to include it here 3) but only because those exist already and many of them are quite good. It is interesting to know about it, so if you need or want to, do read them.

In any case, the conclusion is clear: Asynchronicity is a key aspect of the language and thus it can be expected that JavaScript should provide good mechanisms to use it correctly and efficiently. And so our first conclusion: There are various mechanisms because the subject is important to the language.

I want you back

The truth is that the concept of callbacks is not necessarily tied to the concept or asynchronism. We can use callbacks in a completely synchronous way, and in fact it is done quite frequently 4).

To be precise, callbacks (or passing functions as values) do not provide asynchronism on their own. What happens is that this is the basic mechanism that facilitates it. Facilitates, but not provides. What originally allowed us to use asynchronicity in JavaScript is a model of Events and Observers. In other words, what people saw and focused on, is in the simple mechanism of having functions as values that, then, can be passed as arguments to other functions. But this, on its own, never produced asynchronism.

function doThings(first, second, third) {
    let r = first();
    if (Math.random()>0.3) {
        second(r);
    }
    return third(r);
}

doThings(() => 9, console.log, x => 2*x);

someArray.sort( (a,b) => 2*a - b );

What gives us, initially, asynchronism are the APIs for managing DOM events, and timing events 5). And it happens that these APIs are based on the idea of observers and listeners and they use the basic mechanism of passing functions (the listeners) around. And they do so not only because the idea of functions as values is a fundamental mechanism in the language but also because in that initial design it is the only one available.

someElement.addEventListener('click', someHandlerFunction);
// Or in an older, darker past...
someOtherElement.onclick = someHandlerFunction;
// ...but the idea is the same

Some of the APIs, as is the case of XMLHTTPRequest, apply a pattern closer to a Strategy, and again what we see is the mechanism of passing functions as values.

In any case, the problem with this approach is that in general most people tend to think about processes in a linear way.

Breaking the linear flow

In narrative authors have at their disposal a large number of tools. But if the author wants to communicate effectively with the audience, they need to be used correctly. Some of these tools, e.g. flashbacks, break the temporal linearity of the narrative. William Gibson, to pick a random example, uses one particular structure in a fair number of his novels: he builds a handful of temporal lines that advance parallelly but gradually converge into one another. Sometimes he inserts a twist and when lines converge, it turns out some of them where never parallel to begin with. I think this works well for Gibson because he has an already established audience he has built with each of his novels.

It is known that Jorge Luis Borges, after watching Citizen Kane, strongly criticized Orson Wells precisely for pretending that the audience should make the effort of re-ordering and combining the much fragmented narrative on Kane's life. In general, when a narrative piece opts for a disruption of the linear flow, be that large or small, even when it is masterfully done it ends up requiring more effort and attention from the audience. If this is already problematic when the narrative is non-linear, the problem is much worse when the story itself is non-linear. This is why stories about time travel, even when they are carefully and brilliantly built, quickly become extremely hard to follow for a large part of the audience.

When we tell stories to one another it can be a very valid option and we can sometimes produce a great result. When we're talking about communicating a process, which is mostly what we do when programming, things are a bit harder.

Event driven systems completely break the linear flow of the code into activities or small occurrences mostly independent form one another. Even leaving aside asynchronism, passing functions around also breaks that linear flow. We read some code and we know that the order of execution does not coincide with the order in which the lines are written.

Love is a battlefield

Things could have evolved differently. In an alternate reality, the event driven model might have won. The model is indeed very appropriate for creating visual user interfaces 6). But adding its complexity to other complexities of the web platform, finally developers decided to seek other options 7).

It has to be understood, though, that the appearance of Promises does not imply a new model for asynchronism nor was it intended to do so. The only goal the promises seek is re-inverting the inversion of flow control that was introduced by callbacks. In this classic discussion 8) they called it “modelling the synchronous flow control from imperative languages”. The meaning should be clear.

But what does it mean?

When using callbacks, what we're doing is, essentially, relinquishing execution flow control from the client code and delegating it to the called function. So that we all understand, I'm calling client code to “our code”, the code of the process we're writing, and at a certain moment through that process, we will need to call some function (probably asynchronous, but not necessarily so). In that call, in addition to the arguments for the call itself, we pass our callback. And it is precisely in passing that callback that we are actually encapsulating the continuation of our process and passing it to the asynchronous 9) function. And it is that function that will call our continuation. That is, we've delegated the responsibility 10) for continuing our process when the function deems appropriate.

It is exactly this inversion of control the one that promises try to re-invert.

And how is that done?

The basic problem of asynchronism is that we want to produce some certain value to continue our process but there is a “delay” in obtaining that value. We don't know how much delay either. And so the idea of passing the continuation to the function that produces the value: that function will know and will then continue our process when the value is ready.

Instead of doing this, what promises propose is that the asynchronous 11) function should return something and that it should return it synchronously. While this may sound strange at first, the reality is actually quite simple. We separate the asynchronous operation from the responsibility of continuing. The asynchronous task get queued and it will produce the result when it does. But separately, in a synchronous way, the client code gets some return, an object.

We know that -logically- in the client code we don't yet have the value, but what we do have is we have regained execution control. And so, we've re-inverted that control.

Naturally, if there was nothing more to this, this wouldn't work at all. Because we do have the control, sure, but we still don't have the value we need and we don't know how to get it. But of course there is something more. The object returned is a subject. The asynchronous function will receive a reference to it and when it does finally produce the value, it can trigger some sort of event on it so that client code can receive the value. What we've done is introducing between client code and asynchronous function, a middleman, an observable subject on which both and operate to communicate, but more profoundly, we've extracted the responsibility of continuing out of the asynchronous function. The asynchronous function does no longer care about how to continue and the client code does no longer need to relinquish the responsibility of continuing on the function. Instead, we do so on the middleman subject.

A victory of love

So, promises allow us to regain the control of execution flow, and for a while the populace enjoyed in festivities and celebrations. But soon enough they notice that not all has been “solved” yet. The layout of the code still largely reflects its asynchronous nature and, so, differences in the execution order and reading order still remain 12).

But the re-inversion is, still, very significant. Not only on its own, which it is, but because thanks to it, the last step from the sequentialists becomes a code transformation that is completely local. Meaning that the step that goes from receiving and managing a promise to await becomes a transformation that only affects client code, not the called function. And it is having taken the previous step, arriving at promises, what allows taking the step into await.

As for the motivation for await, it should be fairly obvious: Regaining the “apparent” linear sequentiality of the narrative of source code. That is, await is all about making the asynchronous 13) code appear and read as if it was sequential. 14)

The end of the world

To conclude, as summary, we could say that the following are the various whys of asynchronism 15):

  • Why asynchronism? Because it is fundamental for an execution model driven by events and living inside a document.
  • Why callbacks? Bah, it's just a basic language mechanism, passing functions. The “correct” question is why observers? and the reason is because it is the most immediate pattern in an event driven system.
  • Why promises? Because passing function “accidentally” inverts responsibility and control over execution flow. Promises introduce a middleman to re-invert that control again.
  • Why async/await? Because even with the re-inverted execution control, promises still read non-sequentially and (it seems) most people 16) prefer a narrative with a sequential presentation.

In the end, it's all a bunch of trends fighting for relevance. Currently sequentialists seem to have largely imposed their vision, but as we already know software design comes and goes and returns and forgets and remembers again. Maybe in the future other trends, such as actors, to pick something out, may gain relevance and non-sequential narratives may become popular. Who knows? 17)

1)
Maybe because some never think about it and others simply assume it as a given.
2)
This doesn't mean it is the only way or that events == asincronism, but that it is the better way to implement it.
3)
To do: Not search for some interesting links on MDN, Mr.Aleph's blog, books… and not include them here
5)
setTimeout, setInterval
6)
They have actually been used extensively in most UI systems
7)
No promises, no demands… Love is a battlefield - Pat Benatar
8)
Which would later produce the birth of Fantasy Land, so not all is bad
9) , 11) , 13)
probably asynchronous
10)
and control
12)
Waiting for a change in the weather… A victory of love - Alphaville
14)
As for async, the reason is more mundane. There's no intrinsic motivation other than a technical sacrifice to mark a function as “needing the await transformation”.
16)
*sigh*
17)
Sadly, I don't really hope to live to see that day xD It seems far off yet.