Cookbook approach to asynchronous code in modern JavaScript.
The JavaScript event loop and run-to-completion semantics.
Callbacks for event handlers.
Pyramid of doom.
Taming asynchronous code: promises.
Taming asynchronous code: async, await.
Can use asynchronous code without understanding underlying concepts
by using "keywords" async and await:
If a function is documented as async or as returning a
promise, then it is possible to call it using the await
keyword.
The await returns with the success value only when the
underlying asynchronous operation completes.
If an error occurs in the asynchronous function, then
the resulting exception can be handled using the
usual try-catch.
The await keyword can only be used within functions
declared using the async keyword. Consequently, any
use of asynchronous code within a program will necessitate
declaring the top-level function in the program async.
Enables writing asynchronous code in a synchronous style.
The browser provides fetch() to allow accessing resources asynchronously.
async function getUrl(url) {
const response = await fetch(url);
if (!response.ok) {
throw new Error(`HTTP error '${response.status}'`);
}
else {
return await response.json();
}
}
This was used in the crawler discussed earlier.
Docs for connect(), db, collection, findOne() and close().
const mongo = require('mongodb').MongoClient;
async function(mongoUrl, dbName, collectionName, id) {
//bad code: reuse db conn; close() should be in finally.
try {
const client = await mongo.connect(mongoUrl);
const db = client.db(dbName);
const collection = db.collection(collectionName);
const value = await collection.findOne({_id: id});
await client.close();
return value;
}
catch (err) {
console.error(err);
throw err;
}
}
Modern CPUs have clocks in the low GHz. That means individual CPU operations occupy under 1 nanosecond.
Typically, I/O may take in the order of milliseconds which is around a million times slower than CPU operations.
Highly inefficient to have CPU wait for an I/O operation to complete.
Need to concurrently do other stuff while waiting for I/O to complete.
Note that browser responsiveness is usually controlled by I/O responsiveness.
Two commonly used approaches to concurrency:
When a program attempts to do I/O, the program blocks until the I/O is completed. To allow concurrency, the operating system will schedule some other activity while waiting for the I/O. The unit of scheduling is usually a process or thread; leads to the process/thread model used by many current OS's.
When a program attempts to do I/O, it merely starts the I/O after registering a callback handler to handle the I/O completion event. The program continues running while the I/O is happening concurrently. The completion of the I/O results in a event which results in the registered handler being called.
JavaScript uses the asynchronous event model.
JavaScript started out simply using callbacks for asynchronous programming.
Promises introduced into JavaScript in ES6 around 2012.
Allow using asynchronous functions in a synchronous style
using async/await in ES2017.
Asynchronicity is required for message passing which is an essential ingredient of Reactive Programming.
The top-level JavaScript runtime consists of an event loop which pulls tasks corresponding to completed events off a task queue and calls their handler function.
while (taskQueue.notEmpty()) {
const task = taskQueue.remove();
const handler = task.handler();
handler.call(); //pass suitable arguments
}
//terminate program
The hander.call() runs to completion.
Code does not need to deal with an event handler being interrupted.
Code still needs to deal with the fact that the order of running of event handlers is not defined.
#!/usr/bin/env nodejs
//BAD CODE!!
function sleep(seconds) {
const stop = Date.now() + seconds*1000;
while (Date.now() < stop) {
//busy waiting: yuck!
}
}
setTimeout(() => console.log('timeout'),
1000 /*delay in milliseconds*/);
sleep(5);
console.log('sleep done');
$ ./run-to-completion.js sleep done timeout $
Because of run-to-completion semantics, it will always be the case that the sleep done message will be output before the timeout message.
JavaScript was designed as a language which should be easy for inexperienced programmers to use for scripting dynamic behavior in browsers.
Browser reacts to user actions by generating events like key-press, mouse-click, etc.
Browser programmer needs to provide optional event handlers for these events in order to implement browser dynamic behavior.
Since every event handler runs to completion, programmer can simply concentrate on code for that event, ignoring other events (at least for independent events).
No need for the programmer to understand complex process / threading models.
Lower overhead for I/O bound tasks; well suited for browser environment.
Simulate asynchronous success and failure using async-sim.mjs which uses setTimeout() to run function asynchronously or fail after a 2-second delay:
const TIMEOUT_MILLIS = 2*1000;
export function asyncSucc(fn, ...args) {
setTimeout(fn, TIMEOUT_MILLIS, ...args);
}
export function asyncErr(msg) {
setTimeout(() => { throw new Error(msg) }, TIMEOUT_MILLIS);
}
// Utilities
export const p = console.log;
export function t() { return new Date().toTimeString(); }
> const { asyncSucc, asyncErr, p, t } =
await import('./async-sim.mjs')
undefined
> p(1, 2, 3)
1 2 3
undefined
> t()
'08:56:59 GMT-0400 (Eastern Daylight Time)'
> asyncSucc(() => p('done'))
undefined
> done
> asyncErr('some error message')
undefined
> Uncaught Error: some error message
>
> function f() {
asyncSucc(() => { p('f run'); return 42; });
}
undefined
> let ret = f();
undefined
> f run
> ret
undefined
>
How do I get hold of the 42 return value.
Try using a global var to get hold of return value.
> function f() {
asyncSucc(() => { p('f run'); ret = 42; });
}
undefined
> ret = -1; f(); p(`ret after f() is ${ret}`);
ret after f() is -1
undefined
> f run
> ret
42
>
The only way to access the return value of an asynchronous operation is via a parameter to the callback function.
> asyncSucc(callback => callback(42), v => console.log(v)) undefined > 42 >
We use a simple async simulation of placing
an order. Have an orderer object which simulates consecutive
asynchronous steps:
Validate parameters.
Place order.
Send confirmation email.
The command-line program main.mjs allows up to two arguments:
The module to be used for placing an order.
An optional argument which specifies the step at which an error occurs.
function doOrder(orderer, params) {
orderer.validate(params, succ => {
orderer.placeOrder(succ, succ => {
orderer.sendEmail(succ, succ => {
console.log(succ);
});
});
});
}
$ ./main.mjs happy-cb-order.mjs happy-cb-order validated; order placed; email sent # same result, even tho placing order fails!! # (extra "order" argument forces error when placing order) $ ./main.mjs happy-cb-order.mjs order happy-cb-order validated; order placed; email sent $
//Normal exception catching
> try {
throw 'throwing';
} catch (ex) {
p(`caught ${ex}`);
}
caught throwing
undefined
//Exception in Async not caught
try {
asyncErr('some error');
}
catch (ex) {
p(`caught ${ex}`);
}
undefined
> Uncaught Error: some error
>
for (const path of [ './readfile.js',
'./xxx.js', ]) {
fs.readFile(path, (err, data) => {
if (err) {
console.error(`cannot read ${path}: ${err}`);
}
else {
console.log(`read ${path}:\n${data.slice(0, 15)}...`);
}
});
}
function doOrder(orderer, params) {
orderer.validate(params, (succ, err) => {
if (err) {
console.error(err);
}
else { //validate ok
orderer.placeOrder(succ, (succ, err) => {
if (err) {
console.error(err);
}
else { //placeOrder ok
orderer.sendEmail(succ, (succ, err) => {
if (err) {
console.error(err);
}
else { //email ok
console.log(succ);
}
}); //email
} //placeOrder ok
}); //placeOrder
} //validate ok
}); //validate
}
# happy path $ ./main.mjs err-cb-order.mjs err-cb-order validated; order placed; email sent # catches errors properly $ ./main.mjs err-cb-order.mjs validate validate error $ ./main.mjs err-cb-order.mjs order order error $ ./main.mjs err-cb-order.mjs email email error $
A top-level exception handler does not work for asynchronous callbacks since the handler runs before the callback. Hence exceptions occurring within the callback are not caught by the top-level exception handler.
If an asynchronous function result needs to be further processed by another asynchronous function, then we need to have nested callbacks.
A chain of callbacks leads to the pyramid of doom because of nesting of callbacks.
A Promise is an object representing the eventual completion or failure of an asynchronous operation.
When a function which requires an asynchronous callback as an argument is called, it returns immediately with an object called a pending Promise. Subsequently, the callbacks can be added to the promise. The callbacks will be called after the promise has been settled.
let promise = some_call_which_returns_promise(...); promise. then(callback1). then(callback2). ... catch(errorCallback);
Promises can be chained; this avoids the pyramid of doom.
Callbacks are never called before completion of current run of js event loop.
Callbacks added using then even after completion of the
asynchronous operation will still be called.
then() can be called multiple times on the same promise to
add multiple callbacks (called in order of insertion).
Allows catching errors much more easily using catch();
similar to exception handling.
then() can even be chained after a catch.
new Promise(
/* executor */
function(resolve, reject) { ... }
);
Creates a promise.
resolve and reject are single argument functions.
Executor function executed immediately. Usually will start some kind of asynchronous operation which may return some result.
If the async operation succeeds with some result succ, then
the executor function should call resolve(succ).
If the async operation fails with some error err, then the
executor function should call reject(err).
function doOperation(...params) {
return new Promise((resolve, reject) => {
asyncOperation(...params, (result) => { //callback
if (isOk(result)) {
resolve(result);
}
else {
reject(result);
}
});
});
}
doOperation(...).
then(result => { ... }).
catch( err => ...);
The underlying operation is not yet complete.
The underlying operation completed; it is known whether or not it succeeded resulting in two settled sub-states:
The underlying operation completed successfully.
The underlying operation failed.
A promise is settled only once. The state of the promise will not change once it is settled.
then()somePromise.then(value, err)
Arguments are one argument functions called when somePromise is
settled; specifically value / err are called with fulfillment
/ rejection value depending on settlement.
Usually then() is called with only the value argument,
with rejection of somePromise handled using a catch().
then() itself returns a promise; this allows chaining then's.
If the function passed to then() returns a value,
then the return'd promise fulfills with that value.
If the function passed to then() throws an error,
then the return'd promise rejects with that error.
If the function passed to then() returns a promise,
then the return'd promise has the same settlement
as it.
catch()somePromise.catch(err)
err is a one argument functions called with the rejection value
of promise somePromise.
catch() itself returns a promise; this allows continued promise
chaining. Return value is similar to that of then().
API for nodejs evolved before promises were added to language.
util.promisify(fn) can be used with existing callback-based
nodejs library function fn to make it return a Promise.
For example util.promisify(fs.readFile) returns a wrapped
version of fs.readFile which will return a Promise.
function readFilePromise(path) {
return util.promisify(fs.readFile)(path, 'utf8');
}
for (const path of [ './readfile.js',
'./xxx.js', ]) {
readFilePromise(path)
.then(data => {
console.log(`read ${path}:\n${data.slice(0, 15)}...`);
})
.catch(err => {
console.error(`cannot read ${path}: ${err}`);
});
}
> pr = new Promise((resolve, reject) => resolve(22))
Promise { 22, ... }
> pr.then((v) => p(v))
Promise { <pending>, ... }
> 22
//Promise is settled only once
> pr = new Promise((succ) => { succ(42); succ(22); })
Promise { 42, ... }
> pr.then((v) => p(v))
Promise { <pending>, ... }
> 42
then()'s> function f(a, b) { p(a); return a * b; }
undefined
> pr = new Promise((resolve) => resolve(22))
Promise { 22, ... }
> pr.then((val) => f(val, 2)).
then((val) => f(val, 3)).
then((val) => p(val))
> Promise { <pending>, ... }
> 22
44
132
>
Promise.resolve(value) Returns a promise which is already fulfilled with value.
Promise.reject(err) Returns a promise which is already rejected with err.
> function f(a, b, ret) {
p(`${t()}: ${a}`);
setTimeout(() => ret(a*b), 2000);
}
undefined
> pr = Promise.resolve(22)
> pr.
then((v) => new Promise((succ) => f(v, 2, succ))).
then((v) => new Promise((succ) => f(v, 3, succ))).
then((v) => p(`${t()}: ${v}`))
> 08:21:42 GMT-0400 (Eastern Daylight Time): 22
08:21:44 GMT-0400 (Eastern Daylight Time): 44
08:21:46 GMT-0400 (Eastern Daylight Time): 132
>
> p(t()); pr1 =
Promise.reject(new Error(t())); pr1.catch(()=>{})
08:27:07 GMT-0400 (Eastern Daylight Time)
...
> p(t()); pr1.
then((v) => p(v)).
then((v) => p(v)).catch((err)=>p(err))
08:27:51 GMT-0400 (Eastern Daylight Time)
...
> Error: 08:27:07 GMT-0400 (Eastern Daylight Time)
...
> pr1.
then((v) => p(`got value ${v}`)).
then((v) => p(`got value ${v}`)).
catch((e) => { p(`caught ${e}`); return 42; }).
then((v) => p(`got value ${v}`))
Promise { <pending>, ... }
> caught Error: 08:27:07 GMT-0400 (Eastern Daylight Time)
got value 42
>
then()-chain continues past catch():
> Promise.resolve(1).
then((v) => { p(`then1: ${v}`); return v*2; }).
then((v) => { p(`then2: ${v}`); return v*2; }).
catch((e) => p(`caught ${e}`)).
then((v) => { p(`then3: ${v}`); return v*2; })
Promise { <pending>, ... }
> then1: 1
then2: 2
then3: 4
>
function doOrder(orderer, params) {
orderer.validate(params)
.then(succ => orderer.placeOrder(succ))
.then(succ => orderer.sendEmail(succ))
.then(succ => console.log(succ))
.catch(err => console.error(err));
}
# happy path $ ./main.mjs promise-order.mjs promise-order validated; order placed; email sent # catches errors properly $ ./main.mjs promise-order.mjs validate validate error $ ./main.mjs promise-order.mjs order order error $ ./main.mjs promise-order.mjs email email error./main.mjs happy-cb-order.mjs $
Revisit implementation of main program for asynchronous order simulator.
Promise.all()Given an iterable of promises, returns a promise containing array
of fulfilled values, or rejection if any promise rejected.
(note that mulN(i) returns promise for N*i after 2 second delay):
> Promise.all([mul2(3), mul3(4), mul4(5)]).
then((v) => p(v))
Promise { <pending>, ... }
> [ 6, 12, 20 ]
// err() returns a rejected promise
> Promise.all([mul2(3), err(3)(2), mul3(4), mul4(5)]).
then((v) => p(v)).
catch((e) => p(`caught ${e}`))
Promise { <pending>, ... }
> caught Error: err
Promise.all() ContinuedPromise.all() runs all promises in parallel:
> p(t()); Promise.all([mul2(3), mul3(4), mul4(5)]).
then((v) => p(`${t()}: ${v}`))
15:49:41 GMT-0500 (EST)
Promise { <pending>, ... }
> 15:49:43 GMT-0500 (EST): 6,12,20
Took 2 seconds to run all 3 functions even though each function takes 2 seconds apiece.
Promise.race()Given an iterable of promises, returns a promise containing settlement of which ever incoming promise completes first.
> Promise.race([mul2(3), mul3(4), mul4(5)]).
then((v) => p(v))
Promise { <pending>, ... }
> 6
>
Promise.any() is similar except that it rejects only if
all promises reject (Promise.race() will reject if the
first settled promise rejects).
Combine promises and generators to linearize control flow:
function generatorOrder(orderer, params) {
const gen = doOrder(orderer, params);
gen.next().value //validation promise
.then(succ => gen.next(succ).value) //order result promise
.then(succ => gen.next(succ).value) //email result promise
.then(succ => console.log(succ)) //output overall result
.catch(err => console.error(err)); //catch any error
}
function* doOrder(orderer, params) {
const validationResult = yield orderer.validate(params);
const orderResult =
yield orderer.placeOrder(validationResult);
yield orderer.sendEmail(orderResult);
}
# happy path ./main.mjs generator-order.mjs generator-order validated; order placed; email sent # catches errors properly $ ./main.mjs generator-order.mjs validate validate error $ ./main.mjs generator-order.mjs order order error $ ./main.mjs generator-order.mjs email email error $
async / awaitExtra syntax around promises and generators to allow writing asynchronous code in a synchronous style.
If a function or function expression has the async (contextual)
keyword in front of it, then that function always returns a
promise.
When the await (contextual) keyword is used in front of
an expression which is a promise, it blocks the program
until the promise is settled. The value of an await
expression is the fulfillment value of the promise.
The await keyword can only be used within a async function.
Errors can be handled using try-catch.
Seems a big win.
Note that we may need to fall back on promises using Promise.all()
when we want to run code in parallel rather than sequentially.
async / await Example> function msgPromise() {
return new Promise(function (resolve) {
setTimeout(() =>
resolve(`hello@${t()}`), 2000)});
}
undefined
> async function msg(n) {
const m = await msgPromise();
return `${n}: ${m}`
}
undefined
async / await Example: Invoking using IIFEUntil ES-2022, top-level await not allowed. Used IIFE as workaround.
> ( async () => { //must use async to use await
p(await msg(22));
p(await msg(42));
})() //async IIFE
Promise { <pending>, ... }
> 22: hello@21:06:53 GMT-0500 (EST)
42: hello@21:06:55 GMT-0500 (EST)
>
sleep()> async function sleep(millis) {
return new Promise((resolve) =>
setTimeout(() => resolve(), millis));
}
> p(t()); await sleep(2000); p(t()); //es-2022 top-level await
18:49:06 GMT-0400 (Eastern Daylight Time)
18:49:08 GMT-0400 (Eastern Daylight Time)
undefined
>
async and awaitasync function doOrder(orderer, params) {
try {
const validation = await orderer.validate(params);
const order = await orderer.placeOrder(validation);
const result = await orderer.sendEmail(order);
console.log(result);
}
catch (err) {
console.error(err);
}
}
# happy path ./main.mjs async-await-order.mjs async-await-order validated; order placed; email sent # catches errors properly $ ./main.mjs async-await-order.mjs validate validate error $ ./main.mjs async-await-order.mjs order order error $ ./main.mjs async-await-order.mjs email email error $