How does NodeJS work(Beginner to Advanced)? — Event Loop + V8 Engine + libuv threadpool

I finished a course recently on NodeJS: Advanced Concepts to improve my understanding of the inner working on NodeJS. This blog is based on this course and other research that I did myself during the course. The blog is created in the form of a conversation between you and me. Please be patient with it and everything will come together in the end, I promise. Let’s dive into it then.

I hear that you are explaining about NodeJS internal working. Can you please start by explaining what NodeJS is?

The official documentation states:

Node.js is a JavaScript runtime built on Chrome’s V8 JavaScript engine.

Woah! There were quite a few buzzwords in there which make no sense. Can you please explain them one-by-one? First off, what is a “runtime” and why is NodeJS called a “runtime”?

A “runtime” is any code which is executed to make your code work. The code that we write is not “runtime” code. It is just some code which will be executed by the “runtime”. A “runtime” can consist of a compiler, an interpreter, an engine to execute your code, threadpool to provide threads for your code’s process.

NodeJS is called a “runtime” because it consists of a V8 engine, libuv threadpool and OS Async Helpers(more on these soon). All of these combined give you the power to write the JavaScript code on the server that you are used to writing.

JavaScript is the language. NodeJS is the “runtime” responsible to execute JavaScript code.

So, I understand now that we shouldn’t call NodeJS a language and that it is a “runtime”. Coming back to official NodeJS doc, please explain what an “engine” is.

An “engine” is the actual code responsible to execute your code. Depending on the runtime/language, an engine could consist of a compiler, interpreter, garbage collector, call stack, heap(we’ll expand on these in the V8 section below). For example, if we write the code: console.log("Hello World!");. This code is executed by the “engine”.

I need an analogy for “runtime” and “engine” to clear things up a bit more.

You can think of your code as a train’s coach. Without an “engine”, there is no way a train’s coach could run. “Engine” is the component responsible for running the train’s coach(your code) but the “engine” alone cannot run the coach. It would need train’s tracks, railway stations, signals, fuel, etc. All of these combined create a “runtime” which runs your train’s coach(your code) effectively and gives a seamless experience.

Understood. Thanks, I guess the haze is clearing up now. I keep hearing about this unicorn-like entity called the“V8 engine”. Can you please clear it up for me?

Sure. Let’s start with official documentation:

V8 is Google’s open source high-performance JavaScript and WebAssembly engine, written in C++.

V8 is written in C++. As V8 is an “engine”, it is responsible to provide the features required to execute the JavaScript code that we write. V8’s responsibilities:

1. Compile and execute JavaScript code
2. Provide memory heap to allocate and store your variables.
3. Provide a call stack to run the functions in order.
4. Garbage collector — to garbage collect the object and memory references not in use anymore.
5. Provide data types, operators, object and functions to write the code in JavaScript language.
6. Provide an event loop to run asynchronous code. (Side note: This is not always true. In some cases, like in the browser, browser’s APIs tend to provide the event loop too.)

In a nutshell, from the time you run node <your-file-name>.js for file-based execution (or justnode for REPL execution) to the time your program exits, everything in between to execute your code is handled by “V8 engine”.

If you want to dive deeper into the V8’s responsibilities above, I refer you to JavaScript V8 Engine Explained. The author of the blog has given a comprehensive explanation with relevant links as to how V8 works. Following is an image of V8 from the blog:

Gotcha. So, I understand V8 executes my JS code and provides my program with an “Event Loop”, which is another buzzword I see floating around in the NodeJS world along with “single-threaded”. Can you please explain what an “Event Loop” is and why is NodeJS called “single-threaded”? Also, if you could explain threads to me, that would be great!

Anything your computer does is done using a process. For example, showing the display is done using a display process. A file is opened by a text editor process. Music is played by a music player process.

Similarly, in order to run your node program, a process is required. This process is created as soon as you run node in your terminal.

Each process has threads which actually execute the code inside the program of the process. Threads use your CPU cores to execute the code. A process can have more than one thread. Threads can or cannot share memory among each other — it depends on the runtime and the engine. So, when you run the following code using node: const foo = 2 + 2;, this is run by a thread using a core of your CPU.

Hopefully, this clears up process and thread for you. Now, let’s clear one more thing up before we proceed.

NodeJS process is not “single-threaded”. “Event loop” is single-threaded.

NodeJS, in fact, uses a threadpool provided by libuv(remember we had mentioned this earlier? We’ll continue to dive into it later).

“Event loop” is, well, just a loop like a for loop or a while loop. When you start your node process by running node index.js, the following sequence of steps take place:

1. V8 engine reads through and processes your index.js file. This includes requiring modules and executing any code in it.
2. V8 engine then immediately starts up an “Event Loop”.

As you can see, “Event Loop” is not responsible for executing your code. The V8 engine is. I’ve also been a victim of this misconception for a long time that Event Loop executed my code. “Event Loop” is just a loop. V8 is the one which actually compiles and executes the code.

Ok. I understood processes and threads. I also understood when “Event Loop” starts up but I still don’t know what it does. Why is it even there? Are you trying to dodge my question? 🧐

Of course not. I just wanted you to ask this question yourself (🤥). An “Event Loop” is some piece of code in a loop which picks up tasks from “Job/Task Queue” and sends them to V8 to be executed.

Hold on. What is “Job Queue” now? What are the “tasks” that you mentioned? How do these “tasks” enter the “Job Queue”?

I see nothing gets past you, does it? So, a “Job Queue” is a FIFO(first-in-first-out) data structure which is just used to store some data(or “tasks”). You can think of it as an array of tasks. The “tasks” keep getting pushed to the end of the array and they keep getting picked up by the “Event Loop” from the start of the array.

A “task” is nothing but some code inside a function that needs to be executed.

A “task” enters the queue when a callback function is called after an I/O operation, setTimeout, setInterval, setImmediate, or an OS task complete.

Woah! Hold on again. You said a bunch of stuff again which make no sense. What is a “callback function”? What are I/O operation, setTimeout, setInterval, setImmediate, and OS task?

A “callback function” is just a function which is called after some time. This function is called when an I/O operation, setTimeout, setInterval, setImmediate, or an OS task complete.

For example, the function(){...} in the below setTimeout is a “callback function” which will be called after 1 second when the setTimeout completes:

setTimeout(function() {
    const foo = 2 + 2;
    console.log("Hello World!");
}, 1000);

When the above setTimeout completes, the “callback function” is added to the “Job Queue” to be executed. “Event Loop” then picks up the function from the “Job Queue” and sends it to “V8 Engine” for it to be executed. “V8 engine” “compiles” and “executes” the code using “Call Stack” to assign functions, “Memory Heap” to allocate variables and “Garbage Collector” to remove the variables and function references once the code execution is complete. See how it all came together? 😎

Coming back to your question, similar to setTimeout, we have setInterval(to execute some callback function code at given intervals), setImmediate(to execute some callback function code immediately), I/O operation(eg: file read) callback function, OS task(eg: network call) callback function. All of these callback functions keep getting added to the “Job Queue” as “tasks” for them to be executed.

Gotcha. Now, I understand “Job Queue” completely, what “tasks” are and how they get entered into the “Job Queue”. Now, can we come back to the “Event Loop”? You mentioned that it picks up tasks from “Job Queue”. Is there any priority in which it picks up tasks from the “Job Queue”? Is there anything else that it does apart from this?

Glad to hear all of this is making sense to you. I’ll even give you an analogy in the end if you ask for it. For now, let’s come back to your questions. Yes, the “Event Loop” does pick up “tasks” from the “Job Queue” based on priority. It doesn’t do anything else apart from picking up “tasks” from the “Job Queue”. The priority of the pick up from the “Job Queue” is as follows:

1. Pick up setTimeout and setInterval callbacks.
2. Pick up any pending OS operation callbacks. An example of an OS operation is a network call, like a REST API call to an API to get some data.
3. Pick up any pending I/O operations callbacks which have been executed by libuv threadpool(we’ll come to this in a minute). An example of an I/O operation is reading/writing a file using fs module.
4. Pause and wait for an event to happen. No, really. It just pauses the execution. “Event Loop” is a bit different from the for and while loops we are used to writing. The loops we write just keep executing as fast as possible. “Event Loop” literally pauses and waits for the stuff to happen. An example of the “stuff” is a new request on the NodeJS web server.
5. Pick up setImmediate callbacks.
6. Pick up any callbacks from close events.
7. Exit the loop if refs === 0

Points 4 and 7 above are the reasons why your node process doesn’t stop when you are running a web server(eg: using Express framework or simple http.createServer() using http module). As a result of point 4, Node’s “Event Loop” just keeps waiting for any request to come in so that it can continue execution. It doesn’t keep running the queue as fast as possible. Also, when it does continue the execution, it doesn’t exit the loop because of point 7. refs is an integer variable which increments a value when something is to be executed and decreases a value when that execution is done. When you start a web server, refs never decreases below the value 1. This means that your loop never exits unless there is an uncaught exception.

Points 4 and 7 are also why the node process exits immediately after running if you don’t start a web server.

Point 5 is the reason why if you run the same callback function using setTimeout(function() {...}, 0); and setImmediate(function() {...}), setImmediate's callback will always execute before setTimeout's callback. setTimeout's callback is always picked up in the next tick of the “Event Loop”. setImmediate's callback is picked up immediately.

Point 6 handles any close event handler callbacks that we might have created.

Awesome. I understand the functioning of “Event Loop” but I still have a teeny-tiny question regarding it.

Sure, shoot.

You had mentioned that I can visualize “Job Queue” as an array in which “tasks” get entered. So, how does “Event Loop” decide the priority with just a single array of tasks?

Hmm, good question. So, when I had mentioned that “Job Queue” is an array, I had trivialized it. “Job Queue” is, in fact, composed of 3 arrays which are formed right before the execution of index.js file and the start of “Event Loop”. The 3 arrays are: pendingTimers, pendingOSTasks, pendingIOOperations. Event loop decides the priority based on these 3 arrays. Below is the pseudocode of index.js execution and event loop to help you out. The code below runs when you run node index.js in your terminal:

// node index.js
// contains callbacks for setTimeout, setInterval, and setImmediate
const pendingTimers = [];
// contains callbacks for OS tasks(eg: server listening to a port)
const pendingOSTasks = [];
// contains callbacks for long running operations by libuv
// threadpool(eg: fs module)
const pendingIOOperations = [];
myFile.runContents(); // V8 executes the file contents
// this fn helps event loop decide whether it should continue or not
function shouldContinueEventLoop() {
    return pendingTimers.length || pendingOSTasks.length || pendingIOOperations.length;
}
// start event loop. Entire body of the loop is one tick
while(shouldContinue()) {
    // 1. Look at pendingTimers. pick up tasks, if any.
    // 2. Look at pendingOSTasks. pick up tasks, if any.
    // 3. Look at pendingIOOperations. pick up tasks, if any.
    // 4. Pause execution. Continue when either:
          // - a pending timer is done.
          // - a pending OS task is done.
          // - a pending long running operation is done.
    // 5. Look at pendingTimers and call any setImmediate
    // 6. Handle any 'close' events.
    // 7. Exit if refs === 0
}

Great! Now, the “Event Loop” is not mythical anymore. It is just responsible for picking up tasks from “Job Queue” based on some priority and nothing else. The picked-up code is executed by the “V8 engine”. Also, that “Event Loop” is “single-threaded” and not NodeJS. I have a few more questions, I promise.

Sure but before we continue, go have some beverage of your choice and let everything sink in before continuing. If you have any doubt, you always have the comment section to shoot any question my way.

Cool, I had a cup of joe and I think I understand all of this. I even got a couple more questions while thinking about our conversation. Am I correct in assuming that JavaScript is “single-threaded” because “Event Loop” is “single-threaded” and “Event Loop” is created by “V8 Engine” which is responsible for executing JavaScript code?

Yes, absolutely right. JavaScript is “single-threaded” because JavaScript works on “Event Loop”. NodeJS is NOT “single-threaded” as it has a libuv threadpool in its runtime.

What about browsers then? What is “runtime” in a browser?

Browsers also have their own “runtime”. Browsers also are multi-threaded and those features are provided by tools like Web APIs.

Understood. One main question I have is that I write code in NodeJS which does all sorts of I/O and network requests. I see that “Event Loop” doesn’t make these requests and neither does “V8 Engine”. Then who makes these requests?

Awesome question! Do you remember I have mentioned “libuv threadpool” a few times and I had also mentioned “OS Async Helpers” initially when explaining about NodeJS runtime?

Yes, I do. I had made a mental note of these in order to ask you about these later on.

Cool then, let’s get into these. “libuv” is a library which provides a set of threads(threadpool) to NodeJS runtime to execute long-running tasks, like fs module-based tasks. By default, it provides 4 threads to each node process but you can change the threadpool size by setting process.env.UV_THREADPOOL_SIZE to any value you want. In case you are running node in a cluster mode, each child gets 4 threads by default to play around with and changing process.env.UV_THREADPOOL_SIZE changes the number of threads for each child.

“OS Async Helpers” are used when any low-level OS operations take place. For example, a REST API call using http or https module or creation of a web server using http.createServer(). These operations bypass the threadpool completely and are executed immediately as soon as a CPU core is available for operation.

Once “libuv” or “OS Async Helpers” are done with their tasks, they put the callbacks into the “Job Queue” for the “Event Loop” to pick them up and then send them to “V8 Engine” to be executed.

Awesome! I understand these now. Will you be able to give me an analogy now for all of the above conversations so that I can confirm that my understanding is correct?

Super easy, barely an inconvenience!

Imagine you are at a restaurant. Below are the actors:

1. The restaurant is the NodeJS runtime.
2. The JavaScript code you want to be executed is the food you and other people want to have at the restaurant.
3. The waiter is the V8 engine and the Event Loop.
4. The chefs are the libuv threadpool and OS Async Helpers.

Let’s say you want to eat something(execute some code), you ask the waiter(V8 Engine) to take your order(execute the code). The waiter does not, though, wait for you to decide on the order. If he sees that he is not doing any task, he goes to other people to take their order or to serve their order or to do any other task. Once you are ready to order, only then the waiter comes over to you to take your order. This is Event Loop. The waiter is intelligent and is able to serve more customers in the same amount of time.

Let’s say you gave your order to the waiter. The cooking of the food is a long-running process(I/O operation). The waiter doesn’t cook the order himself. He forwards it to the chefs(threadpool) and they end up cooking the order(performing I/O). Once the order is ready, they inform the waiter(Event Loop). The waiter then picks up the order and serves to you.

Eventually, all the customers end up being satisfied at the NodeJS runtime restaurant.

Awesome! Thanks for that. I think this clears up most of the NodeJS internal working to me. I still have a few questions around compilation, interpretation, garbage collection, clustering, etc. You had mentioned these during the conversation.

Patience you must have, my young padawan. Let everything brew in your mind for now. Try to relate these concepts to the code you write daily. We’ll come back to the questions above in the later blogs/conversations and I’ll link them here.

Sure, Thanks. See you soon.

I hope this blog was a learning experience for you. The purpose of writing it was to give back my learnings to the community and to start a conversation around it. Another purpose was to solidify my own learning and learn something new in the process too.

If you had fun reading the blog, please show some ❤️ by clapping for it, sharing it with others, and also follow me on Twitter and Medium to get notified whenever I blog. Feel free to comment too.