Node.js Design Patterns

My notes & review of Node.js Design Patterns by Luciano Mammino & Mario Casciaro

---

Notes

Chapter 2: The Module System

Loading Phases

1. Construction (or parsing): The interpreter identifies all imports and recursively loads the content of each module from their respective files.
2. Instantiation: For each exported entity in every module, the interpreter creates a named reference in memory, but it does not assign it a value yet. References are created for all the import and export statements to track the dependency relationships between them (liking). No JavaScript code is executed during this phase.
3. Evaluation: The Node.js interpreter executes the code so that all the previously instantiated entities can get an actual value. Now, running the code starting from the entry point is possible because all the blanks have been filled.

• We could say that Phase 1 is about finding all the dots, Phase 2 connects those does creating paths, and finally Phase 3 walks through the paths in the right order

  Really like this short summary of how to remember the various phases

Modules that modify other modules

This technique, where a module modifies other modules or objects in the global scope, is known as monkey patching. Monkey patching refers to the practice of altering existing objects at runtime to change or extend their behavior, or to apply temporary fixes.

The Role of the TypeScript Compiler

• Module Loading: Will it load a TypeScript file or a pre-compiles JavaScript file?
• Module type and module resolution: What kind of module format does the target system expected. What module type is the loaded file using?
• Output transformation: How will the module syntax be transformed during the output process?
• Compatibility: Can the detected module types interact correctly based on teh syntax transformation?

Chapter 3: Callbacks and Events

setImmediate() gives callbacks lower priority than process.nextTick() or event setTimeout(callback, 0). Callbacks deferred with process.nextTick() are called microtasks and they are executed just after the current operation completes, even before any other I/O event is fired. With setImmediate(), on the other hand, the execution is queued in an event loop that comes after all I/O events have been processed.

• Node.js emits a special event called uncaughtExceptions

Observer Pattern

Reminder we have the following patterns that have been introduced

• Reactor Pattern: The main idea behind this pattern is to have a handler associated with each I/O operation. A handler in Node.js is represented by a callback function.
• Callback Pattern: Functions triggered to handle the result of an operation.
• Observer Pattern: Defines an object (called subject) that can notify an observer (or listeners) when a change in state occurs.

The main difference from the Callback pattern is that the subject can notify multiple observers, while a traditional CPS (Continuation-Passing Style) callback will usually propagate its result to only one listener, the callback.

Important❓

What’s the difference between the Reactor Pattern and Observer Pattern?

In traditional OOP, the Observer pattern requires interfaces, concrete classes, and a hierarchy… In Node.js the Observer pattern is already built into the core and available through the EventEmitter class. The EventEmitter class allows us to register one or more functions as listeners, which will be invoked when a particular event type is fired.

TipProject Idea

Design your own EventEmitter class

When subscribing to observables with a long life span, it is extremely important that we unsubscribe our listeners once they are no longer needed. This allows us to release the memory used by the objects in a listener’s scope and prevent memory leaks. Unreleased EventEmitter listeners are the main source of memory leaks in Node.js (and JavaScript in general).

Chapter 4: Asynchronous Control Flow Patterns with Callbacks

I really like the demonstration of how to solve a race condition in spider example. Spider concurrently downloads all links that it finds on a webpage and downloads it. This can link to race condition if it finds the same link on the page twice. While the function does check if the file is already downloaded it could check that, the event loop might switch to the next function which also checks it and then both functions are now going to download the file.

The key point to solving it is right here

all we need is a variable to mutually exclude spider() tasks running on the same URL

So you can just add the url to a Set and have additional check to that this url isn’t being processed. Removing the downloaded url from the set after the file downloaded is good practice from having it grow indefinitely. The exists() will still catch other future calls to not download the file. In my head I like to think it as a way to indicate that this url is getting being processed.

This might be one of these best chapters I have read in the JS/TS/Node.js ecosystem. I’ll admit though, this callback way of programming is kicking my ass and I am really struggling. Will need to come back to this chapter and reread it few times for to settle. It’s hard for me to articulate what exactly it is I am struggling with. I think the two main things are,

1. You never return a result you always call the callback with an error or the result you want to pass in. This makes it weird because the callback based function doesn’t usually have a return value but instead it’s the callback that has the value.
2. Now this might be more of a recursion problem but I struggle with properly passing data from one callback to itself or another callback. It always seems you need multiple functions to accomplish “one true” function

Chapter 5: Asynchronous Control Flow Patterns with Promises and Async/Await

A promise is an object that represents the eventual result (or error) of an async operation… a Promise is pending when the async operation is not yet complete, it’s fulfilled when the operation successfully completes, and it’s rejected when the operation terminates with an error. Once a Promise is either fulfilled or rejected, it’s considered settled.

Promises are executed in the microtask queue.

As a result of the adoption of the Promises/A+ standard, many Promise implementations, including the native JavaScript Promise API, will consider any object with a then() method a Promise-like object, also called thenable.

The difference with Promise.all() is that Promise.allSettled() will always wait for each Promise to either fulfill or reject, instead of immediately rejecting when one of the promises reject.

Chapter 6: Streams

Use a PassThrough stream when you need to provide a placeholder for data that will be read or written in the future.

The example they gave was you wanted to upload a file that took in a readableStream but you want to do some processing on the file stream before upload e.g. compress or encrypt data. What I didn’t understand about this was why not create the readableStream first then pipe the stream through encrypting the stream or compressing and then call upload at the end? Like why do you have to call the upload first with this placeholder?

• Every time you call createReadStream() from the fs module, this will open a file descriptor every time a new stream is created, even before you start to read from those streams.

TipProject Idea

Design your own lazy Readable and Writeable streams using the PassThrough stream as a proxy until _read() mthod is invoked for the first time.

Chapter 7: Creation Design Patterns

• Factory: Decouples the creation of an object from the specific implementation. My mental model of a factory is a function that returns an instance of a class that is setup for a specific purpose.

• Builder: When I think of a builder pattern I think of method chaining where each method of a class returns (this in JS). Good example of this is drizzle to build SQL queries: db.select().from(users).where(eq(users.id, 42));

DI is a very simple pattern in which the dependencies of a component are provided as input by an external entity, often referred to as the injector

Another pattern, called Inversion of Control, allows us to shift the responsibility of wiring the modules of an application to a third-party entity. This entity can be a service locator & or a DI container

Chapter 8: Structural Design Patterns

Proxy: Control access to an object by standing in for it

Decorator: Dynamically extend or modify and object’s behavior

Adapter: Bridge incompatible interfaces to enable smooth collaboration

Main advantages of a Proxy:

• Data Validation: can validate data before forwarding it to the subject
• Security: can verify client is authorized to perform action before forwarding to subject
• Caching: can keep a cached for frequently access data from the subject
• Lazy initialization: If creating the subject is an expensive operation we can delay the creation until it’s actual needed
• Observability: By intercepting calls to the subject we can log those actions

const evenNumbers = new Proxy([], {
    get: (target, index) => index * ,
    has: (target, number) => number % 2 === 0
})

Looking at the implementation, this proxy uses an empty array at the target and then defines the get and has traps in the handler:

• The get trap intercepts access to the array elements, returning the even number for the given index
• The has trap intercepts the usage of the in operator and checks whether the given number is even or not

Important❓

What I don’t understand about this Proxy object as how do we know that has trap intercepts in operator? Like where is this defined`

• Using Level through the filesystem API I thought was a great example of the Adapter pattern.

Chapter 9: Behavioral Design Patterns

• One good way to remember the Strategy Pattern is that it’s very similar to the Template Pattern with the main difference being implementation differences. The template pattern is easier to remember because Template usually refers to an abstract base class (ABC) that defines the blueprint of the methods that need to be implemented. A subclass of the ABC is similar to a specific strategy.

If you are still struggling to grasp the conceptual difference between an iterable and an iterator, you can think of them this way:

• Iterable: An object that represents a collection of items that you can iterate on
• Iterator: An object that allows you to move from item to the next in a collection

This is very nice way to define the two. I often get the two mixed up because it’s common to see a data type / class to be both an Iterable and Iterator.

Some differences between streams and async iterators:

• Streams are push whereas async iterators are pull
• Streams are ideal for binary data and high-throughput because of built-in buffering and backpressure mechanisms.
• Async Iterators shine in composability and clarity, where data is produces lazily or on demand. They integrate naturally with for await...of

Command pattern is any object that encapsulates all information necessary to perform an action later. The advantages are:

• A command can be scheduled for execution later
• Can be easily serialized and sent over the network.
• Easy to keep a history of all operations executed on a system
• Important part of some algorithm for data synchronization and conflict resolution
• Command scheduled for execution can be canceled or reverted
• Several commands can be grouped together to create atomic transactions

Chapter 10: Testing: Patterns and Best Practices

Test Doubles are stand in components used to isolate the subject under test from its dependencies. The various types of test doubles are:

• Stubs provide predetermined, static responses to method calls. They ignore the context, no matter what input they receive.
• Spies take stubs a step further but retain memory to see how many times a method was called, what arguments were passed each time, the order of sequence calls.
• Mocks are spies with a rulebook. They combine stub like responses with predefined expectations and actively enforce them.

Three types of tests:

• Unit Tests for isolation and validating individual components.
• Integration Tests for ensuring modules collaborate seamlessly.
• E2E Tests for simulating real-world user journeys.

Wanted to callout the Testing async code section of this chapter has a bunch of great examples that can be used as reference.