Js to download file

Got is a Node.js library for making HTTP requests. It has both promise and stream based APIs and in this post I want to explore how to use the stream API to download files.

Using Got

If you use HTTP libraries for making API requests, then the promise method is likely the best for you. Making a basic HTTP request with Got looks like this:

The stream API gives us some extra benefits though. The promise API will load responses into memory until the response is finished before fulfilling the promise, but with the stream API you can act on chunks of the response as they arrive. This makes streams more memory efficient, particularly for large responses.

Streaming a file download with Got

You can create a stream with Got using the method or by setting to in the options.

A Got stream is a duplex stream, which means it is both readable and writable. For the purposes of downloading a file, we will just be using its readable properties.

To download a file we need to send the response to the file system somehow. Streams allow you to pipe the data from one stream to another. To write to the file system we can create a writable stream using the module’s .

To test this out we’ll need a file we can stream. The URL in the following examples is a 500KB gif that you might like.

The simplest way to use a Got stream and write the file to the file system looks like this:

This code creates a Got stream of the image URL and pipes the data to a stream that writes the data into a file called “image.jpg”.

Handling progress and errors

The above code will download the file as long as there are no problems. If an error occurs the code will crash with an unhandled exception. There is also no feedback, so if your file is large you will not see any result until the download is complete. We can listen to events on the stream to handle both of these cases.

Let’s start by rearranging the code above. We’ll get individual handles to the Got stream and the file writer stream.

Now, before we pipe the into the attach some event listeners.

To get feedback on the progress of the download we can listen to the event on the . The event fires with an object with 3 properties:

• : the number of bytes transferred so far
• : the total number of bytes
• : the proportion of the transfer that is complete (between 0 and 1)

If the server you are downloading from doesn’t return a header for the file, then will be undefined and will be 0 until the download is complete.

We can handle errors on both the and by listening for the event. It is good to handle both of these errors as it gives us information over what failed. If the emits an error then there is a problem with the URL, the network or the remote server. If the emits an error then there is a problem with the file system.

For one last piece of feedback, we can also listen to the event on the . This event is fired once all data has been written to the file system.

Let’s complete the above code by adding these events and piping the to the .

If you run the above code in a terminal you will see your download progress logged to the terminal and the image will be downloaded successfully.

Getting fancy with more stream methods

Using streams to download files is more efficient than Got’s promise methods, but the code above has taken a bit of a backward step in terms of developer experience. Rather than dealing with promises, which could be simplified with , we now have to handle events with calbacks.

We can get back to this experience using the Stream module function. takes a number of streams as arguments and pipes the data between them. It also takes a callback function which is called if there is an error within the pipeline or once the pipeline is finished.

This still deals with callbacks, but we can use the Util module’s function to turn it into a promise.

Putting this together, we can simplify the above code to the following:

Or adding in for the final step:

Node streams are cool 😎

Downloading a file is just one use of Node streams, you can find streams popping up all over the place. In this post we used a readable stream to download the file and a writable stream to write it to disk. You can also create readable streams of files and, if you are making requests with Got, you can stream the upload of data too. Objects like , and are streams, as are HTTP requests and responses.

For more on streams, check the Node.js stream documentation and, for more in depth understanding, this guide on backpressuring in streams.