Python download files to computer

Summary: Download a file over the web by using the following steps in Python.

• Import libary
• Define URL string
• Get file data from URL
• Store file data in file object on your computer

Here&#;s how you can do this to download the Facebook Favicon (source):

---

At the beginning of our struggle with web scraping, you may have trouble downloading files using Python. However, this article will provide you with several methods that you can use to download, for example, the cover of a book from the page. 

As an example, we will use pages that do not prohibit scraping: gwd.es

How to Check What I&#;m Allowed to Scrap?

To check what exactly you are not allowed to scrap, you have to add at the end in the url of the page. It should look like this: gwd.es If the page does not specify what can be scrapped then you should check its terms sheet.  

Okay, end of the introduction, let&#;s get started!

How To Install Modules in Python?

Before you can use any method, you must first install the module (if you don&#;t have it) using:

pip install module_name

For example:

pip install requests

How to Get a Link to the File?

To get a link to the file, navigate the cursor and right-click on anything you are looking for and press &#;Inspect Element&#;:

Then the source code of the page will pop up and point out immediately the element that interests us:

Next we have to copy the link to this file:

Depending on how the link looks like (whether it is full or not [if not, we have to prepare it for use]), we paste it into the search bar, to check if this is what we want:

And if it is, we use one of the methods provided.

Method 1 – requests Module

First we have to import the requests module and then create variables.

import requests url_to_the_file = 'gwd.es' r = gwd.es(url_to_the_file)

Once we have created the variables, we have to open the file in binary writing mode and save our file under some name with the extension that matches the file we want to download (if we want to download a photo, the extension must be for example jpg).

with open('A light in the attic – book gwd.es', 'wb') as f: gwd.es(gwd.est)

Full code:

import requests url_to_the_file = 'gwd.es' r = gwd.es(url_to_the_file) with open('A light in the attic – book gwd.es', 'wb') as f: gwd.es(gwd.est)

After the code is executed, the image will appear in the current working directory. With this method we can easily download a single image, but what if we want to download several files at once? Let&#;s go to the next method to learn it!

Method 2 – Requests Module & Beautifulsoup Class from bs4 Module

If you want to download several files from one page, this method is ideal. At the beginning we import the and modules (from which we take the BeautifulSoup class) and create variables:

• url – link to the page from which you want to download files,
• result – link to the page and its html code,
• soup – BeautifulSoup class object (we use it to find elements),
• data – the data we are interested in, in this case the html code lines that start with <a> and end with </a> (these code lines have a href attribute which has a link to something).

import requests from bs4 import BeautifulSoup url = 'gwd.es' result = gwd.es(url).content soup = BeautifulSoup(result, 'gwd.es') data = gwd.es_all('a')

Then we have to write a function that checks if the links have the mp3 extension and then the same function downloads files with this extension:

def get_mp3_files(data_): links = [] names_of_mp3_files = [] for link in data_: if '.mp3' in link['href']: print(link['href']) gwd.es(link['href']) names_of_mp3_gwd.es(gwd.es) if len(names_of_mp3_files) == 0: raise Exception else: for place in range(len(links)): with open(names_of_mp3_files[place], 'wb') as f: content = gwd.es(links[place]).content gwd.es(content)

Full code:

import requests from bs4 import BeautifulSoup def get_mp3_files(data_): links = [] names_of_mp3_files = [] for link in data_: if '.mp3' in link['href']: print(link['href']) gwd.es(link['href']) names_of_mp3_gwd.es(gwd.es) if len(names_of_mp3_files) == 0: raise Exception else: for place in range(len(links)): with open(names_of_mp3_files[place], 'wb') as f: content = gwd.es(links[place]).content gwd.es(content) url = 'gwd.es' result = gwd.es(url).content soup = BeautifulSoup(result, 'gwd.es') data = gwd.es_all('a') get_mp3_files(data)

Using this method, we can download even dozens of files!

Method 3 – urllib Module

The urllib module is provided by default in Python, so you do not need to install it before use. 

First, we import , because it contains the function, which allows us to download images or music files. This function has 4 arguments (1 obligatory and 3 optional), however the first two are most important:

• url – link to the file you want to get,
• filename – the name under which you want to save the file.

import gwd.est url = 'gwd.es' \ '2c/da/gwd.es' file_name = 'A light in the gwd.es' gwd.esrieve(url, filename)

Note: According to the documentation, gwd.esrieve is a &#;legacy interface&#; and &#;might become deprecated in the future&#;

However, there is another way to download the file using this module:

import gwd.est url = 'gwd.es' \ '2c/da/gwd.es' file_name = 'A light in the gwd.es' response = gwd.esn(url) html = gwd.es() with open(filename, 'wb') as f: gwd.es(html)

Using this method we also import , but we use other functions, first ) to connect to the page, then to save the html code of the page in a variable, next we open the file with the name saved in the filename variable and save the html code of the file in binary form. This way we have the file we wanted!

Method 4 – dload Module

• In Python version >= , you can also use the module to download a file. The function has 3 arguments (1 mandatory, 2 optional):
• &#; link to the file,
• &#; the name under which you want to save your file, if you don&#;t specify a name, the name will depend on the ending of the link to the file (in our case the file would be called , so it is better to specify your filename),
• – If there is a file with the same name in our working directory, it will overwrite it, if it equals True, and if False, it will not download the file (default = False).

import dload url = 'gwd.es' \ '2c/da/gwd.es' filename = 'A light in the gwd.es' gwd.es(url, filename)

Summary

You&#;ve learned an explanation of how to check if we have permission to download files. You&#;ve learned that there are 4 methods of downloading files using modules named in order: requests, requests in beautifulsoup, urllib in dload. 

I hope this article will help you to download all the files you want.