How To Download Files with curl In Linux Command Line?

How To Download Files with curl In Linux Command Line?

by

The curl tool or command is used to download different files and directories from the command line. The curl is provided by all major Linux distributions like Ubuntu, Debian, Mint, CentOS, etc. The curl command provides a lot of features for different needs of downloading files. One of the most popular features is the ability to download a file via a command-line interface without a GUI which is very useful for system administrators to download files via Terminal, SSH, etc. The wget command is a popular alternative to the curl command.

Install curl

The curl tool or command is not installed by default. In order to use it, it should be installed with the apt install or dnf install command. In order to install Ubuntu, Debian, Mint, and Kali use the following command.

$ sudo apt install curl

The CentOS, Fedora, and RHEL use the following command in order to install curl.

$ sudo dnf install curl

curl Version

The installed curl command version can be printed with the –version option. This will also print the supported protocols list and some required libraries about the curl command.

$ curl --version

The output is like below.

curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.9 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh/0.9.3/openssl/zlib nghttp2/1.41.0 librtmp/2.3
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

curl Help

The curl is a very advanced tool that provides a lot of different options. Help information about curl and these options can be displayed with the -h or –help option like below.

$ curl -h
curl Help

Download File

By default, the downloaded content will be printed to the standard output which is the current terminal, and not stored as a file. The -o options can be used to store as a file where we should also provide the file name. In the following example, we will set the downloaded file name as sample.txt.

$ curl -o sample.txt https://filesamples.com/samples/document/txt/sample1.txt
Download File with curl

We can see that a lot of information about the download is displayed.

  • % Total is the total amount of the data to be retrieved.
  • % Received is the percentage of data retrieved so far.
  • % Xferd is the percentage of data uploaded so far.
  • Average Download Speed
  • Average Upload Speed
  • Time Total
  • Time Spent
  • Time Left
  • Current Speed

Alternatively, the stdout operator can be used to redirect the downloaded content into a file. With the following command, the downloaded file content will be redirected into the file named sample.txt.

$ curl https://filesamples.com/samples/document/txt/sample1.txt > sample.txt

Do Not Show Progress Bar

By default, the progress bar about the download is displayed. Even though this progress bar provides useful information we may want to disable or hide this progress bar and the download will be completed silently. The silent option can be provided with the -s or –silent option like below.

$ curl --silent -o sample.txt https://filesamples.com/txt/sample1.txt

Show Progress Bar As #

By default the progress bar provides information as text but also the progress bar can be displayed as # for downloaded percentage. Also, the downloaded percentage will be displayed as a number.

$ curl -# -o sample.txt  https://filesamples.com/samples/document/txt/sample1.txt
Show Progress Bar As #

Resume Interrupted Download

While downloading big files like movies, media, images, ISO files, etc may can a lot of time. If the download is interrupted for different reasons like system shutdown, network problem, or remote server problem even we can cancel it. We can continue the download from where we left and already downloaded content or data will be preserved and not downloaded again and it will continue where we left. the -C which is the short form of the –continue can be used to resume interrupted download.

$ curl -C -o ubuntu.iso  https://filesamples.com/ubuntu.iso

Download Multiple URLs From File

The xargs command can be used to download multiple URLs from a file. The xargs is used to redirect the content of the file which are URLs to the curl command. In the following example, we download URLs provided via the downloadlist.txt file.

$ xargs -n 1 curl -O < downloadlist.txt

Download with Username and Password

Some protocols like FTP or HTTP may use authentication by using the username and password. The curl supports authentication for the FTP and HTTP protocols. The username and password are provided by using the -u option. 

$ curl -u username:password ftp://linuxtect.com/file.txt

Redirect Downloaded File Content Into Another Command

The curl command is generally used with the -o option where the downloaded file name is provided and the downloaded file content will be put into this file. If not provided the content is printed into the standard output which is the current terminal. We can use this default behavior in order to redirect downloaded content or data into another command by using the bash pipe operators. In the following example, we will redirect the file content into the wc command which will count the characters, words, and sentences in the given data.

$ curl -s  https://filesamples.com/samples/document/txt/sample1.txt | wc

The output is like below where the sentence count is 3, word count is 88 and character count is 607.

3 88 607

Leave a Comment