A curl command is a tool used to transfer data between different hosts. These hosts are generally called servers where different protocol types are supported by curl. The curl supports protocol types like FTP, FTPS, HTTP, HTTPS, IMAP, SCP, SFTP, SMB, TFTP, etc. Interestingly email protocols lşike IMAP, IMAPS, POP3, etc. are supported by the curl command too. The curl command provides different options and features like proxy configuration, certificate management, authentication, etc. for the data transmission. The data transmission generally occurs as transmitting single or more files and folders.
curl Command Syntax
The curl command has the following syntax where the options are provided after the curl command and the URL is provided as the last parameter.
curl OPTION URL- OPTION is single or more options in order to set different features like proxy, authentication, recursive etc.
- URL is the protocol and path where we want to download data. The protocol can be FTP, SFTP, SCP, HTTP etc. and path is the resource identifier.
Help Information
The curl command provides a lot of features with different options. These options and related information can be displayed with the --help option like below.
$ curl --help
Download with curl
Downloading a file with the curl is very easy where we provide the file URL to the curl command. In the following example, we download a file by using the HTTP protocol. In the following example, we download the file named “presentation.pdf” from linuxtect.com by using the HTTP protocol.
$ curl https://linuxtect.com/presentation.pdfDownload Web Page
The curl command can be also used to download Web pages too. The web pages can be in different names or programming languages. The php, html, aspx pages can be easily downloaded like below.
$ curl https://linuxtect.com/presentation.php
$ curl https://linuxtect.com/presentation.html
$ curl https://linuxtect.com/presentation.aspxDownload File From FTP Server
Another popular usage for the curl command is downloading a file from the FTP server. Everything is the same with the HTTP where the protocol is specified as FTP.
$ curl ftp://linuxtect.com/presentation.pdfDownload Multiple Files with Similar Name
The bash provides glob operations where a single link can be used to download multiple files with a similar name. This is generally useful where the name of the files are the same their names also include numbers. For example “picture1.png” , “picture2.png” etc. These files can be expressed as “picture{1,2}.png” in the terminal.
$ curl "https://linuxtect.com/picture{1,2}.png"Show Download Progress
A file can be in different sizes and the download may take different times to complete according to the file size and internet speed. The progress can be displayed by the curl command which shows the percentage of the downloaded size and simple graphic consist of “#” which shows the completed size. The -# option is used to show the download progress.
$ curl -# "https://linuxtect.com/picture.png"Specify Name For Downloaded File
By default, the downloaded file is saved with its original name. But in some cases, we may need to specify a different name than its original name or the original name can be very ugly. The -o option can be used to specify different than the original name.
$ curl -o "mypicture.png" "https://linuxtect.com/picture.png"Save File with The Save Name As Downloaded File
The -O option is used to save the download file with its original file name which is provided in the URL.
$ curl -O "https://linuxtect.com/picture.png"Resume Interrupted/Previous Download
Another powerfull and useful feature of the curl command is the ability to remove interrupted or previous downloads. If the downloaded file size is very high or the internet speed is low the download can take a lot of time. If the download is interruppted intentiaonally or accidently the -C option can be used to resume download.
$ curl -C "https://linuxtect.com/bigfile.zip"Limit Speed/Bandwith Rate
The curl command try to use all existing bandwith during the download process. This may create some problems for other applications or internet users in the same network. The curl command provides the --limit-rate method in order to limit download rate. The download rate can be provided as Kbps or kilobitpersecond by using the K unit specifier. In the following example we set the download limit rate as 500 Kbps.
$ curl --limit-rate 500K "https://linuxtect.com/bigfile.zip"FTP Authentication with Username and Password
Some protocols like FTP can use aser authentication in order to access FTP resources. Generally a username and password is used for FTP user authentication. The -u option is used to specify the username and password. The username password specified as USERNAME:PASSWORD format. In the following example we use the username ismail and password mypass to authentication FTP server by using curl command and download file.
$ curl -u ismail:mypass "ftp://linuxtect.com/bigfile.zip"Upload File To FTP Server
The curl command not only used to download file from the FTP server. It can be also used to upload file to the remote FTP server. The syntax of the curl FTP upload is like below.
curl -T FILE_NAME FTP_PATH- FILE_NAME is the name of the file we want to upload. The FILE_NAME can also contain the path if the file is different than the current working directory.
- FTP_PATH is the remote FTP server information with the FTP path we want to put the file.
In the following example we put the file named backup.zip into the FTP server.
$ curl -T backup.zip "ftp://linuxtect.com/ismail/"Specify Proxy
The proxy is used to to access other networks or internet in a controlled and secure way. Some organizations use proxies for the internet access. The curl command can be with proxy where generally a username and password is used to connect proxy and authenticate users personally. The -x option is used to specify the proxy IP address or hostname with port information.
$ curl -x 192.168.1.10:8080 "https://linuxtect.com/bigfile.zip"Also a user can authenticate to user a proxy with the -u option with the USERNAME:PASSWORD format. In the following example we authentication for a proxy with a username password by using curl command.
$ curl -x 192.168.1.10:8080 -u ismail:mypass "https://linuxtect.com/bigfile.zip"