The curl is a command-line tool that is used to download and upload files from the command line without any GUI. Even the curl is a cross-platform tool it is mainly designed and used in Linux distributions like Ubuntu, Mint, Debian, Kali, CentOS. Proxy is an intermediate system in a network that is used to access another part of the network or the internet. The curl command can be also used with proxy but we should provide proxy-related configuration with parameters. In this tutorial, we will learn how to use the curl command for a different type and configuration proxy systems.
Use curl with HTTP Proxy
We will start with a basic curl proxy usage example. The proxy IP address and the proxy port number are provided with the -x or –proxy parameter. In the following example, we will use the proxy with IP address 192.168.100.150 and port number 8080.
$ curl -x 192.168.100.150:8080 http://linuxtect.com
If the DNS is configured to solve the proxy server name into the IP address we may provide the proxy server hostname or domain name like below.
$ curl -x proxy.test.com:8080 http://linuxtect.com
Debug curl Proxy Communication
Proxy usage with the curl command can be problematic. In order to solve the problems, we should get detailed information about the problem. the detailed information can get with a verbose output or debugging the connection with the curl command -v parameter like below.
$ curl -v -x 192.168.100.150:8080 https://linuxtect.com
Use curl with HTTP Proxy and HTTPS URL
HTTPS is the secure and encrypted version of the HTTP protocol. The proxy can be used with the HTTPS protocol where the target URL will be an HTTPS URL. Actually there no change in usage of the curl command and proxy setting where only the target URL will be HTTPS like below.
$ curl -x 192.168.100.150:8080 https://linuxtect.com
Use curl with HTTP Proxy For FTP Protocol
HTTP proxy means providing the proxy for the HTTP protocol. But HTTP proxies can be used for other protocols like HTTPS, FTP etc. We can use the HTTP proxy for the FTP like below.
$ curl -x 192.168.100.150:8080 ftp://linuxtect.com/file.txt
Use curl with HTTP Proxy Tunelling
The curl proxy HTTP tunnelling can be done with the -p parameter like below.
$ curl -p -x 192.168.100.150:8080 http://linuxtect.com
Use curl with SOCKS ( SOCKS4 , SOCKS5 ) Proxy
Up to now, we have worked with the HTTP proxies but there are alternative protocols for proxying. SOCKS is a popular protocol used to communicate with the proxy. Recently the SOCKS4 and SOCKS5 are popular versions of the SOCKS. The curl can also support SOCKS with SOCKS4 and SOCKS5. The SOCKS protocol usage does not require a port number as we can see below.
$ curl -x socks4://192.168.100.150 http://linuxtect.com $ curl --socks4 192.168.100.150 http://linuxtect.com
$ curl -x socks4a://192.168.100.150 http://linuxtect.com $ curl --socks4a 192.168.100.150 http://linuxtect.com
$ curl -x socks5://192.168.100.150 http://linuxtect.com $ curl --socks5 192.168.100.150 http://linuxtect.com
$ curl -x socks5h://192.168.100.150 http://linuxtect.com $ curl --socks5-hostname 192.168.100.150 http://linuxtect.com
Use curl with Proxy Authentication (Username and Password)
Normally proxies can be used without any authentication but some proxies may require authentication with the username and password. The curl command supports the authentication for proxy using different authentication mechanisms like NTLM. The -U parameter is used with the username and password like ismail:pass123 .
$ curl -U ismail:pass123 -x 192.168.100.150:8080 http://linuxtect.com
If the proxy requires NTLM based authantiation the –proxy-ntlm parameter should be provided to the curl command.
$ curl -U ismail:pass123 --proxy-ntlm -x 192.168.100.150:8080 http://linuxtect.com
Set Proxy Configuration As Environment Variables
The curl command can read some proxy configuration from the environment variables. The following environment variable line can be used to enable proxy for curl and set the proxy server and port number.
Alternatively the IP address can be used like below.
Disable curl Proxy Configuration
The curl proxy can be configured in the curl configuration file or in the environment variables. This makes usage of the curl easier and less typing. But in some cases, we may need to disable the current proxy configuration for curl and connect directly. The –noproxy parameter can be used to disable the currently existing proxy configuration.
$ curl --noproxy http://linuxtect.com