VPS

What is a Curl Command And How to Use It?

What is a Curl command? It’s a command available on most Unix based systems.  This stands as an abbreviation to “Client URL”. Curl commands are designed to work as a way to check connectivity to URLs and a as a great tool to transfer data. Let’s learn how you can start using it.

Curl command supports the below list of protocols:

  • HTTP and HTTPS
  • FTP and FTPS
  • IMAP and IMAPS
  • POP3 and POP3S
  • SMB and SMBS
  • SFTP
  • SCP
  • TELNET
  • GOPHER
  • LDAP and LDAPS
  • SMTP and SMTPS

These are the most important supported protocols, but there are some others as well. Curl is powered by libcurl, which is a free URL transfer library at the client side

Check Curl Version:

Before we begin working with Curl, we need to log into our VPS. If you need help, check out this tutorial about SSH.

First let’s check what version of Curl is available with the following command:

curl --version

The output will show the Curl version a list of supported protocols. Now we can look at some Curl command examples

Basic Curl Command Syntax

Lets learn how to use Curl commands. The basic syntax of Curl looks like this:

curl [OPTIONS] [URL]

The simplest use of Curl is to display the contents of a page. The below example will render the home web page of testdomain.com.

curl testdomain.com

This will render the complete source code of the homepage for the domain. If no protocol is specified curl will interpret this to HTTP.

Curl Command File options

Curl commands can download files from a remote location. You can do it in two different ways:

  • -O will save the file in the current working directory with the same file name as remote.
  • -o lets you specify a different file name or location

An example of this is as shown below:

curl -O http://testdomain.com/testfile.tar.gz

The above command will save this as testfile.tar.gz.

curl -o newtestfile.tar.gz http://testdomain.com/testfile.tar.gz

The above command will save this as newtestfile.tar.gz.

If for some reason, the download gets interrupted, you can resume it using curl. You can do this with the following command:

curl -C - -O  http://testdomain.com/testfile.tar.gz

Using curl, we can also download multiple files, like shown bellow:

curl -O http://testdomain.com/testfile.tar.gz -O http://mydomain.com/myfile.tar.gz

If you want to download multiple files from multiple URL, list all of them in a file. Curl commands can be combined with xargs to download the different URLs.

For instance, if we have a file allUrls.txt which contains a list of all URLs to be downloaded, then the below example can be used to download all files.

xargs –n 1 curl -O < allUrls.txt

Curl Commands for Http

Curl can also be used when there is a proxy server. If you are behind a proxy server listening on port 8090 at sampleproxy.com, download the files as shown below:

curl -x  sampleproxy.com:8090 -U username:password -O http:// testdomain.com/testfile.tar.gz

In the above example, you can skip -U username:password if the proxy does not require an authentication method.

A typical HTTP request will always contain a header. The HTTP header sends additional information about the remote web server along with the actual request. While through a browser’s developer tools you can check the header information, you can verify it using a curl command.

Below is an example of how to retrieve header information from a website.

curl -I www.testdomain.com

Using curl, you can make a GET and a POST request. A GET request will be as follows:

curl http://mydomain.com

A sample of a POST request will be as shown below:

curl –data “text=Hello” https://myDomain.com/firstPage.jsp

Over here text=Hello is the POST request parameter. This behavior would be similar to HTML forms.

You can also specify multiple HTTP methods in a single curl command. Do this by using –next option, like this:

curl –data “text=Hello” https://myDomain.com/firstPage.jsp --next https://myDomain.com/displayResult.jsp

This contains a POST request followed by a GET request.

Every HTTP request will have a user agent which is sent as part of the request. This indicates the web browser details of the client. By default, a curl request contains curl and the version number as the user agent details. A sample output is as shown below:

“GET / HTTP/1.1” 200 “_” ”curl/7/29/0”

You can change this default user agent information using the below command:

curl -I http://mydomain.com –-user-agent “My new Browser”

Now the changed output will be:

“GET / HTTP/1.1” 200 “_” ”My new Browser”

Curl for Cookies

Curl commands can be used to check what cookies get downloaded on any URL. So, if you are accessing https://www.samplewebsite.com, then you can output to a file, save the cookies and access them using cat or a VM editor.

Below is a sample of such command:

curl --cookie-jar Mycookies.txt https://www.samplewebsite.com /index.html -O

Similarly, if you have the cookies in a file, then you can send it to the website. An example of such command is as shown below:

curl --cookie Mycookies.txt https://www. samplewebsite.com

Curl for FTP

Curl command supports FTP! You can use them to download files from a remote server.

curl -u username:password -O ftp://sampleftpserver/testfile.tar.gz

In the above command, ftp://sampleftpserver is an FTP server which accepts connections. The username and password can be skipped for anonymous FTP connections. Type in the command and watch the progress bar fill up.

You can upload files as well with the command bellow:

curl -u username:password -T testfile.tar.gz ftp://sampleftpserver

Again, we can skip the username and password for anonymous FTP connections.

Limiting Curl Output

While using a curl you can’t know how big the output will be. You can restrict bandwidth, to make sure it’s not throttled by curl.

The below command restricts the bandwidth to 100K:

curl --limit-rate 100K http://testdomain.com/samplefile.tar.gz -O

Conclusion

Curl is a powerful and widely used command. It’s useful when you’re dependent on the command line. It has several options and supports multiple protocols. That’s a pretty great reason to learn this command!

Remember, if you want to learn some advanced commands, simply refer to the manual that should be in all Unix versions:

man curl

We hope this tutorial gave you a good jumping off point for using Curl! How will you use this command? Let us know in the comments!

 

About the author

Edward S.

Edward is Hostinger's copywriter. He's an expert communicator with years of experience in IT as a writer, marketer, and Linux enthusiast. IT is a core pillar of his life, personal and professional. Edward's goal is to encourage millions to achieve an impactful online presence. He also really loves dogs, guitars, and everything related to space.

Add Comment

Click here to post a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Get Exclusive Content

Join thousands of webmasters who get our free newsletter with tips on how to drive more traffic and revenue to their websites!

Please wait...

Thank you for sign up!

More in VPS
How to Install Laravel on Ubuntu 18.04 with Apache
Close