Introduction
Transferring data to and from a server requires tools that support the necessary network protocols. Linux has multiple tools created for this purpose, the most popular being curl and wget.
This tutorial will show you how to use the curl
command and provide you with an exhaustive list of the available options.
Prerequisites
- Access to the terminal
- Internet access
- curl installed
Note: If you do not have curl installed, install it by typing the following in the terminal:
sudo apt install curl
What Is the curl Command?
curl
(short for “Client URL”) is a command line tool that enables data transfer over various network protocols. It communicates with a web or application server by specifying a relevant URL and the data that need to be sent or received.
curl
is powered by libcurl, a portable client-side URL transfer library. You can use it directly on the command line or include it in a script. The most common use cases for curl
are:
- Downloading files from the internet
- Endpoint testing
- Debugging
- Error logging
curl Syntax
The basic curl
syntax is as follows:
curl [options/URLs]
For example:
curl https://www.gnu.org/gnu/gnu.html
The system outputs the HTML contents found on the URL provided after the curl
command.
If you specify a URL that leads to a file, you can use curl
to download the file to your local system:
curl [url] > [local-file]
The progress bar shows how much of the file has been downloaded so far.
The syntax of URLs that are part of the command depends on the protocol. Multiple URLs that differ in one part are written together using braces:
http://example.{first,second,third}.com
Alphanumeric series are written with brackets:
ftp://ftp.url.com/file[1-100].txt
While nested sequences are not supported, multiple sequences are allowed:
http://url.com/archive[2010-2020]/vol[1-4]/part{a,b,c}.html
curl Protocols
curl supports numerous protocols for data transfer. Find the complete list below.
Protocol | Description |
---|---|
DICT | A dictionary network protocol for querying dictionary servers about the meanings of words. |
FILE | A URL scheme for obtaining a file from the local file system using curl. |
FTP, FTPS | File Transfer Protocol, used for file transfers between clients and servers. FTPS is the version of the same protocol with an added SSL/TLS security layer. |
GOPHER, GOPHERS | An old protocol for searching, retrieving, and distributing internet documents, a precursor of HTTP. GOPHERS is the version of the same protocol with an added SSL/TLS security layer. |
HTTP, HTTPS | Hypertext Transfer Protocol, used for web and internet data transfer. HTTPS is the version of the same protocol with an added SSL/TLS security layer. |
IMAP, IMAPS | Internet Message Access Protocol, used for email access and management. IMAPS is the version of the same protocol with an added SSL/TLS security layer. |
LDAP, LDAPS | Lightweight Directory Access Protocol, used for distributed directory information access and management. LDAPS is the version of the same protocol with an added SSL/TLS security layer. |
MQTT | Message Queuing Telemetry Transport – a protocol for data exchange between small devices, usually IoT systems. |
POP3, POP3S | Post Office Protocol version 3 – a protocol for email retrieval from a server. POP3S is the version of the same protocol with an added SSL/TLS security layer. |
RTMP | Real-Time Messaging Protocol – a streaming protocol for audio, video, and other data. |
RTSP | Real Time Streaming Protocol, used for streaming media servers management. |
SCP | Secure Copy – a protocol for copying files to and from an SSH server. |
SFTP | SSH File Transfer Protocol – a version of the File Transfer Protocol using the SSH connection. |
SMB, SMBS | Server Message Block – a protocol for managing shared access to files and computer peripherals. SMBS is the version of the same protocol with an added SSL/TLS security layer. |
SMTP, SMPTS | Simple Mail Transfer Protocol – an email protocol for easy transmission of email. SMTPS is the version of the same protocol with an added SSL/TLS security layer. |
TELNET | An application layer protocol for bidirectional interactive text-oriented communication. |
TFTP | Trivial File Transfer Protocol, used for uploading or downloading files to or from a remote host. |
curl Command Options
curl
accepts a wide array of options, which makes it an extremely versatile command. Options start with one or two dashes. If they do not require additional values, the single-dash options can be written together. For example, the command that utilizes the -O
, -L
, and -v
options can be written as:
curl -OLv [url]
The list of all the available options is given below.
Option | Description |
---|---|
--abstract-unix-socket <path> |
Connect through abstract Unix socket instead through a network.
Example: |
--alt-svc <file name> | Enable alt-svc parser.
Example: |
--anyauth | Curl finds and uses the most secure authentication method for the given HTTP URL.
Example: |
-a, --append | Append to the target file.
Example: |
--aws-sigv4 <provider1[:provider2[:region[:service]]]> | Use AWS V4 signature authentication.
Example: |
--basic | Use HTTP basic authentication.
Example: |
--cacert <file> | Use the specified file for certificate verification.
Example: |
--capath <dir> | Use the specified directory to look for the certificates.
Example: |
--cert-status | Verify the server certificate status.
Example: |
--cert-type <type> | Specify the type of the provided certificate. The recognized types are PEM (default), DER , ENG and P12 .
Example: |
-E, --cert <certificate[:password]> | Use the provided certificate file when working with a SSL-based protocol.
Example: |
--ciphers <list of ciphers> | Provide ciphers to be used in the connection.
Example: |
--compressed-ssh | Enable built-in SSH compression.
Example: |
--compressed | Request to receive a compressed response.
Example: |
-K, --config <file> | Provide a text file with curl arguments, instead of writing them on the command line.
Example: |
--connect-timeout <fractional seconds> | Specify maximum time a curl connection may last.
Example: |
--connect-to <HOST1:PORT1:HOST2:PORT2> | Provide a connection rule to direct requests at a specific server cluster node.
Example: |
-C, --continue-at <offset> | Resume file transfer at the offset specified.
|
-c, --cookie-jar <filename> | Specify a file for storing cookies.
|
-b, --cookie <data|filename> | Read cookies from a file.
Example: |
--create-dirs | Create the local directories for the --output option.
Example: |
--create-file-mode <mode> | Specify which mode to set upon file creation.
Example: |
--crlf | Convert LF to CRLF.
|
--crlfile <file> | Provide a Certificate Revocation List for peer certificates.
|
--curves <algorithm list> | Provide curves for establishing an SSL session.
Example: |
--data-ascii <data> | See -d , --data .
Example: |
--data-binary <data> | Post data as specified, without extra processing.
|
--data-raw <data> | Same as -d , --data , but the @ character is not treated differently from the rest.
|
--data-urlencode <data> | Same as -d , --data , but perform URL encoding.
Example: |
-d, --data <data> | Send data to a HTTP server in a POST request.
Example: |
--delegation <LEVEL> | Specify when the server is allowed to delegate credentials.
Example: |
--digest | Enable HTTP Digest authentication.
Example: |
--disable-eprt | Disable EPRT and LPRT commands for active FTP transfers.
Example: |
--disable-epsv | Disable EPSV for passive FTP transfers.
Example: |
-q, --disable | Disable the reading of the curlrc config file.
|
--disallow-username-in-url | Exit if provided with a URL that contains a username.
Example: |
--dns-interface <interface> | Specify an interface for outgoing DNS requests.
Example: |
--dns-ipv4-addr <address> | Specify an IPv4 address from which the DNS requests will come.
Example: |
--dns-ipv6-addr <address> | Specify an IPv6 address from which the DNS requests will come.
Example: |
--dns-servers <addresses> | Specify your own list of DNS servers.
Example: |
--doh-cert-status | --cert-status for DNS-over-HTTPS.
Example: |
--doh-insecure | -k , --insecure for DoH.
Example: |
--doh-url <URL> | Specify a DoH server for hostname resolution.
Example: |
-D, --dump-header <filename> | Specify a file for writing protocol headers.
Example: |
--egd-file <file> | Provide a path for the EGD socket.
Example: |
--engine <name> | Specify an OpenSSL crypto engine.
Example: |
--etag-compare <file> | Request an ETag read from a file.
Example: |
--etag-save <file> | Save a HTTP ETag to a file.
Example: |
--expect100-timeout <seconds> | Maximum wait time for a 100-continue response.
Example: |
--fail-early | Tell curl to fail and exit when it detects the first error in transfer.
Example: |
--fail-with-body | If the server returns an error with code 400 or greater, curl saves the content and returns error 22.
Example: |
-f, --fail | If the server returns an error, curl fails silently and returns error 22.
Example: |
--false-start | Use false start on TLS handshake.
Example: |
--form-string <name=string> | Similar to -F , --form , but the value strings are processed literally.
Example: |
-F, --form <name=content> | Emulate a form with a Submit button that has been pressed. The @ sign forces the content to be a file. The < sign extracts only the content part of the file.
Example: |
--ftp-account <data> | Specify the account data for the FTP server.
Example: |
--ftp-alternative-to-user <command> | Specify the command to be sent if the username and password authentication fails.
Example: |
--ftp-create-dirs | If the specified directory does not exist, curl will attempt to create it.
Example: |
--ftp-method <method> | Specify a method to be used for obtaining files over FTP. Available methods are multicwd , nocwd , and singlecwd .
Example: |
--ftp-pasv | Use passive data connection mode.
Example: |
-P, --ftp-port <address> | Reverse the default roles for the FTP connection.
Example: |
--ftp-pret | Send the PRET command before PASV/EPSV.
Example: |
--ftp-skip-pasv-ip | Do not use the IP address suggested by the server. curl will use the control connection IP.
Example: |
--ftp-ssl-ccc-mode <active/passive> | Set the Clear Command Channel (CCC) mode.
Example: |
--ftp-ssl-ccc | After the authentication is complete, the SSL/TLS layer is eliminated, allowing for unencrypted communication.
Example: |
--ftp-ssl-control | Use SSL/TLS for logging in, stop the encryption when the data transfer starts.
Example: |
-G, --get | Use HTTP GET request instead of POST.
Example: |
-g, --globoff | Disable the URL globbing parser.
Example: |
--happy-eyeballs-timeout-ms <milliseconds> | Use the Happy Eyeballs algorithm for connecting to dual-stack hosts.
Example: |
--haproxy-protocol | Use HAProxy PROXY protocol v1 header.
Example: |
-I, --head | Obtain only headers.
Example: |
-H, --header <header/@file> | Specify an additional header to be sent in the HTTP request.
Example: |
-h, --help <category> | See help for a specific category. all lists all the available options.
Example: |
--hostpubmd5 <md5> | Pass a 32-digit hexadecimal string.
Example: |
--hsts <file name> | Enable HSTS.
Example: |
--http0.9 | Accept a HTTP version 0.9 response.
Example: |
-0, --http1.0 | Use HTTP version 1.0.
Example: |
--http1.1 | Use HTTP version 1.1.
Example: |
--http2-prior-knowledge | Use HTTP version 2.0. Use this option if you know that the server supports this HTTP version.
Example: |
--http2 | Attempt to use HTTP version 2.0.
Example: |
--http3 | Use HTTP version 3.0. This is an experimental option.
Example: |
--ignore-content-length | Ignore the Content-Length header.
Example: |
-i, --include | Specify that the output should include the HTTP response headers.
Example: |
-k, --insecure | Allow curl to work with insecure connections.
Example: |
--interface <name> | Specify the interface for performing an action.
Example: |
-4, --ipv4 | Only resolve names to IPv4 addresses.
Example: |
-6, --ipv6 | Only resolve names to IPv6 addresses.
Example: |
-j, --junk-session-cookies | Discard session cookies.
Example: |
--keepalive-time <seconds> | Specify the idle time for the connection before it sends keepalive probes.
Example: |
--key-type <type> | Specify the type of the private key. Available types are PEM (default), DER , and ENG .
Example: |
--key <key> | Specify the file containing the private key.
Example: |
--krb <level> | Enable and use Kerberos authentication. Available levels are clear , safe , confidential , and private (default).
Example: |
--libcurl <file> | Obtain C source code for the specified command line operation.
Example: |
--limit-rate <speed> | Specify the maximum upload and download transfer rate.
Example: |
-l, --list-only | Force a name-only view.
Example: |
--local-port <num/range> | Specify the port numbers to be used for the connection.
Example: |
--location-trusted | Similar to -L , --location , but enables you to send name and password to all redirections.
Example: |
-L, --location | Allow curl to follow any redirections.
Example: |
--login-options <options> | Specify the login options for email server authentication.
Example: |
--mail-auth <address> | Provide a single address as the identity.
Example: |
--mail-from <address> | Provide a single “from” address.
Example: |
--mail-rcpt-allowfails | Allows curl to continue with SMTP conversation if one of the recipients fails.
Example: |
--mail-rcpt <address> | Provide a single “to” address.
Example: |
-M, --manual | Read the curl manual.
Example: |
--max-filesize <bytes> | Provide the maximum size of the file to be downloaded. Example: curl --max-filesize 500K https://example.com |
--max-redirs <num> | Specify the maximum number of redirections when --location is active.
Example: |
-m, --max-time <fractional seconds> | Specify the maximum time for an operation.
Example: |
--metalink | Specify a metalink resource. This option is disabled in the newest versions of curl.
Example: |
--negotiate | Enable SPNEGO authentication.
Example: |
--netrc-file <filename> | Like --n , --netrc , but allows you to specify the file to be used.
Example: |
--netrc-optional | Like --n , --netrc , but using netrc is optional.
Example: |
-n, --netrc | Search the netrc file for login information.
Example: |
-:, --next | Use the option to separate URL requests.
Example: |
--no-alpn | Disable ALPN TLS extension.
Example: |
-N, --no-buffer | Disable output stream buffer.
Example: |
--no-keepalive | Disable keepalive messages.
Example: |
--no-npn | Disable NPN TLS extension.
Example: |
--no-progress-meter | Disable the progress bar but display any other message.
Example: |
--no-sessionid | Disable the caching of SSL session-ID.
Example: |
--noproxy <no-proxy-list> | List the hosts which should not use a proxy.
Example: |
--ntlm-wb | Like --ntlm , but also hands authentication to ntlmauth.
Example: |
--ntlm | Enable NTLM authentication.
Example: |
--oauth2-bearer <token> | Provide a Bearer Token for OAUTH 2.0.
Example: |
--output-dir <dir> | Specify the output file directory.
Example: |
-o, --output <file> | Store output in a file. The output is not shown in stdout.
Example: |
--parallel-immediate | Prefer parallel connections to waiting for new connections or multiplexed streams.
Example: |
--parallel-max <num> | Specify the maximum number of parallel connections.
Example: |
-Z, --parallel | Perform transfers in parallel.
Example: |
--pass <phrase> | Specify a private key passphrase.
Example: |
--path-as-is | Prevent curl from merging /./ and /../ sequences.
Example: |
–-pinnedpubkey <hashes> | Specify a public key for curl to use.
Example: |
--post301 | Prevent curl from converting POST to GET requests after a 301 redirection.
Example: |
--post302 | Prevent curl from converting POST to GET requests after a 302 redirection.
|
--post303 | Prevent curl from converting POST to GET requests after a 303 redirection.
Example: |
--preproxy [protocol://]host[:port] | Use the SOCKS proxy as a pre-proxy.
Example: |
-#, --progress-bar | Use the simple progress bar.
Example: |
--proto-default <protocol> | Specify which protocol curl should use for URLs without a scheme name.
Example: |
--proto-redir <protocols> | Specify which protocols curl should use on redirection.
Example: |
--proto <protocols> | Specify which protocols curl should use for transfers.
Example: |
--proxy-anyauth | Curl should choose an appropriate authentication method.
Example: |
--proxy-basic | Use HTTP Basic for communication with a proxy.
Example: |
--proxy-cacert <file> | --cacert for HTTPS proxies.
Example: |
--proxy-capath <dir> | --capath for HTTPS proxies.
Example: |
--proxy-cert-type <type> | --cert-type for HTTPS proxies.
Example: |
--proxy-cert <cert[:passwd]> | -E , --cert for HTTPS proxies.
Example: |
--proxy-ciphers <list> | --ciphers for HTTPS proxies.
|
--proxy-crlfile <file> | --crlfile for HTTPS proxies.
Example: |
--proxy-digest | Use HTTP Digest authentication with a proxy.
Example: |
--proxy-header <header/@file> | -H , --header for proxy communication.
Example: |
--proxy-insecure | -k , --insecure for HTTPS proxies.
Example: |
--proxy-key-type <type> | --key-type for HTTPS proxies.
Example: |
--proxy-key <key> | --key for HTTPS proxies.
Example: |
--proxy-negotiate | --negotiate for proxy communication.
Example: |
--proxy-ntlm | Use HTTP NTLM authentication with a proxy.
Example: |
--proxy-pass <phrase> | --pass for HTTPS proxies.
Example: |
--proxy-pinnedpubkey <hashes> | Specify the public key for proxy verification.
Example: |
--proxy-service-name <name> | Specify the service name for proxy communciation.
Example: |
--proxy-ssl-allow-beast | --ssl-allow-beast for HTTPS proxies.
Example: |
--proxy-ssl-auto-client-cert | --ssl-auto-client-cert for HTTPS proxies.
Example: |
--proxy-tls13-ciphers <ciphersuite list> | Specifies the list of cipher suites to use in negotiating TLS 1.3 for proxies.
Example: |
--proxy-tlsauthtype <type> | --tlsauthtype for HTTPS proxies.
Example: |
--proxy-tlspassword <string> | --tlspassword for HTTPS proxies.
Example: |
--proxy-tlsuser <name> | --tlsuser for HTTPS proxies.
Example: |
--proxy-tlsv1 | -1 , --tlsv1 for HTTPS proxies.
Example: |
-U, --proxy-user <user:password> | Specify the username and password for authenticating with a proxy.
Example: |
-x, --proxy [protocol://]host[:port] | Specify a proxy to use.
Example: |
--proxy1.0 <host[:port]> | Specify a HTTP 1.0 proxy to use.
Example: |
-p, --proxytunnel | Create a proxy tunnel.
Example: |
--pubkey <key> | Provide a file containing a public key.
Example: |
-Q, --quote <command> | Send a command to a FTP or SFTP server, to be executed before the transfer.
Example: |
--random-file <file> | Specify a file containing random data. This file will be used for seeding the random engine.
Example: |
-r, --range <range> | Obtain a range of bytes.
Example: |
--raw | Disable HTTP content decoding and obtain raw data.
Example: |
-e, --referer <URL> | Send Referrer Page information.
Example: |
-J, --remote-header-name | Use header name specified by the server instead of obtaining it from the URL.
Example: |
--remote-name-all | Apply the -O , --remote-name option to all the URLs.
Example: |
-O, --remote-name | Specify that the local file should have the name of the remote file that was downloaded.
Example: |
-R, --remote-time | Specify that the local file should have the timestamp of the remote file that was downloaded.
Example: |
--request-target <path> | Specify an alternative target path.
Example: |
-X, --request <command> | Specify a request method for communication with the server.
Example: |
--resolve <[+]host:port:addr[,addr]...> | Specify a custom address for a host/port.
Example: |
--retry-all-errors | Force retrying on all errors.
Example: |
--retry-connrefused | Add ECONNREFUSED to the list of errors that are eligible for --retry .
Example: |
--retry-delay <seconds> | Specify the amount of time between retries.
Example: |
--retry-max-time <seconds> | Specify the maximum amount of time for --retry attempts.
Example: |
--retry <num> | Specify the number of retries after curl encounters and error.
Example: |
--sasl-authzid <identity> | Specify an additional authentication identity for SASL PLAIN authentication.
Example: |
--sasl-ir | Enable initial response during SASL authentication.
Example: |
--service-name <name> | Specify the SPNEGO service name.
Example: |
-S, --show-error | Show an error message event with the -s , --silent option enabled.
Example: |
-s, --silent | Turn on the silent mode. This option mutes curl.
Example: |
--socks4 <host[:port]> | Specify a SOCKS4 proxy.
Example: |
--socks4a <host[:port]> | Specify a SOCKS4a proxy.
Example: |
--socks5-basic | Use the basic authentication method (username/password) with a SOCKS5 proxy.
Example: |
--socks5-gssapi-nec | Allow protection mode negotiation to be unprotected.
Example: |
--socks5-gssapi-service <name> | Change the name of a socks server.
Example: |
--socks5-gssapi | Use GSS-API authentication with a SOCKS5 proxy.
Example: |
--socks5-hostname <host[:port]> | Specify the SOCKS5 proxy to use.
Example: |
--socks5 <host[:port]> | Specify the SOCKS5 proxy to use. The hostname is resolved locally.
Example: |
-Y, --speed-limit <speed> | Set the lower limit for the download speed.
Example: |
-y, --speed-time <seconds> | Set the time period for the speed limit measurement.
Example: |
--ssl-allow-beast | Tell curl to ignore the BEAST security flaw in the SSL3 and TLS1.0 protocols.
Example: |
--ssl-auto-client-cert | Obtain and use a client certificate automatically.
Example: |
--ssl-no-revoke | Do not check for certificate revocation.
Example: |
--ssl-reqd | Require SSL/TLS.
Example: |
--ssl-revoke-best-effort | Ignore certificate revocation checks if they failed because of missing distribution points.
Example: |
--ssl | Attempt to use SSL.
Example: |
-2, --sslv2 | Use SSLv2. Newer curl versions ignore this request due to security concerns with SSLv2.
Example: |
-3, --sslv3 | Use SSLv3. Newer curl versions ignore this request due to security concerns with SSLv3.
Example: |
--stderr <file> | Output stderr to a file. The - symbol tells curl to output stderr to stdout.
Example: |
--styled-output | Enable bold fonts for HTTP header terminal output.
|
--suppress-connect-headers | Prevent curl from outputting CONNECT headers.
Example: |
--tcp-fastopen | Enable TCP Fast Open.
Example: |
--tcp-nodelay | Enable TCP_NODELAY.
Example: |
-t, --telnet-option <opt=val> | Pass the TTYPE , XDISPLOC , and NEW_ENV options to the telnet protocol.
Example: |
--tftp-blksize <value> | Set the value of TFTP BLKSIZE. Must be a value larger than 512.
Example: |
--tftp-no-options | Prevents curl from sending requests for TFTP options.
Example: |
-z, --time-cond <time> | Request a document that was modified after a certain date and time. For documents modified before the time, prefix the date expression with a dash. Example: curl -z "Wed 01 Sep 2021 12:18:00" https://example.com |
--tls-max <VERSION> | Specify the newest TLS version that is supported.
Example: |
--tls13-ciphers <ciphersuite list> | Specifies the list of cipher suites to use in negotiating TLS 1.3
Example: |
--tlsauthtype <type> | Specify the TLS authentication type.
Example: |
--tlspassword <string> | Specify the TLS password.
Example: |
--tlsuser <name> | Specify the TLS username.
Example: |
--tlsv1.0 | Tell curl to use TLS1.0 or newer.
Example: |
--tlsv1.1 | Tell curl to use TLS1.1 or newer.
Example: |
--tlsv1.2 | Tell curl to use TLS1.2 or newer.
Example: |
--tlsv1.3 | Tell curl to use TLS1.3 or newer.
Example: |
-1, --tlsv1 | Specify that curl should use at least 1.x version of TLS.
Example: |
--tr-encoding | Ask for a compressed Transfer-Encoding response.
Example: |
--trace-ascii <file> | Enable a full trace dump to a file. Eliminates the hex part and shows only ASCII.
Example: |
--trace-time | Require a time stamp on each trace or verbose line.
Example: |
--trace <file> | Enable a full trace dump to a file.
Example: |
--unix-socket <path> | Specify a Unix socket path.
Example: |
-T, --upload-file <file> | Upload a file to the URL.
Example: |
--url <url> | Provide a URL to be fetched.
Example: |
-B, --use-ascii | Enable ASCII transfer.
Example: |
-A, --user-agent <name> | Specify the user agent name.
Example: |
-u, --user <user:password> | Provide the username and password for authentication.
Example: |
-v, --verbose | Tell curl to be verbose.
Example: |
-V, --version | See the installed versions of curl and libcurl.
Example: |
-w, --write-out <format> | Tell curl to show information about the completed transfer on stdout.
Example: |
--xattr | Store file metadata in file attributes.
Example: |
For comprehensive descriptions of the options, execute the curl --manual
command in the terminal.
Conclusion
After reading this tutorial, you should know how to use the curl command along with its numerous options. To find out more about what curl can do, read How to Set or Change User Agent with curl.
原创文章,作者:奋斗,如若转载,请注明出处:https://blog.ytso.com/224143.html