HTTP Headers are used for passing information between the client and the server.
Some headers are only used with either requests or responses, while some are common to both.
There are different types of headers such as:
- General Headers
- Entity Headers
- Request Headers
- Response Headers
- Security Headers
Let’s take look at each category one by one.
General Headers
General Headers are used both in HTTP request and response.
They are used to describe the message, rather than its content.
Below is the example:
| Header | Example | Description |
|---|---|---|
Date | Date: Wed, 16 Feb 2022 10:38:44 GMT | Holds the date and time at which the message originated. It’s preferred to convert the time to the standard UTC time zone. |
Connection | Connection: close | Dictates if the current network connection should stay alive after the request finishes. Two commonly used values for this header are close and keep-alive. The close value from either the client or server means that they would like to terminate the connection, while the keep-alive header indicates that the connection should remain open to receive more data and input. |
Entity Headers
Entity Headers is common to both request and response.
Entity Headers are used to describe the content transferred by a message.
They are commonly found in responses and POSTor PUT requests.
| Header | Example | Description |
|---|---|---|
Content-Type | Content-Type: text/html | Used to describe the type of resource being transferred. The value is automatically added by the browsers on the client-side and returned in the server response. The charset field denotes the encoding standard, such as UTF-8. |
Media-Type | Media-Type: application/pdf | The media-type is similar to Content-Type, and describes the data being transferred. This header can play a crucial role in making the server interpret our input. The charset field may also be used with this header. |
Boundary | boundary="b4e4fbd93540" | Acts as a marker to separate content when there is more than one in the same message. For example, within a form data, this boundary gets used as --b4e4fbd93540 to separate different parts of the form. |
Content-Length | Content-Length: 385 | Holds the size of the entity being passed. This header is necessary as the server uses it to read data from the message body, and is automatically generated by the browser and tools like cURL. |
Content-Encoding | Content-Encoding: gzip | Data can undergo multiple transformations before being passed. For example, large amounts of data can be compressed to reduce the message size. The type of encoding being used should be specified using the Content-Encoding header. |
Request Headers
Request Headers are used in an HTTP request and do not relate to the content of the message.
Example below:
Host: carabinersec.com - Specify the host being queried.
User-Agent: curl/7.77.0 - Describe the client requesting resource.
Referer: http://carabinersec.com - Denotes where the current request is coming from. If we click a link from a Google search, referer would be https://google.com.
Accept: */* - Describes which media types the client can understand.
Cookie: PHPSESSID=b4e4fbd93540 - Contains cookie value. Cookie is a piece of data stored on the client side and on the server, which acts as an identifier. They can be used for maintaining client’s access, save user preferences or sessions tracking.
Autorization: BASIC cGFzc3dvcmQK - Method for the server to identify clients. After successful authentication, server returns a token unique to the client. Unlike cookie, tokens are only stored on the client side.
Response Headers
Response Headers are used in an HTTP response and do not relate to the content.
Server: Apache/2.2.14 (win32) - Contains info about the web server.
Set-Cookie: PHPSESSID=b4e4fbd93540 - Contains cookie that is needed for client identification.
WWW-Authenticate: BASIC realm=“localhost” - Notifies the client about the type of authentication required to access the requested resource.
Security Headers
Security Headers are a class of response headers used to specify certain rules and policies to be followed by the browser while accessing the website.
Content-Security-Policy: script-src ‘self’ - Dictates the website’s policy towards externally injected resources. Example header restrict the execution of JavaScript only to scripts that originates from the same domain as the web page. This helps protect against attacks like XSS.
Strict-Transport-Security: max-age=31536000 - Prevents the browser from accessing the website over HTTP and forces all communication to go through HTTPS.
Referrer-Policy: origin - Dictates whether the browser should include the value specified via the Referrer header or not. It can help in avoiding disclosing sensitive URLs and information while browsing the website.
cURL
We can use the -I flag to send a HEAD request and only display the response response headers.
We can use -i flag to display both the headers and the response body.
So what is the difference?
-I sends a HEAD requests, -i sends any request we specify and prints the headers as well.
yoon@yoon-XH695R:~/Downloads$ curl -I https://jadu101.github.io
HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
permissions-policy: interest-cohort=()
last-modified: Mon, 26 Aug 2024 10:19:52 GMT
access-control-allow-origin: *
strict-transport-security: max-age=31556952
etag: "66cc56c8-c104"
expires: Sun, 01 Sep 2024 14:30:49 GMT
cache-control: max-age=600
x-proxy-cache: MISS
x-github-request-id: A5A8:39FC12:9770BD:9C3CE7:66D4783F
accept-ranges: bytes
age: 0
date: Mon, 02 Sep 2024 11:35:22 GMT
via: 1.1 varnish
x-served-by: cache-lax-kwhp1940126-LAX
x-cache: HIT
x-cache-hits: 0
x-timer: S1725276923.864538,VS0,VE111
vary: Accept-Encoding
x-fastly-request-id: bc443b35e483ab4d243461c3bb0ea11417429973
content-length: 49412
We can set request headers with -H flag.
We can use -A flag to set our User-Agent as such:
yoon@yoon-XH695R:~/Downloads$ curl https://jadu101.github.io -A 'Mozilla/5.0'
<!DOCTYPE html>
<html lang="en"><head><title>Welcome to my blog!</title><meta charset="utf-8"/><link rel="preconnect" href="https://fonts.googleapis.com"/><link rel="preconnect" href="https://fonts.gstatic.com"/><link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM Plex Mono&family=Schibsted Grotesk:wght@400;700&family=Source Sans Pro:ital,wght@0,400;0,600;1,400;1,600&display=swap"