HTTP Request/Response

An introduction to HTTP requests and responses, including URLs, headers, status codes, and message structure.

HTTP

Network Stack (A simplified view)

  • Enter HTTP

Packet Structure:

  1. IP Headers
  2. TCP Headers
  3. HTTP Headers
  4. Content

HTTP - Documentation

  • HTTP/1.1 is defined by RFC2616 of the IETF
  • RFC
    • Request For Comments
    • Submit an RFC for public discussion or to publish information
  • IETF
    • Internet Engineering Task Force
    • Adopts some RFC's as Internet standards

HyperText Transfer Protocol (HTTP)

  • HTTP is an application layer protocol
    • Protocols used by our applications
    • Protocols that are not concerned with the transmission of data
  • [Almost] Always uses TCP for reliable communication
    • Always in this course

HTTP

  • HTTP is a protocol used to access content from a web server
  • Protocol: An agreed upon set of rules
    • HTTP: Defines the format of messages sent to/from a web server
  • HTTP is a Request - Response protocol
    • Client makes request to server
    • Server returns a response
    • Ex. Request The latest tweets from a user. Twitter server returns the tweets in its response
  • Response may require more requests
    • Ex. Get HTML which requires CSS/JS/Images

Web Server

  • Software that "speaks" HTTP
  • Listens for HTTP requests and responds with HTTP responses
  • We want to host our web pages/apps on the Internet using HTTP
  • Terminology:
    • Front End - The part of a web app that runs in the browser (HTML/CSS/JS)
      • The front end will send HTTP requests to the back end
    • Back End - The web server and all software that does not run on the user's machine
      • The back end will send HTTP responses to the front end

Loading a Web Site

sequenceDiagram
  participant Browser as Browser (Front End)
  participant Server as Web Server (Back End)
  Browser->>Server: 1. Request HTML
  Server->>Browser: 2. Sends HTML
  Browser->>Server: 3. Request Each url in HTML
  Server->>Browser: 4. Sends Content
  Note over Browser: 5. Browser renders all content and runs JavaScript

The Browser and the Web Server communicate across the Network (Internet).

HTTP Request

  • Each HTTP request will contain the request type:
    • GET: Request information from a server
    • POST: Send information to a server
    • PUT: Add information to a service
    • PATCH: Partial update of existing data
    • DELETE: Delete information from a service
    • HEAD: Request only the headers of a response
  • To start, we'll focus on GET and POST only

HTTP Request

  • HTTP GET Request
    • Used when requesting content from a server
    • [Typically] Only contains a URL and HTTP headers
    • When you click a link, your browser makes a GET request
    • Requesting HTML/CSS/Javascript/Images/etc are GET requests
  • HTTP POST Request
    • Used when sending data to a website
    • Contains a URL and a body [And HTTP headers]
    • When you submit a form, your browser [typically] makes a POST request
    • The contents of the form are sent in the body of the request

HTTP Request

Protocol://host:port/path?query_string#fragment
  • Each request is made for a specific URL (Uniform Resource Location)
    • A URL uniquely identifies a resource and has the following parts
  • Protocol - The protocol being used (ex. file, HTTP, HTTPS, FTP)
  • Host - The IP address or domain name of the server
    • Used to route the request to the appropriate machine
  • Port - The TCP port number of the host server
    • Defaults to 80/443 for HTTP/HTTPS respectively
  • Path - Specifies the specific resource being requested from the server

HTTP Request

Protocol://host:port/path?query_string#fragment

HTTP is Stateless

  • HTTP is a stateless protocol
  • Each request is handled in isolation even if a client just made another request
  • If state is desired (ex. Login), the state must be sent with each request
    • Cookies
  • When handling an HTTP request, do not have to care who sent it

xkcd comic 869, "Server Attention Span": a person speaks to a server, which replies to every message with an identical, indifferent response, illustrating that the server keeps no memory of previous requests.

https://xkcd.com/869/

New Lines

  • A new line character in an HTTP request/response must be:
    • \r\n
      • Carriage return (From the days of typewriters)
      • New line
    • In the documentation this is referred to as a CRLF
    • CRLF == Carriage Return Line Feed
  • Be aware of this while parsing
  • Use \r\n for new lines when preparing your responses

HTTP GET Request

GET Request

  • We'll use this simple request as an example
GET / HTTP/1.1
Host: cse312.com
Connection: keep-alive
Accept-Language: en-US,en

GET Request

  • More accurately, it will be this
GET / HTTP/1.1\r\nHost: cse312.com\r\nConnection: keep-alive\r\nAccept-Language: en-US,en\r\n\r\n
  • In the slides, we'll render "\r\n" as a new line
  • Note that there is a blank line at the end of the request

The Request Line

  • The first line of the request is always the request line
  • The request line has 3 values separated by spaces
    • The request type (GET/POST/PUT/DELETE/etc)
    • The path of the request (ex. "/") - Everything after the port in the requested url
    • The HTTP Version
      • We'll always use HTTP/1.1 in this course
GET / HTTP/1.1
Host: cse312.com
Connection: keep-alive
Accept-Language: en-US,en

The Request Line

  • Parse the request line by looking for the 2 space characters
    • Separate the values and check the strings
  • Typically: When the root path "/" is requested, serve the HTML of your home page
    • By convention, web servers look for index.html to serve at the root path
  • If the url contains a different path, it will appear in the request line

Requesting http://cse312.com/

Requesting http://cse312.com/static_files/slides/1_2_HTTP.pdf

GET / HTTP/1.1
Host: cse312.com
Connection: keep-alive
Accept-Language: en-US,en
GET /static_files/slides/1_2_HTTP.pdf HTTP/1.1
Host: cse312.com
Connection: keep-alive
Accept-Language: en-US,en

Headers

  • Following the request line are any number of headers
  • HTTP Headers
    • Key-Value pairs
    • Key and value separated by a colon ":"
  • Each header will be on a new line (look for \r\n)
  • To parse, look for the colon ":" and read the key and value
    • There is an optional space after the colon which should be removed if present <-- don't forget to do this! Leaving in the space will cause bugs that are very difficult to detect
GET / HTTP/1.1
Host: cse312.com
Connection: keep-alive
Accept-Language: en-US,en

HTTP Response

Response

  • Your web server will listen for HTTP requests over the TCP sockets and respond with HTTP responses
  • Send a response back to the client to serve them the requested content
  • This response will display "hello" in their browser
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

Response

  • More accurately, this is the response
HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 5\r\n\r\nhello

Status Line

  • The first line of the response must be the status line
  • Status line contains 3 values separated by spaces
    • The HTTP version
    • The status code
    • The status message (Reason phrase in docs)
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

Response Codes

  • Tells the browser the nature of the response
    • 200-level codes: Everything went well
    • 300-level codes: Redirect the request
    • 400-level codes: Error caused by the client
    • 500-level codes: Error caused by the server
  • Include a human readable message
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

Response Headers

  • The headers in the response follow the same format as request headers
  • Should have at least two headers
    • Content-Type - Tells the browser how to parse this content
    • Content-Length - How many bytes should be read from the body of the response
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

Blank Line

  • There is a blank line (\r\n\r\n) separating the headers and the body of the response
  • The blank line is very important
    • It tells the browser that you're done with headers and the next bytes will be the body of the response
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

Body

  • After the blank line are the bytes of the body
  • The body contains the content that is being served
  • The number of bytes in the body exactly match the value of the Content-Length header
    • The browser will read exactly this many bytes
  • Note: There is no required format for the body. The format depends on the type of content being served
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 5

hello

404 Not Found

  • If a path is requested that your server does not handle
    • Respond with a 404 Not Found
    • Note: Spaces are allowed in your reason message
  • The response format is the same as a 200 response
    • Include content type and length
    • Include a body that will be displayed to the client
HTTP/1.1 404 Not Found
Content-Type: text/plain
Content-Length: 36

The requested content does not exist

Further Reading