Understanding HTTP at a Low Level: A Developer’s Guide with C++

The Hypertext Transfer Protocol (HTTP) is the backbone of data communication on the web. As developers, we often use high-level libraries and frameworks to interact with it, but a deeper understanding of how HTTP works at a low level can be incredibly useful. In this guide, we’ll walk through the creation of a simple HTTP server in C++ to demystify how web servers handle client requests, respond to them, and manage resources.

Table of Contents:

1. Introduction to HTTP
2. Sockets and How They Work
3. Setting Up a Basic HTTP Server in C++
4. Handling Client Requests and Routes
5. Serving Static Files and Dynamic Content
6. Conclusion

1. Introduction to HTTP

HTTP is a protocol that allows for communication between a client (usually a web browser) and a server. It operates based on requests and responses. The client sends an HTTP request, and the server responds with an HTTP response. Understanding how this works at a low level involves looking into:

• Request methods like GET, POST, PUT, DELETE.
• Response status codes like 200 OK, 404 Not Found, 500 Internal Server Error.
• Headers that contain metadata (e.g., Content-Type, Content-Length).

When you type a URL into your browser, the browser makes a GET request to the server for the resource. The server processes the request and responds with the requested resource or an error.

2. Sockets and How They Work

At the heart of HTTP communication are sockets. Sockets enable network communication between two machines. In the case of HTTP, the client opens a socket connection to the server. Once the connection is established, data (HTTP requests and responses) is transmitted over this connection.

For our HTTP server, we’ll use sockets in C++. Below is an overview of the key socket-related functions:

• socket(): Creates a socket.
• bind(): Assigns an address and port to the socket.
• listen(): Puts the socket in listening mode to accept incoming connections.
• accept(): Accepts a connection from a client.
• recv(): Receives data from the client.
• send(): Sends data back to the client.

Now, let’s move forward and set up our HTTP server.

3. Setting Up a Basic HTTP Server in C++

Here’s how you can set up a simple HTTP server using C++ and the socket API. This server will handle incoming client requests and serve static files like HTML.

Creating a Server Class

First, we define a http_server class to handle all server-side logic. This class will include methods to:

• Initialize the server.
• Accept client connections.
• Read and respond to HTTP requests.

class http_server {
public:
    // Constructor: Initializes Winsock
    http_server() {
        WSADATA wsaData;
        // Initialize Winsock version 2.2
        int iResult = WSAStartup(MAKEWORD(2, 2), &wsaData);
        if (iResult != 0) {
            // If Winsock initialization fails, print an error message
            std::cerr << "WSAStartup failed: " << iResult << std::endl;
            return;
        }
    }

    // Destructor: Cleans up Winsock when the server is done
    ~http_server() noexcept {
        WSACleanup(); // Frees up Winsock resources
    }

    // Method to start the server on a given port
    void start(int port) {
        // Create a TCP/IP socket for listening (IPv4, TCP)
        SOCKET listen_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
        if (listen_socket == INVALID_SOCKET) {
            // If socket creation fails, print the error and exit
            std::cerr << "Error at socket(): " << WSAGetLastError() << std::endl;
            return;
        }

        // Set up the sockaddr_in structure to specify the server details
        sockaddr_in service;
        service.sin_family = AF_INET; // IPv4 family
        service.sin_addr.s_addr = INADDR_ANY; // Accept connections from any IP address
        service.sin_port = htons(port); // Set the port (converted to network byte order)

        // Bind the socket to the specified address and port
        if (bind(listen_socket, (SOCKADDR*)&service, sizeof(service)) == SOCKET_ERROR) {
            // If binding fails, print an error and exit
            std::cerr << "Bind failed: " << WSAGetLastError() << std::endl;
            closesocket(listen_socket); // Close the listening socket
            return;
        }

        // Start listening on the socket with a maximum queue of 1 pending connection
        if (listen(listen_socket, 1) == SOCKET_ERROR) {
            std::cerr << "Listen failed: " << WSAGetLastError() << std::endl;
            closesocket(listen_socket);
            return;
        }

        std::cout << "Server is listening on port " << port << std::endl;

        // Infinite loop to continuously accept incoming client connections
        while (true) {
            std::cout << "Waiting for client connection..." << std::endl;

            // Accept a client connection
            SOCKET client_socket = accept(listen_socket, nullptr, nullptr);
            if (client_socket == INVALID_SOCKET) {
                std::cerr << "Accept failed: " << WSAGetLastError() << std::endl;
                closesocket(listen_socket);
                return;
            }

            std::cout << "Client connected!" << std::endl;

            // Handle client communication (receive request and send response)
            handle_client(client_socket);

            // Close the client socket after handling the request
            closesocket(client_socket);
        }

        // Clean up and close the listening socket (this line is unreachable in this loop)
        closesocket(listen_socket);
    }

private:
    // Function to handle incoming client requests
    void handle_client(SOCKET client_socket) {
        char recvbuf[512]; // Buffer to store incoming client request data
        int recvbuflen = 512; // Length of the receive buffer

        // Receive data from the client
        int bytes_received = recv(client_socket, recvbuf, recvbuflen, 0);
        if (bytes_received > 0) {
            // Null-terminate the received data for safe printing
            recvbuf[bytes_received] = '\0';
            std::cout << "Received request:\n" << recvbuf << std::endl;

            // Send a basic HTTP response back to the client
            send_response(client_socket);
        } else {
            // If no data is received, print an error message
            std::cerr << "recv failed: " << WSAGetLastError() << std::endl;
        }
    }

    // Function to send a basic HTTP response to the client
    void send_response(SOCKET client_socket) {
        // Craft an HTTP response (simple HTML page)
        std::stringstream response;
        response << "HTTP/1.1 200 OK\r\n";
        response << "Content-Type: text/html\r\n";
        response << "Content-Length: 46\r\n"; // Length of the HTML content
        response << "\r\n";
        response << "<html><body><h1>Hello, World!</h1></body></html>";

        // Send the HTTP response to the client
        int bytes_sent = send(client_socket, response.str().c_str(), response.str().length(), 0);
        if (bytes_sent == SOCKET_ERROR) {
            // If sending fails, print an error
            std::cerr << "send failed: " << WSAGetLastError() << std::endl;
        } else {
            // Print the number of bytes sent
            std::cout << "Sent " << bytes_sent << " bytes to client." << std::endl;
        }
    }
};

This code creates a basic HTTP server that listens on a port, accepts client requests, and responds with a simple HTML page.

4. Handling Client Requests and Routes

The server should not just serve a fixed HTML page but be able to handle multiple routes, such as /about, /contact, etc. To accomplish this, we define routes and associate them with specific request-handling functions.

void get(const std::string& path, std::function<void(Request&, Response&)> callback) {
    assignHandler("GET", path, callback);
}

By allowing for GET and POST methods, our server can support dynamic content handling. We use regex to parse dynamic route parameters and extract query strings from URLs.

5. Serving Static Files and Dynamic Content

A real-world server needs to serve static files (HTML, CSS, JavaScript, etc.) and handle dynamic content. The serve_static_file() method takes care of static file handling:

void serve_static_file(const std::string& path, Response& response) {
    std::fstream file(publicDirPath + path, std::ios::in | std::ios::binary);
    if (file.is_open()) {
        response << std::string((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
        file.close();
    } else {
        response.status_code(404) << "File Not Found";
    }
}

The publicDirPath ensures the server knows where to find static assets like index.html, styles.css, or script.js.

Here is the complete code:

#include <iostream>
#include <string>
#include <sstream>
#include <fstream>
#include <map>
#include <functional>
#include <thread>
#include <chrono>
#include <regex>

#ifdef _WIN32
#include <winsock2.h>
#include <ws2tcpip.h>
#include <WS2spi.h>
#define SOCKET_CLOSE(sock) closesocket(sock)
#else
#include <unistd.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#define SOCKET int
#define INVALID_SOCKET -1
#define SOCKET_CLOSE(sock) close(sock)
#endif

//Request and Response classes
class Request {
public:
    std::string method; // GET, POST, PUT, DELETE
    std::string path; // /about, /contact
    std::string body; 
    std::map<std::string, std::string> headers;
    std::map<std::string, std::string> query;
    std::map<std::string, std::string> params; // /post/:id

    Request() {}
};

class Response {
public:
    std::string status; // 200 OK, 404 Not Found
    std::string body;
    std::string headers;

    // Overload the << operator to append data to the body. Like res << "Hello World";
    template <typename T>
    Response& operator<<(const T& data) {
        body += data;
        return *this;
    }

    // Add a header to the response
    Response& header(const std::string& key, const std::string& value) {
        headers += key + ": " + value + "\r\n";
        return *this;
    }

    // Set the status code of the response
    Response& status_code(const int& code) {
        status = std::to_string(code) + " OK\r\n";
        return *this;
    }

    // Send an HTML file as the response
    Response& html(const std::string& path){
        header("Content-Type", "text/html");
        std::fstream file(path, std::ios::in | std::ios::binary);
        if(file.is_open()){
            body = std::string((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
            file.close();
        } else {
            status_code(404) << "File Not Found";
        }
        return *this;
    }

    // Send a JSON response
    Response& json(const std::string& data) {
        header("Content-Type", "application/json");
        body = data;
        return *this;
    }

    //Response constructor
    Response() {
        status = "200 OK\r\n";
    }
};

// Split a string by a delimiter. Used to parse the request path.
// Example: split("/about/us", '/') => ["", "about", "us"]
std::vector<std::string> split_(const std::string& path, char delimiter) {
    std::vector<std::string> tokens;
    std::string token;
    std::istringstream iss(path); // Convert the string to a stream
    while (std::getline(iss, token, delimiter)) {
        tokens.push_back(token); // Add the token to the vector
    }
    return tokens;
}

// HTTP server class
class http_server {
public:
    http_server() {
        WSADATA wsaData; // Initialize Winsock
        int iResult = WSAStartup(MAKEWORD(2, 2), &wsaData); // Start Winsock
        if (iResult != 0) {
            std::cerr << "WSAStartup failed: " << iResult << std::endl;
            return;
        }
    }

    ~http_server() noexcept {
        WSACleanup();
    }

    // Set the public directory path
    inline void publicDir(const std::string& dir) {
        publicDirPath = dir;
    }

    // Start the server on the specified port
    void start(int port) {
        SOCKET listen_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); // Create a socket
        if (listen_socket == INVALID_SOCKET) {
            std::cerr << "Error at socket(): " << WSAGetLastError() << std::endl;
            return;
        }

        sockaddr_in service; // The sockaddr_in structure specifies the address family, IP address, and port for the socket that is being bound
        service.sin_family = AF_INET;  // The Internet Protocol version 4 (IPv4) address family
        service.sin_addr.s_addr = INADDR_ANY; // IP address of the server
        service.sin_port = htons(port); // The port number

        if (bind(listen_socket, (SOCKADDR*)&service, sizeof(service)) == SOCKET_ERROR) {
            std::cerr << "bind() failed." << std::endl;
            SOCKET_CLOSE(listen_socket);
            return;
        }

        if (listen(listen_socket, SOMAXCONN) == SOCKET_ERROR) {
            std::cerr << "Error listening on socket." << std::endl;
            SOCKET_CLOSE(listen_socket);
            return;
        }

        std::cout << "Server is listening on port " << port << std::endl;

        while (true) {
            SOCKET client_socket = accept(listen_socket, NULL, NULL); // Accept a client socket
            if (client_socket == INVALID_SOCKET) {
                std::cerr << "accept failed: " << WSAGetLastError() << std::endl;
                SOCKET_CLOSE(listen_socket);
                WSACleanup(); // Clean up Winsock
                return;
            }

            std::thread t(&http_server::handle_client, this, client_socket); // Create a new thread to handle the client
            t.detach();
        }
    }

    // Add a new route with a GET method
    void get(const std::string& path, std::function<void(Request&, Response&)> callback) {
        assignHandler("GET", path, callback);
    }

    // Add a new route with a POST method
    void post(const std::string& path, std::function<void(Request&, Response&)> callback) {
        assignHandler("POST", path, callback);
    }

private:
    // Map to store the routes
    std::map<std::string, std::map<std::string, std::pair<std::string, std::function<void(Request&, Response&)>>>> routes;
    // Path to the public directory
    std::string publicDirPath;

    // Add event handler to the routes map
    void assignHandler(const std::string& method, const std::string& path, std::function<void(Request&, Response&)> callback){
        std::string newPath = std::regex_replace(path, std::regex("/:\\w+/?"), "/([^/]+)/?");
        routes[method][newPath] = std::pair<std::string, std::function<void(Request&, Response&)>>(path, callback);
    }

    // Handle the client request
    void handle_client(SOCKET client_socket) {
        std::string request = read_request(client_socket);
        std::string method = request.substr(0, request.find(' '));
        std::string path = request.substr(request.find(' ') + 1, request.find(' ', request.find(' ') + 1) - request.find(' ') - 1);

        Response response;
        Request req;

        if (path.find('?') != std::string::npos) {
            std::string query_string = path.substr(path.find('?') + 1);
            path = path.substr(0, path.find('?'));
            std::istringstream query_iss(query_string);
            std::string query_pair;
            while (std::getline(query_iss, query_pair, '&')) {
                std::string key = query_pair.substr(0, query_pair.find('='));
                std::string value = query_pair.substr(query_pair.find('=') + 1);
                req.query[key] = value;
            }
        }


        std::smatch match;
        for (auto& route : routes[method]) {
            std::string route_path = route.first;

            if (std::regex_match(path, match, std::regex(route_path))) {
                std::regex token_regex(":\\w+");
                std::string originalPath = routes[method][route_path].first;

                std::vector<std::string> tokens = split_(originalPath, '/');
                while (std::regex_search(originalPath, match, token_regex) ) {
                    const std::string match_token = match.str();
                    int position = 0;
                    for (int i = 0; i < tokens.size(); i++) {
                        if (tokens[i] == match_token) {
                            position = i;
                            break;
                        }
                    }

                    std::vector<std::string> path_tokens = split_(path, '/');
                    req.params[match_token.substr(1)] = path_tokens[position];

                    originalPath = match.suffix();
                }

                routes[method][route_path].second(req, response);
                serve_static_file(path, response);
                send_response(client_socket, response);
                SOCKET_CLOSE(client_socket);
                return;
            }
        }

        // Serve static files if not matched by any route
        serve_static_file(path, response);
        send_response(client_socket, response);
        SOCKET_CLOSE(client_socket);
    }

    void serve_static_file(const std::string& path, Response& response) {

        // Determine the content type based on the file extension
        std::string content_type;
        std::string file_extension = path.substr(path.find_last_of('.') + 1);
        if (file_extension == "html") {
            content_type = "text/html";
        } else if (file_extension == "css") {
            content_type = "text/css";
        } else if (file_extension == "js") {
            content_type = "application/javascript";
        } else if (file_extension == "json") {
            content_type = "application/json";
        } else if (file_extension == "jpg" || file_extension == "jpeg") {
            content_type = "image/jpeg";
        } else if (file_extension == "png") {
            content_type = "image/png";
        } else if (file_extension == "gif") {
            content_type = "image/gif";
        } else {
            // Default to octet-stream for unknown file types
            content_type = "application/octet-stream";
        }
        std::fstream file( publicDirPath + path, std::ios::in | std::ios::binary);
        if (file) {
            
            response << std::string((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
            file.close();
            // Set the response headers
            response.header("Content-Type", content_type);
        }
    }

    // Read the request from the client
    std::string read_request(SOCKET client_socket) {
        std::string request;
        char buffer[1024]; // Buffer to store the request
        int bytes_received; // Bytes received from the client
        do {
            bytes_received = recv(client_socket, buffer, 1024, 0); // Receive the request
            if (bytes_received > 0) {
                request.append(buffer, bytes_received);
            }
        } while (bytes_received == 1024);
        return request;
    }

    void send_response(SOCKET client_socket, Response response) {
        response.header("Content-Length", std::to_string(response.body.size()));
        response.header("X-Powered-By", "Xebec-Server/0.1.0");
        response.header("Programming-Language", "C++");
        response.headers += "\r\n";
        std::string res = "HTTP/1.1 " + response.status + response.headers + response.body;
        send(client_socket, res.c_str(), res.size(), 0);
    }
};

int main() {

    http_server server; // Create a new server instance

    server.publicDir("public"); // Set the public directory

    // Add a new route
    server.get("/", [](Request& req, Response& res) {
        //send the html file
        res.html("index.html");
    });

    server.get("/about", [](Request& req, Response& res) {
        res.status_code(301) << "About page";
    });

    server.get("/contact", [](Request& req, Response& res) {
        res << "Contact page";
    });

    server.get("/echo/:message", [](Request& req, Response& res) {
        res << "Echo: " << req.params["message"];
    });

    server.post("/post", [](Request& req, Response& res) {
        res << "POST request";
    });

    server.post("/post/:id", [](Request& req, Response& res) {
        res << "POST request with id: " << req.params["id"];
    });

    server.get("/json", [](Request& req, Response& res) {
        res.json("{\"name\": \"John\", \"age\": 30, \"city\": \"New York\"}");
    });

    server.start(4119); // Start the server on port 4119
    return 0;
}

For more help, you can visit this github repo: https://github.com/itsfuad/Xebec-Server

Running the Code

You can compile this program on Windows using a C++ compiler that supports Winsock (such as Visual Studio). Once compiled and executed, the server will listen on port 8080. You can then open a browser or use a tool like curl to connect to http://localhost:8080 and see the response.

Let me know if you have further questions!