This post is part of the page requests to the metal series where we cover all the steps from a web request down to the electrons that move in the CPU, see the intro/overview page to see where this fits in.

The browser has sent the information from the clients browser over the network and made it to the web server. The portion of dealing with those requests is often categorized as the "back end" and we will run with that terminology to group the next few steps.

Handling the web request

Once the server has received the packets those get passed up the networking stack, and the packets with the correct TCP port will then get routed over to the web server. At a very basic level we can write code to handle the accepting of an incoming connection like this:

serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((socket.gethostname(), 80))
serversocket.listen(5)

Then we can create a main loop to handle accepting incoming connections:

while True:
    (clientsocket, address) = serversocket.accept()
    # now do something with the clientsocket

However doing this with CPython wouldn't scale all that well (we had to re-engineer a system that connected to IoT devices via TCP/IP that handled incoming connections in a naive way because it had problems scaling when they got more customers). For hosting websites we would do well to use specialized software that handles HTTP requests and handles them well, implementing your own code to efficiently handle many connections concurrently is a difficult task.

In practice production HTTP servers are often running something like Apache or nGinx. In the case of NGINX have a look at this article that outlines how the internal architecture works: https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/

A quick overview of NGINX is:

• NGINX creates worker processes
• These worker processes listen for incoming connections
• When a connection is accepted the connection is assigned to a state machine that handles the flow of data for that connection
• When the state machine requires content to be generated it will forward to our Python code via uWSGI

Getting the web request into Python - WSGI

Given that in production we will be using a dedicated HTTP server that's not implemented in Python we need to get the request over to Python at some point.

The WSGI PEP explains the interface for connecting the web server to the Python world which in the case of Python backend software is often sitting behind a WSGI implementation such as uWSGI. It's the bridge between the server software and the Python code which processes the request and then forwards it on to the Django URL router to get dealt with.

Django

At this point the code enters into our web framework, in this case Django.

Django URL routing:

The web framework must create the needed content by first dealing with the incoming URL to decide what content needs to be generated and how errors are to be handled. The first step is parsing the incoming URL paths and then dispatching to the relevant functions that must be called for that URL.

The Django URL dispatcher takes the incoming path and then routes it to the appropriate view function that will handle that incoming request. In Django's settings.py we have the following:

ROOT_URLCONF = 'ToTheMetal.urls'

This specifies the Python module where Django has the entry point for URL dispatching. This is the first place that will be searched for a variable called  urlpatterns which should be a list containing instances of django.conf.urls.url() types.

The base urls file (ToTheMetal/ToTheMetal/urls.py) is fairly simple:

from django.conf.urls import url, include
from django.contrib import admin

urlpatterns = [
    url(r'^admin/', admin.site.urls),
    url(r'^

You'll notice that there are url "tuples" of regular expressions that define URL paths and functions. When a regex gets a match Django will call the associated function. Django's URL dispatcher works by linearly searching this list until the first regex match occurs at which point the associated URL handler function gets called with any of the results in regex capture groups getting forwarded the function as arguments. If we match '^admin/ it takes us to the admin page and otherwise if we match ^$ (which matches an empty string, the base URL) it will then take us to the URLs defined in the python module add_two_numbers.urls. Because we are using Python module paths match filesystem paths so that module is found at ToTheMetal/add_two_numbers/urls.py :

# URL routing for the add_two_numbers app

from django.conf.urls import url

from . import views

urlpatterns = [
    url(r'^

Now if we match the regex ^$ again we call the function in the second argument which is views.add_two_numbers_page. This is our view function.

If no match can be made by the end of the urlpatterns Django will call the 404 error handler and if no custom handler is defined it will use the default one to respond with a 404 status code.

Django view

The Django view is the code that deals with the incoming request and generates the HTTP response.

In this case we have a function add_two_numbers_page in ToTheMetal/add_two_numbers/views.py

from django.template import loader, RequestContext
from django.http import HttpResponse

from .forms import AddTwoNumbersForm

def add_two_numbers(first, second):
    """Add two numbers together
    :first: first number
    :second: second number
    """
    result = first + second
    return result

def add_two_numbers_page(request):
    """Add two numbers"""
    if request.method == "GET":
        form = AddTwoNumbersForm()
        template = loader.get_template("add_two_numbers/add_two_numbers.html")
        return HttpResponse(template.render({'form': form}, request))
    elif request.method == "POST":
        form = AddTwoNumbersForm(request.POST)
        if form.is_valid():
            result = add_two_numbers(
                form.cleaned_data["number1"],
                form.cleaned_data["number2"]
            )
            return HttpResponse("Answer is {}".format(result))

This takes the incoming request and deals with what's needed to handle HTTP responses and render the page.

When a user submits a POST request we go down the "POST" branch of the conditional and then need to check that the form input is valid. It turns out the properly handling forms is a bit complex so instead of writing a lot of code for processing forms each time we have a new project we can use the well tested forms functionality found in Django. So our form code looks like this (ToTheMetal/add_two_numbers/forms.py):

from django import forms

class AddTwoNumbersForm(forms.Form):
    """A form for adding two numbers together"""
    number1 = forms.IntegerField(label="number1")
    number2 = forms.IntegerField(label="number2")

This generated the HTML for the form earlier and now will generate all the form validation code for when we call .is_valid() as well. This saves us a lot of time (for example it implemented the CSRF protection for us) but introduces another layer of abstraction.

After this we again use the Django forms functionality to clean the data before calling the add_two_numbers function.

In the next steps we will see how the add_two_numbers function creates the answer by looking into how Python itself executes this code.