Introducing HTTP

In [32]:
import os
import os.path
import sys
import importlib

if os.path.isdir(os.path.join("../../..", "modules")):
    module_dir = os.path.join("../../..", "modules")
else:
    module_dir = os.path.join("../..", "modules")

module_path = os.path.abspath(module_dir)
if not module_path in sys.path:
    sys.path.append(module_path)

import mysocket as sock
importlib.reload(sock)
Out[32]:
<module 'mysocket' from '/Users/tcbressoud/Dropbox/cs181-DataSystems/cs181-bressoud/f20_class/modules/mysocket.py'>

Request/Response Sequence

  1. Establish a TCP socket connection with server machine $\textit{host}$ at port $\textit{port}$; call it connection.
  2. Build a correctly formatted HTTP request string and assign it to a string variable, request_message. This will use a an HTTP method (GET) and will include a header Host with value $\textit{host}$ and use \textit{resource path} as the URI in the request-line.
  3. Perform a send of the string request_message over connection.
  4. Perform a receive of the HTTP message response from connection. Assuming a valid response message, this must retrieve the string of characters up to and including the \textit{empty-line} after the message headers, and then, based on additional information, must retrieve the body of the response.
  5. Perform a close() operation on connection.

Provided Helper Functions

In module mysocket, imported above as

import mysocket as sock
Function Description
makeConnection(host, port) Establish a TCP connection from the client machine to a server at the given machine host and listening at the given port. This returns the socket connection. This corresponds to Step 1 of the client-side steps.
sendString(conn, s) Given an established socket conn, take s, a string, and send it over the connection. This corresponds to Step 3 of the client-side steps, where s would define all the characters making up a complete HTTP request.
receiveTillClose(conn) This performs a socket recv() from the connection, consuming data until the server closes the connection. This returns the complete HTTP response message. This corresponds to Step 4 of the client-side steps, and assumes that a connection close will define the end of the response message.

Step 1

In [33]:
connection = sock.makeConnection("httpbin.org", 80)
assert connection is not None

Step 2

In [34]:
request_line = 'GET / HTTP/1.1\r\n'
host_line = 'Host: httpbin.org\r\n'
one_and_done = 'Connection: close\r\n'
empty_line = '\r\n'

request_message = request_line + host_line + \
                  one_and_done + empty_line

Step 3

In [35]:
sock.sendString(connection, request_message)

Step 4

In [36]:
response = sock.receiveTillClose(connection)

Step 5

In [37]:
connection.close()

Showing the first 250 characters in the response ...

In [38]:
print(repr(response[:250]))

'HTTP/1.1 200 OK\r\nDate: Thu, 29 Oct 2020 20:27:10 GMT\r\nContent-Type: text/html; charset=utf-8\r\nContent-Length: 9593\r\nConnection: close\r\nServer: gunicorn/19.9.0\r\nAccess-Control-Allow-Origin: *\r\nAccess-Control-Allow-Credentials: true\r\n\r\n<!DOCTYPE html>\n'

And printing the first 250 characters of the respose ...

In [39]:
print(response[:250])

HTTP/1.1 200 OK
Date: Thu, 29 Oct 2020 20:27:10 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 9593
Connection: close
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

<!DOCTYPE html>

Q Now repeat the process for the resource /basic.html at the server datasystems.denison.edu, still using port 80. Name the connection c2, the full message to send request, and the resoonse r.

In [48]:
# YOUR SOLUTION HERE
In [43]:
url = "http://datasystems.denison.edu/basic.html"
In [44]:
s = !curl -v -s $url
In [45]:
s
Out[45]:
['*   Trying 140.141.2.184:80...',
 '* Connected to datasystems.denison.edu (140.141.2.184) port 80 (#0)',
 '> GET /basic.html HTTP/1.1',
 '> Host: datasystems.denison.edu',
 '> User-Agent: curl/7.71.1',
 '> Accept: */*',
 '> ',
 '* Mark bundle as not supporting multiuse',
 '< HTTP/1.1 200 OK',
 '< Date: Thu, 29 Oct 2020 20:28:39 GMT',
 '< Server: Apache',
 '< Accept-Ranges: bytes',
 '< Content-Length: 496',
 '< Connection: close',
 '< Content-Type: text/html; charset=UTF-8',
 '< ',
 '{ [496 bytes data]',
 '* Closing connection 0',
 '<!DOCTYPE html>',
 '<html lang="en">',
 '  <head>',
 '    <title>Data Systems Basic HTML Page</title>',
 '  </head>',
 '  <body>',
 '    <h1>First Level Heading</h1>',
 '',
 '    <p>Paragraph defined in <b>body</b>.',
 '',
 '    <h2>Second Level Heading</h2>',
 '',
 '    <a href="http://docs.python.org">Link</a> to Python documentation.',
 '    </p>',
 '',
 '    <ul>',
 '      <li>Item 1',
 '      <ol>',
 '        <li>Item 1 nested</li>',
 '        <li>Item 2 nested</li>',
 '      </ol>',
 '      </li>',
 '      <li>Item 2</li>',
 '      <li>Item 3</li>',
 '    </ul>',
 '  </body>',
 '</html>']
In [47]:
help(sock.makeConnection)

Help on function makeConnection in module mysocket:

makeConnection(location, port=80, proxy=False)
    Description: Establish a TCP connection from the client machine and
    process running this function to a presumed server listening at the
    given port (80 by default) and located at Internet endpoint given by
    location.
    
    Returns an established socket connection, if successful, and None on
    failure.
In [ ]: