{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Denison CS181/DA210 Homework\n",
    "\n",
    "Before you turn this problem in, make sure everything runs as expected. This is a combination of **restarting the kernel** and then **running all cells** (in the menubar, select Kernel$\\rightarrow$Restart And Run All).\n",
    "\n",
    "Make sure you fill in any place that says `YOUR CODE HERE` or \"YOUR ANSWER HERE\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# HTTP Introductory Homework\n",
    "\n",
    "> This set of exercises focus on \"raw\" HTTP, where requests are built from Python strings and replies will entail the full set of bytes as received over the network from a connected server.  For simple interaction with the `sockets` interface, they will presume use of the `mysocket` module described in the book, and available at the author's web site for the book.  This module should be added to the user's environment so that an `import mysocket` is possible before solving these exercises.  See the Appendix A.2 in the book for documentation on the `mysocket` module.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4425ca38ce8727c571bf1f16ffe1e078",
     "grade": false,
     "grade_id": "cell-b9abcf27cf7faf8f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import os.path\n",
    "import sys\n",
    "import importlib\n",
    "\n",
    "if os.path.isdir(os.path.join(\"../../..\", \"modules\")):\n",
    "    module_dir = os.path.join(\"../../..\", \"modules\")\n",
    "else:\n",
    "    module_dir = os.path.join(\"../..\", \"modules\")\n",
    "\n",
    "module_path = os.path.abspath(module_dir)\n",
    "if not module_path in sys.path:\n",
    "    sys.path.append(module_path)\n",
    "\n",
    "import mysocket as sock\n",
    "importlib.reload(sock)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "5488800fab4fbc87db7ce2c1ad33e1da",
     "grade": false,
     "grade_id": "cell-afb078ae39cd7d9f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## Socket Programming Requests\n",
    "\n",
    "The first set of exercises are about *making requests*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "f3ff789e838ced12128b6321d7812a20",
     "grade": false,
     "grade_id": "cell-b0630fc52541276a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q1** Suppose we wish to retrieve (GET) a file via HTTP (so port 80) from `datasystems.denison.edu`.  The resource path of the file is `/data/ind0.json`.  We wish to use version 1.1 of HTTP and to request that the connection be closed after a single request/reply exchange.  We will need a header line to satisfy the HTTP 1.1 requirement of a valid `Host` header.  Write a sequence of code to compose a valid HTTP request as a Python string, and assign the result to `message`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "98b4d5b92050b39164ffe73b3df717ce",
     "grade": false,
     "grade_id": "cell-cae4365673329fd8",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "print(message)\n",
    "print(\"--------------------\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "95eb520c5d1f6600409984dcb17ae253",
     "grade": true,
     "grade_id": "cell-c37e0233f773840c",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert type(message) == str\n",
    "assert message[:3] == \"GET\"\n",
    "assert message[4:4+len(\"/data/ind0.json\")] == \"/data/ind0.json\"\n",
    "assert \"Host: datasystems.denison.edu\" in message\n",
    "assert message.count('\\r\\n') == 4\n",
    "assert message[-4:] == '\\r\\n\\r\\n'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "be3bdee9a1558d735ca8d017fe03c1fa",
     "grade": false,
     "grade_id": "cell-71dd066a007cc94e",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q2** Write a sequence of code to establish a connection to the host `datasystems.denison.edu` at port 80, to send the string `message` from the previous problem to the host, receive the reply from the host until the server closes the connection, assigning the reply to `reply`, and close the connection.  Note: if the request is not completely correct, a network connection can wait forever for a reply that will never come.  So if you have difficulty here, double check your answer to the previous problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "2f71c9946d84bfddab0acc242b69efa2",
     "grade": false,
     "grade_id": "cell-59b25ae0233eb2f6",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "print(reply)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d6656362f5c6e74e4e051f6dd3c9e7cc",
     "grade": true,
     "grade_id": "cell-a262284d7dd83250",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "assert type(reply) == str\n",
    "assert \"200 OK\" in reply\n",
    "assert \"application/json\" in reply\n",
    "assert reply.endswith(\"19485.4}}}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "5d87df8e48d6980c81b479dd3a57bde9",
     "grade": false,
     "grade_id": "cell-3a541deeebbe933f",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q3** Suppose we want to generalize the scenario from the first exercise, where the two things that can change are the *host location* and the *resource path*.  For example, we might want to change the host to `httpbin.org` and the resource path to `/`, or many other combinations.  Write a function\n",
    "\n",
    "    buildRequest(location, resource)\n",
    "    \n",
    "that constructs and returns a Python string containing a valid HTTP GET request that incorporates the parameters `location` and `resource` into the request at the appropriate places, and includes the appropriate header lines (for the required `Host` and to request the server close the connection after the exchange)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "03a6242e28c1bda5fcbb946bf42b8b37",
     "grade": false,
     "grade_id": "cell-fdcdf3fee2f5371c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "print(buildRequest(\"httpbin.org\", \"/get\"))\n",
    "print(\"---------------------\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "8b48a658cb08f13258818fc2cbb8baa0",
     "grade": true,
     "grade_id": "cell-81a9009c478f82fa",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "r1 = buildRequest(\"datasystems.denison.edu\", \"/data/ind0.json\")\n",
    "assert r1[:3] == \"GET\"\n",
    "assert r1[4:4+len(\"/data/ind0.json\")] == \"/data/ind0.json\"\n",
    "assert \"Host: datasystems.denison.edu\" in r1\n",
    "assert r1.count('\\r\\n') == 4\n",
    "assert r1[-4:] == '\\r\\n\\r\\n'\n",
    "r2 = buildRequest(\"httpbin.org\", \"/get\")\n",
    "assert r2[:3] == \"GET\"\n",
    "assert r2[4:4+len(\"/get\")] == \"/get\"\n",
    "assert \"Host: httpbin.org\" in r2\n",
    "assert r2.count('\\r\\n') == 4\n",
    "assert r2[-4:] == '\\r\\n\\r\\n'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "d51915b5cb1442d677ce07f519fee8ed",
     "grade": false,
     "grade_id": "cell-6002097ec5363a4b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q4** Write a function\n",
    "\n",
    "    makeRequest(location, resource)\n",
    "\n",
    "that first constructs a valid HTTP GET request for `resource` at host `location`, as a Python string (using your function from the previous question), and then performs the  request-reply steps of making the connection, sending the string request, receiving a reply until the connection closes, and finally closing the client side of the connection.  The function should return the reply."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "30b19964bf942508b8b6d992a796dc21",
     "grade": false,
     "grade_id": "cell-b0ae35dd066cdd61",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "print(makeRequest(\"datasystems.denison.edu\", \"/basic.html\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "ba04b19e9ece1d78a0377739d87620ff",
     "grade": true,
     "grade_id": "cell-558109be645f32b3",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "resp1 = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "#print(resp1)\n",
    "assert \"200 OK\" in resp1\n",
    "assert \"text/html\" in resp1\n",
    "assert resp1.endswith(\"</html>\\n\")\n",
    "\n",
    "resp2 = makeRequest(\"datasystems.denison.edu\", \"/data/ind0.json\")\n",
    "#print(resp2)\n",
    "assert \"200 OK\" in resp2\n",
    "assert \"application/json\" in resp2\n",
    "assert resp2.endswith(\"19485.4}}}\")\n",
    "\n",
    "resp3 = makeRequest(\"httpbin.org\", \"/get\")\n",
    "#print(resp3)\n",
    "assert \"200 OK\" in resp3\n",
    "assert \"application/json\" in resp3\n",
    "assert resp3.endswith(\"\"\"\"url\": \"http://httpbin.org/get\"\\n}\\n\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "ab62f69ff64a86cbcffbd134ca05c332",
     "grade": false,
     "grade_id": "cell-570f8c0f34f6b769",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "## Programming Response Replies\n",
    "\n",
    "The next set of exercises are about parsing through the reply resulting from a request.  If we consider an HTTP reply, we can partition it into a status line, the set of headers, and the body.  The exercises ask for functions that, given a reply, and parse the reply and return each of these pieces."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "1f52ffc6364db8f1a2336389169d5efa",
     "grade": false,
     "grade_id": "cell-355896e383db531b",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q5:** Write a function\n",
    "\n",
    "    parseStatus(reply)\n",
    "\n",
    "that finds and returns a Python string consisting of only the status line of a reply.  The returned value should include the line-terminating `\"\\r\\n\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "e9d2a665cca49b0688e3c11efcddfd07",
     "grade": false,
     "grade_id": "cell-db4474c167be79ae",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "print(repr(parseStatus(reply)))\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "print(repr(parseStatus(reply)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "10a0e5e628cbe7eca5f58731c99d6c08",
     "grade": true,
     "grade_id": "cell-4e8d16f523ad3818",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "r1 = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "s1 = parseStatus(r1)\n",
    "assert s1 == \"HTTP/1.1 200 OK\\r\\n\"\n",
    "\n",
    "r2 = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "s2 = parseStatus(r2)\n",
    "assert s2 == \"HTTP/1.1 404 Not Found\\r\\n\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "9acc8db47c6d12cd54bb50d77ba72bd6",
     "grade": false,
     "grade_id": "cell-00719c363cfc65b8",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q6:** Write a function\n",
    "\n",
    "    parseHeaders(reply)\n",
    "\n",
    "that finds and returns a single Python string that starts with the first header in the reply and continues up through the last header in the reply, including the line-terminating `\"\\r\\n\"`, but *not* the empty line separating the headers from the body."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "a2c00409bce5c88e1922e9cf5bb6267a",
     "grade": false,
     "grade_id": "cell-4985de7e00373e2d",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "print(repr(parseHeaders(reply)))\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "print(repr(parseHeaders(reply)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7da0d77518b26e6f9d72170f37adff78",
     "grade": true,
     "grade_id": "cell-c10750ab8472692f",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "r1 = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "h1 = parseHeaders(r1)\n",
    "assert \"Server: Apache\" in h1\n",
    "assert \"Connection: close\\r\\n\" in h1\n",
    "assert \"Content-Type: text/html\" in h1\n",
    "r2 = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "h2 = parseHeaders(r2)\n",
    "assert \"Server: Apache\" in h2\n",
    "assert \"Connection: close\\r\\n\" in h2\n",
    "assert \"Content-Type: text/html\" in h2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "markdown",
     "checksum": "1d1b31a15d1ced695c444dc1c4cf0065",
     "grade": false,
     "grade_id": "cell-1a7baf449889a33a",
     "locked": true,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "source": [
    "**Q7:** Write a function\n",
    "\n",
    "    parseBody(reply)\n",
    "\n",
    "that finds and returns a single Python string that starts with the beginning of the body (i.e. after the empty line of the reply) and continues to the end of the reply."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "129baf755382dd175737885fb65c504f",
     "grade": false,
     "grade_id": "cell-fd88ef8bb50a41c1",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# YOUR CODE HERE\n",
    "raise NotImplementedError()\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "print(parseBody(reply))\n",
    "reply = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "print(parseBody(reply))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4993b452f397939e4b35b63c8e821b7c",
     "grade": true,
     "grade_id": "cell-7859319f27b6d899",
     "locked": true,
     "points": 2,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "r1 = makeRequest(\"datasystems.denison.edu\", \"/basic.html\")\n",
    "b1 = parseBody(r1)\n",
    "r2 = makeRequest(\"datasystems.denison.edu\", \"/foobar.txt\")\n",
    "b2 = parseBody(r2)\n",
    "assert b1.startswith(\"<!DOCTYPE html>\")\n",
    "assert b1.endswith(\"</html>\\n\")\n",
    "assert b2.startswith(\"<!DOCTYPE HTML\")\n",
    "assert b2.endswith(\"</body></html>\\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}