diff --git a/.coveragerc b/.coveragerc
new file mode 100644
index 0000000..9e59f5e
--- /dev/null
+++ b/.coveragerc
@@ -0,0 +1,7 @@
+[run]
+source = packtml
+include = */packtml/*
+omit =
+ */packtml/setup.py
+ */packtml/utils/plotting.py
+ */setup.py
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..987b39f
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,119 @@
+# scratch code
+scratch/
+
+# Any data unpackaged by tensorflow
+MNIST_data/
+
+# In-progress word docs
+~$*.doc*
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Ignore PyCharm stuff...
+.idea/
+
+# Mac stuff
+.DS_Store
+
+# C extensions
+*.so
+
+# Testing
+.pytest_cache/
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+# Usually these files are written by a python script from a template
+# before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# dotenv
+.env
+
+# virtualenv
+.venv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..cf50d8e
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,28 @@
+language: python
+sudo: required
+
+cache:
+ apt: true
+ directories:
+ - $HOME/.cache/pip
+ - $HOME/.ccache
+
+before_install:
+ - source build_tools/travis/before_install.sh
+env:
+ global:
+ - TEST_DIR=/tmp/packtml
+
+matrix:
+ include:
+ - os: linux
+ dist: trusty
+ env: PYTHON_VERSION="3.6"
+
+ - os: linux
+ dist: trusty
+ env: PYTHON_VERSION="2.7"
+
+install: source build_tools/travis/install.sh
+before_script: bash build_tools/travis/before_script.sh
+script: bash build_tools/travis/test_script.sh
diff --git a/MANIFEST.in b/MANIFEST.in
new file mode 100644
index 0000000..29ea7b7
--- /dev/null
+++ b/MANIFEST.in
@@ -0,0 +1 @@
+recursive include packtml/*
\ No newline at end of file
diff --git a/README.md b/README.md
index 9c14a5e..a4d5f03 100644
--- a/README.md
+++ b/README.md
@@ -1,2 +1,142 @@
# Hands-on-Supervised-Machine-Learning-with-Python
+
Published by Packt, Hands-on Supervised Machine Learning with Python
+
+### Learn the underpinning os many supervised learning algorithms, and develop rich python coding practices in the process.
+
+*Supervised learning—help teach a machine to think for itself!*
+
+## Overview
+
+These days machine learning is everywhere, and it’s here to stay. Understanding the core principles that drive how a machine “learns” is a critical skill for any would-be practitioner or consumer alike. This course will introduce you to supervised machine learning, guiding you through the implementation and nuances of many popular machine learning algorithms while facilitating a deep understanding along the way.
+
+In this course, we’ll cover parametric models such as linear and logistic regression, non-parametric methods such as decision trees and boosting, various clustering techniques, and we’ll wrap up with a brief foray into neural networks.
+
+This video course highlights clean coding techniques, object-oriented class design, and general best practices in machine learning
+
+## Target audience
+
+This course is designed for those who would like to understand supervised machine learning algorithms at a deeper level. If you’re interested in understanding how and why an algorithm works rather than simply how to call its API, this course might be for you. Intermediate Python knowledge and at least an intermediate understanding of mathematical concepts is assumed. While notions in this course will be broken down into bits as granular as absolutely possible, terms and ideas such as “matrix transposition,” “gradient,” “dot product,” and “time complexity” are assumed to be understood without further explanation.
+
+## What you will learn
+
+* Understand the fundamental and theoretical differences between parametric and non-parametric models, and why you might opt for one over the other.
+* Discover how a machine can learn a concept and generalize its understanding to new data
+* Implement and grok several well-known supervised learning algorithms from scratch; build out your github portfolio and show off what you’re capable of!
+* Learn about model families like recommender systems, which are immediately applicable in domains such as ecommerce and marketing.
+* Become a much stronger python developer
+
+### Project layout
+
+All **[source code](packtml/)** is within the `packtml` folder, which serves as the python
+package for this course. Within the [examples](examples/) directory, you'll find a
+number of short Python scripts that serve to demonstrate how various classes in the `packtml`
+submodules work. Each respective folder inside the `examples/` directory corresponds to a
+submodule inside of the `packtml` python package.
+
+### Getting started
+
+To get your environment set up, make sure you have Anaconda installed and on your path.
+Then simply run the following:
+
+```bash
+$ conda env create -f environment.yml
+```
+
+To activate your environment in a Unix environment:
+
+```bash
+$ source activate packt-sml
+```
+
+In a Windows environment:
+
+```
+activate packt-sml
+```
+
+### Set up the python package (in your activated environment):
+
+```bash
+(packt-sml) $ python setup.py install
+```
+
+## What you'll learn
+
+In this course and within this package, you'll learn to implement a number of
+commonly-used supervised learning algorithms, and when best to use one type of
+model over another. Below you'll find in-action examples of the various algorithms
+we implement within this package.
+
+### Regression
+
+The classic introduction to machine learning, not only will we learn about linear regression,
+we'll code one from scratch so you really understand what's happening
+[under the hood](packtml/regression/simple_regression.py). Then we'll
+[apply one in practice](examples/regression/example_linear_regression.py) so you can see
+how you might use it.
+
+
+
+Next, we'll dive into logistic regression, which is linear regression's classification cousin. See
+the full logistic regression example [here](examples/regression/example_logistic_regression.py)
+or the algorithm's [source code](packtml/regression/simple_logistic.py) if you're interested.
+
+
+
+### KNN clustering
+
+During our exploration of non-parametric models, we'll explore clustering.
+The `packtml` package implements a simple, but effective k-Nearest Neighbor classifier.
+Here is its output on the iris dataset. For the full code example, head to the
+[examples directory](examples/clustering/example_knn_classifier.py) and then to the
+[source code](packtml/clustering/knn.py) to see how it's implemented.
+
+
+
+### Decision trees
+
+In this course, we'll also implement a CART decision tree from scratch (for both
+regression and classification). Our classification tree's performance and potential
+is shown at varying tree depths in the images below. The classification tree example
+is located [here](examples/decision_tree/example_classification_decision_tree.py), and
+the source code can be found [here](packtml/decision_tree/cart.py).
+
+
+
+In addition to classification, we can build a tree as a non-linear regression
+model, as shown below. The regression tree example is located
+[here](examples/decision_tree/example_regression_decision_tree.py). Check out the
+[source code](packtml/decision_tree/cart.py) to understand how it works.
+
+
+
+### Deep learning
+
+One of the hottest topics of machine learning right now is deep learning and neural
+networks. In this course, we'll learn how to code a multi-layer perceptron classifier
+from scratch. The full example code is located [here](examples/neural_net/example_mlp_classifier.py)
+and this is the [source code](packtml/neural_net/mlp.py).
+
+
+
+Next, we'll show how we can use the weights the MLP has learned on previous data to
+learn new classification labels via transfer learning. For further implementation
+details, check out the [example code](examples/neural_net/example_transfer_learning.py)
+or the [source code](packtml/neural_net/transfer.py).
+
+
+
+### Recommendation algorithms
+
+These days, everything is available for purchase online. E-commerce sites have devoted
+lots of research to algorithms that can learn your preferences. In this course, we'll
+learn two such algorithms:
+
+* [Item-to-item](packtml/recommendation/itemitem.py) collaborative filtering
+* [Alternating least squares](packtml/recommendation/als.py) (matrix factorization)
+
+The [example ALS code](examples/recommendation/example_als_recommender.py) shows how
+train error decreases by iteration:
+
+
diff --git a/build_tools/README.md b/build_tools/README.md
new file mode 100644
index 0000000..63bd2fd
--- /dev/null
+++ b/build_tools/README.md
@@ -0,0 +1,3 @@
+# CI/CD build tools
+
+The scripts contained here are simply used for building the CI/CD pipeline.
\ No newline at end of file
diff --git a/build_tools/travis/before_install.sh b/build_tools/travis/before_install.sh
new file mode 100644
index 0000000..d28766f
--- /dev/null
+++ b/build_tools/travis/before_install.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+# only build on linux for travis, so this will work
+set -e
+sudo apt-get -qq update
diff --git a/build_tools/travis/before_script.sh b/build_tools/travis/before_script.sh
new file mode 100644
index 0000000..780c4c5
--- /dev/null
+++ b/build_tools/travis/before_script.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+set -e
+
+export DISPLAY=:99.0
+sh -e /etc/init.d/xvfb start
+sleep 5 # give xvfb some time to start by sleeping for 5 seconds
\ No newline at end of file
diff --git a/build_tools/travis/install.sh b/build_tools/travis/install.sh
new file mode 100644
index 0000000..cfb4eba
--- /dev/null
+++ b/build_tools/travis/install.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+# This script is meant to be called by the "install" step defined in
+# .travis.yml. See http://docs.travis-ci.com/ for more details.
+# The behavior of the script is controlled by environment variables defined
+# in the .travis.yml in the top level folder of the project.
+
+set -e
+
+echo 'List files from cached directories'
+echo 'pip:'
+ls $HOME/.cache/pip
+
+# for caching
+export CC=/usr/lib/ccache/gcc
+export CXX=/usr/lib/ccache/g++
+# Useful for debugging how ccache is used
+# export CCACHE_LOGFILE=/tmp/ccache.log
+# ~60M is used by .ccache when compiling from scratch at the time of writing
+ccache --max-size 100M --show-stats
+
+# Deactivate the travis-provided virtual environment and setup a
+# conda-based environment instead.
+deactivate || echo "No virtualenv or condaenv to deactivate"
+
+# install conda
+wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
+MINICONDA_PATH=/home/travis/miniconda
+
+# append the path, update conda
+chmod +x miniconda.sh && ./miniconda.sh -b -p $MINICONDA_PATH
+export PATH=$MINICONDA_PATH/bin:$PATH
+conda update --yes conda
+
+# Create the conda env and install the requirements
+conda create -n testenv --yes python=${PYTHON_VERSION}
+source activate testenv
+pip install -r requirements.txt
+pip install pytest pytest-cov
+
+# set up the package
+python setup.py install
diff --git a/build_tools/travis/test_script.sh b/build_tools/travis/test_script.sh
new file mode 100644
index 0000000..82cbf81
--- /dev/null
+++ b/build_tools/travis/test_script.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+set -e
+
+run_tests() {
+ oldpwd=`pwd`
+
+ # Move to another directory to test
+ cd ..
+ mkdir -p ${TEST_DIR} && cd ${TEST_DIR}
+ pytest --cov packtml
+
+ # move back to original dir
+ cd ${oldpwd}
+}
+
+run_tests
diff --git a/curriculum.docx b/curriculum.docx
new file mode 100644
index 0000000..59af30d
Binary files /dev/null and b/curriculum.docx differ
diff --git a/environment.yml b/environment.yml
new file mode 100644
index 0000000..c7c0e2b
--- /dev/null
+++ b/environment.yml
@@ -0,0 +1,9 @@
+name: packt-sml
+
+dependencies:
+ - python=3.6
+ - numpy
+ - scipy
+ - scikit-learn
+ - pandas
+ - matplotlib
\ No newline at end of file
diff --git a/examples/1.1 Supervised Learning Demo.ipynb b/examples/1.1 Supervised Learning Demo.ipynb
new file mode 100644
index 0000000..1f38691
--- /dev/null
+++ b/examples/1.1 Supervised Learning Demo.ipynb
@@ -0,0 +1,853 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Hands-on Supervised Machine Learning with Python\n",
+ "\n",
+ "Predicting whether an email is spam or not!"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## The Data\n",
+ "\n",
+ "You can find the data at the [UCI repository](https://archive.ics.uci.edu/ml/datasets/Spambase).\n",
+ "\n",
+ "### Files:\n",
+ "\n",
+ "1. [Spam data](https://archive.ics.uci.edu/ml/machine-learning-databases/spambase/spambase.data)\n",
+ "2. [Metadata](https://archive.ics.uci.edu/ml/machine-learning-databases/spambase/spambase.names)\n",
+ "3. [Documentation](https://archive.ics.uci.edu/ml/machine-learning-databases/spambase/spambase.DOCUMENTATION)\n",
+ "\n",
+ "## Reference\n",
+ "\n",
+ "__Creators:__\n",
+ "\n",
+ "*Mark Hopkins, Erik Reeber, George Forman, Jaap Suermondt \n",
+ "Hewlett-Packard Labs, 1501 Page Mill Rd., Palo Alto, CA 94304 *\n",
+ "\n",
+ "__Data Set Information:__\n",
+ "\n",
+ "*The \"spam\" concept is diverse: advertisements for products/web sites, make money fast schemes, chain letters, etc...*\n",
+ "\n",
+ "*Our collection of spam e-mails came from our postmaster and individuals who had filed spam. Our collection of non-spam e-mails came from filed work and personal e-mails, and hence the word 'george' and the area code '650' are indicators of non-spam. These are useful when constructing a personalized spam filter. One would either have to blind such non-spam indicators or get a very wide collection of non-spam to generate a general purpose spam filter. *\n",
+ "\n",
+ "__For background on spam: __\n",
+ "\n",
+ "*Cranor, Lorrie F., LaMacchia, Brian A. Spam! \n",
+ "Communications of the ACM, 41(8):74-83, 1998. *\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Found proxy settings\n",
+ "Already have data/spam.csv; will not download\n"
+ ]
+ }
+ ],
+ "source": [
+ "from urllib.request import urlretrieve, ProxyHandler, build_opener, install_opener\n",
+ "import requests\n",
+ "import os\n",
+ "\n",
+ "pfx = \"https://archive.ics.uci.edu/ml/machine-learning-databases/spambase/\"\n",
+ "data_dir = \"data\"\n",
+ " \n",
+ "# We might need to set a proxy handler...\n",
+ "try:\n",
+ " proxies = {\"http\": os.environ['http_proxy'],\n",
+ " \"https\": os.environ['https_proxy']}\n",
+ " print(\"Found proxy settings\")\n",
+ "\n",
+ " #create the proxy object, assign it to a variable\n",
+ " proxy = ProxyHandler(proxies)\n",
+ " # construct a new opener using your proxy settings\n",
+ " opener = build_opener(proxy)\n",
+ " # install the opener on the module-level\n",
+ " install_opener(opener)\n",
+ " \n",
+ "except KeyError:\n",
+ " pass\n",
+ "\n",
+ "\n",
+ "# The following will download the data if you don't already have it...\n",
+ "def get_data(link, where):\n",
+ " # Append the prefix\n",
+ " link = pfx + link\n",
+ " \n",
+ " # make the parent dir if its not there\n",
+ " if not os.path.exists(data_dir):\n",
+ " os.mkdir(data_dir)\n",
+ " \n",
+ " # download if necessary\n",
+ " if os.path.exists(where):\n",
+ " print(\"Already have %s; will not download\" % where)\n",
+ " return\n",
+ " \n",
+ " print(\"Downloading %s to %s\" % (link, where))\n",
+ " urlretrieve(link, where)\n",
+ " \n",
+ "\n",
+ "# Get the core spam data\n",
+ "get_data(link=\"spambase.data\", where=os.path.join(data_dir, \"spam.csv\"))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Load and split our data\n",
+ "\n",
+ "We've now downloaded the data into the `data/` directory. Let's load it into Pandas and do a bit of exploring..."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "\n",
+ "
\n",
+ " \n",
+ " \n",
+ " | \n",
+ " word_freq_make | \n",
+ " word_freq_address | \n",
+ " word_freq_all | \n",
+ " word_freq_3d | \n",
+ " word_freq_our | \n",
+ " word_freq_over | \n",
+ " word_freq_remove | \n",
+ " word_freq_internet | \n",
+ " word_freq_order | \n",
+ " word_freq_mail | \n",
+ " ... | \n",
+ " word_freq_conference | \n",
+ " char_freq_; | \n",
+ " char_freq_( | \n",
+ " char_freq_[ | \n",
+ " char_freq_! | \n",
+ " char_freq_$ | \n",
+ " char_freq_# | \n",
+ " capital_run_length_average | \n",
+ " capital_run_length_longest | \n",
+ " capital_run_length_total | \n",
+ "
\n",
+ " \n",
+ " \n",
+ " \n",
+ " | 0 | \n",
+ " 0.00 | \n",
+ " 0.64 | \n",
+ " 0.64 | \n",
+ " 0.0 | \n",
+ " 0.32 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 0.00 | \n",
+ " 0.000 | \n",
+ " 0.0 | \n",
+ " 0.778 | \n",
+ " 0.000 | \n",
+ " 0.000 | \n",
+ " 3.756 | \n",
+ " 61 | \n",
+ " 278 | \n",
+ "
\n",
+ " \n",
+ " | 1 | \n",
+ " 0.21 | \n",
+ " 0.28 | \n",
+ " 0.50 | \n",
+ " 0.0 | \n",
+ " 0.14 | \n",
+ " 0.28 | \n",
+ " 0.21 | \n",
+ " 0.07 | \n",
+ " 0.00 | \n",
+ " 0.94 | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 0.00 | \n",
+ " 0.132 | \n",
+ " 0.0 | \n",
+ " 0.372 | \n",
+ " 0.180 | \n",
+ " 0.048 | \n",
+ " 5.114 | \n",
+ " 101 | \n",
+ " 1028 | \n",
+ "
\n",
+ " \n",
+ " | 2 | \n",
+ " 0.06 | \n",
+ " 0.00 | \n",
+ " 0.71 | \n",
+ " 0.0 | \n",
+ " 1.23 | \n",
+ " 0.19 | \n",
+ " 0.19 | \n",
+ " 0.12 | \n",
+ " 0.64 | \n",
+ " 0.25 | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 0.01 | \n",
+ " 0.143 | \n",
+ " 0.0 | \n",
+ " 0.276 | \n",
+ " 0.184 | \n",
+ " 0.010 | \n",
+ " 9.821 | \n",
+ " 485 | \n",
+ " 2259 | \n",
+ "
\n",
+ " \n",
+ " | 3 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.0 | \n",
+ " 0.63 | \n",
+ " 0.00 | \n",
+ " 0.31 | \n",
+ " 0.63 | \n",
+ " 0.31 | \n",
+ " 0.63 | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 0.00 | \n",
+ " 0.137 | \n",
+ " 0.0 | \n",
+ " 0.137 | \n",
+ " 0.000 | \n",
+ " 0.000 | \n",
+ " 3.537 | \n",
+ " 40 | \n",
+ " 191 | \n",
+ "
\n",
+ " \n",
+ " | 4 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.00 | \n",
+ " 0.0 | \n",
+ " 0.63 | \n",
+ " 0.00 | \n",
+ " 0.31 | \n",
+ " 0.63 | \n",
+ " 0.31 | \n",
+ " 0.63 | \n",
+ " ... | \n",
+ " 0.0 | \n",
+ " 0.00 | \n",
+ " 0.135 | \n",
+ " 0.0 | \n",
+ " 0.135 | \n",
+ " 0.000 | \n",
+ " 0.000 | \n",
+ " 3.537 | \n",
+ " 40 | \n",
+ " 191 | \n",
+ "
\n",
+ " \n",
+ "
\n",
+ "
5 rows × 57 columns
\n",
+ "