Linear Regression¶
In [2]:
%matplotlib inline
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
Example: Boston Housing Dataset¶
In [3]:
from sklearn.datasets import load_boston
boston = load_boston()
In [4]:
boston.keys()
Out[4]:
In [5]:
boston.feature_names
Out[5]:
In [6]:
print(boston.DESCR)
In [7]:
boston.data.shape
Out[7]:
Lets create a Pandas DataFrame object for this dataset.
In [8]:
data = pd.DataFrame(boston.data, columns=boston.feature_names)
In [9]:
data.head()
Out[9]:
Lets add the target column also to the dataset.
In [10]:
data['PRICE'] = boston.target
In [11]:
data.head()
Out[11]:
In [12]:
data.describe()
Out[12]:
First thing to do when we get a dataset is get a sense of the dataset by visualizing it. Lets try to see if there is any correlation between various columns.
In [13]:
data.plot(kind="scatter", x='RM', y='PRICE')
Out[13]:
In [14]:
plt.hist(data.CRIM)
plt.title("CRIM")
plt.xlabel("Crime rate per capita")
plt.ylabel("Frequencey")
plt.show()
It looks like boston was very peaceful city back then.
In [15]:
data.PRICE.hist()
Out[15]:
Fitting Linear Regression using sklearn¶
In [16]:
from sklearn.linear_model import LinearRegression
X = data.drop('PRICE', axis = 1)
model = LinearRegression()
model.fit(X, data.PRICE)
Out[16]:
In [17]:
print('Estimated intercept coefficient:', model.intercept_)
In [18]:
print("coefficients:")
print(model.coef_)
In [19]:
for col, coef in zip(X.columns, model.coef_):
print("{:+6.4f} {}".format(coef, col))
The coefficients indicate a lot about the important features.
The house price seems to be highly effected by RM, the average number of rooms per dwelling and RAD, accessibility to radial highways and negatively effected by NOX, the pollution and DIS, the average distance to employment centers.
In [20]:
plt.scatter(np.log(data.DIS), data.PRICE)
Out[20]:
In [21]:
import sklearn
import numpy as np
X_train, X_test, Y_train, Y_test = sklearn.cross_validation.train_test_split(
X, data.PRICE, test_size=0.33, random_state = 5)
print(X_train.shape)
print(X_test.shape)
print(Y_train.shape)
print(Y_test.shape)
In [22]:
model = LinearRegression()
model.fit(X_train, Y_train)
Out[22]:
In [23]:
pred_train = model.predict(X_train)
pred_test = model.predict(X_test)
In [24]:
print("MSE train:", np.mean((Y_train - model.predict(X_train)) ** 2))
print("MSE test:", np.mean((Y_test - model.predict(X_test)) ** 2))
In [27]:
m = model
plt.scatter(m.predict(X_train), m.predict(X_train) - Y_train, c='b', s=40, alpha=0.5, label="train")
plt.scatter(m.predict(X_test), m.predict(X_test) - Y_test, c='g', s=40, label="test")
plt.hlines(y = 0, xmin=0, xmax = 50)
plt.ylabel('Residuals')
plt.legend()
Out[27]:
In [28]:
model
Out[28]:
In [27]:
model.residues_
Out[27]:
In [33]:
model.get_params()
Out[33]:
In [30]:
import statsmodels
In [ ]: