Machine Learning & Regression¶

1. 컨셉¶

$$y = f(x)$$¶

$y$ = 해결해야 하는 문제 (종속변수, target data)
$f()$ = 머신러닝 알고리즘 (모델)
$x$ = 훈련데이터 (설명변수, train data)

x -> y 를 잘 맞추는 모델 (함수) 를 만드는 것이 목표이고, 이를 데이터를 기반으로 학습해서 만들겠다는 것이 머신 러닝 입니다.

2. 종류¶

2.1. 회귀 (Regression) 또는 예측 (Prediction)¶

output y 가 연속적인 value 인 경우
ex. 주가 예측

2.2. 분류 (Classification)¶

output y 가 비연속적인 경우 (카테고리 타입)
ex. 제품이 불량인지 양품인지 분류하기

2.3. 군집화 (Clustering)¶

output y 가 똑같이 카테고리이나, 정답 없이 학습함
비지도 학습 (Unsupervised Learning)

3. Machine Learning 예시¶

여러 정보를 이용해서 주택 가격을 맞춰보자!
data 폴더의 boston_housing.csv 로 예시

참조 - 캐글의 Boston 집값 데이터 열람

3.1. dataset 로드하기¶

전체 (학습) 데이터를 로드합니다.

In [14]:

Copied!

import pandas as pd 

column_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']

df = pd.read_csv("../../data/boston_housing.csv",  header=None, delimiter=r"\s+", names=column_names)
df
import pandas as pd 

column_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV']

df = pd.read_csv("../../data/boston_housing.csv",  header=None, delimiter=r"\s+", names=column_names)
df

Out[14]:

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT	MEDV
0	0.00632	18.0	2.31	0	0.538	6.575	65.2	4.0900	1	296.0	15.3	396.90	4.98	24.0
1	0.02731	0.0	7.07	0	0.469	6.421	78.9	4.9671	2	242.0	17.8	396.90	9.14	21.6
2	0.02729	0.0	7.07	0	0.469	7.185	61.1	4.9671	2	242.0	17.8	392.83	4.03	34.7
3	0.03237	0.0	2.18	0	0.458	6.998	45.8	6.0622	3	222.0	18.7	394.63	2.94	33.4
4	0.06905	0.0	2.18	0	0.458	7.147	54.2	6.0622	3	222.0	18.7	396.90	5.33	36.2
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
501	0.06263	0.0	11.93	0	0.573	6.593	69.1	2.4786	1	273.0	21.0	391.99	9.67	22.4
502	0.04527	0.0	11.93	0	0.573	6.120	76.7	2.2875	1	273.0	21.0	396.90	9.08	20.6
503	0.06076	0.0	11.93	0	0.573	6.976	91.0	2.1675	1	273.0	21.0	396.90	5.64	23.9
504	0.10959	0.0	11.93	0	0.573	6.794	89.3	2.3889	1	273.0	21.0	393.45	6.48	22.0
505	0.04741	0.0	11.93	0	0.573	6.030	80.8	2.5050	1	273.0	21.0	396.90	7.88	11.9

506 rows × 14 columns

3.2. X, Y 정의 하기¶

In [16]:

Copied!





# X / Y Split
Y = df['MEDV']
X = df.drop('MEDV', axis=1)
X
# X / Y Split
Y = df['MEDV']
X = df.drop('MEDV', axis=1)
X

Out[16]:

	CRIM	ZN	INDUS	CHAS	NOX	RM	AGE	DIS	RAD	TAX	PTRATIO	B	LSTAT
0	0.00632	18.0	2.31	0	0.538	6.575	65.2	4.0900	1	296.0	15.3	396.90	4.98
1	0.02731	0.0	7.07	0	0.469	6.421	78.9	4.9671	2	242.0	17.8	396.90	9.14
2	0.02729	0.0	7.07	0	0.469	7.185	61.1	4.9671	2	242.0	17.8	392.83	4.03
3	0.03237	0.0	2.18	0	0.458	6.998	45.8	6.0622	3	222.0	18.7	394.63	2.94
4	0.06905	0.0	2.18	0	0.458	7.147	54.2	6.0622	3	222.0	18.7	396.90	5.33
...	...	...	...	...	...	...	...	...	...	...	...	...	...
501	0.06263	0.0	11.93	0	0.573	6.593	69.1	2.4786	1	273.0	21.0	391.99	9.67
502	0.04527	0.0	11.93	0	0.573	6.120	76.7	2.2875	1	273.0	21.0	396.90	9.08
503	0.06076	0.0	11.93	0	0.573	6.976	91.0	2.1675	1	273.0	21.0	396.90	5.64
504	0.10959	0.0	11.93	0	0.573	6.794	89.3	2.3889	1	273.0	21.0	393.45	6.48
505	0.04741	0.0	11.93	0	0.573	6.030	80.8	2.5050	1	273.0	21.0	396.90	7.88

506 rows × 13 columns

3.3. Train / Text (Valid) 스플릿¶

In [8]:

Copied!

%pip install -U scikit-learn
%pip install -U scikit-learn

Collecting scikit-learn
  Downloading scikit_learn-1.6.0-cp312-cp312-macosx_12_0_arm64.whl.metadata (31 kB)
Requirement already satisfied: numpy>=1.19.5 in /Users/jonhpark/workspace/courses_archive/mkdocs_venv/lib/python3.12/site-packages (from scikit-learn) (2.0.1)
Collecting scipy>=1.6.0 (from scikit-learn)
  Downloading scipy-1.14.1-cp312-cp312-macosx_14_0_arm64.whl.metadata (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 4.5 MB/s eta 0:00:00
Collecting joblib>=1.2.0 (from scikit-learn)
  Using cached joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB)
Collecting threadpoolctl>=3.1.0 (from scikit-learn)
  Using cached threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB)
Downloading scikit_learn-1.6.0-cp312-cp312-macosx_12_0_arm64.whl (11.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.2/11.2 MB 19.6 MB/s eta 0:00:0000:0100:01
Using cached joblib-1.4.2-py3-none-any.whl (301 kB)
Downloading scipy-1.14.1-cp312-cp312-macosx_14_0_arm64.whl (23.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 23.1/23.1 MB 22.8 MB/s eta 0:00:0000:0100:01
Using cached threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Installing collected packages: threadpoolctl, scipy, joblib, scikit-learn
Successfully installed joblib-1.4.2 scikit-learn-1.6.0 scipy-1.14.1 threadpoolctl-3.5.0

[notice] A new release of pip is available: 24.0 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

In [17]:

Copied!

# 데이터셋 분리
from sklearn.model_selection import train_test_split

X_train, X_valid, Y_train, Y_valid = train_test_split(X,Y)
# 데이터셋 분리
from sklearn.model_selection import train_test_split

X_train, X_valid, Y_train, Y_valid = train_test_split(X,Y)

3.4. Model 정의하기¶

In [18]:

Copied!

# 모델 정의
from sklearn.linear_model import LinearRegression

model = LinearRegression()
# 모델 정의
from sklearn.linear_model import LinearRegression

model = LinearRegression()

3.5. Model 학습하기¶

In [19]:

Copied!

# 모델 학습
model.fit(X_train, Y_train)
# 모델 학습
model.fit(X_train, Y_train)

Out[19]:

LinearRegression()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

3.6. 모델 평가하기¶

In [20]:

Copied!





from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error


# Predict on the validation set
Y_pred = model.predict(X_valid)

# Calculate scores
r2 = r2_score(Y_valid, Y_pred)
mse = mean_squared_error(Y_valid, Y_pred)
print(f"R^2 Score: {r2:.2f}")
print(f"Mean Squared Error: {mse:.2f}")
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error


# Predict on the validation set
Y_pred = model.predict(X_valid)

# Calculate scores
r2 = r2_score(Y_valid, Y_pred)
mse = mean_squared_error(Y_valid, Y_pred)
print(f"R^2 Score: {r2:.2f}")
print(f"Mean Squared Error: {mse:.2f}")

R^2 Score: 0.72
Mean Squared Error: 25.41

In [22]:

Copied!





import plotly.express as px

# Convert X_valid to DataFrame for easier hover display
if hasattr(X_valid, "toarray"):  # Handle sparse matrices
    X_valid_df = pd.DataFrame(X_valid.toarray())
else:
    X_valid_df = pd.DataFrame(X_valid)

# Prepare data for plotting
scatter_data = pd.DataFrame({
    "Actual": Y_valid,
    "Predicted": Y_pred,
    "X Values": X_valid_df.values.tolist()  # List of X values for hover
})

# Create scatter plot with Plotly
fig = px.scatter(
    scatter_data,
    x="Actual",
    y="Predicted",
    hover_data={"X Values": True},  # Show X values in hover
    labels={"x": "Actual Values", "y": "Predicted Values"},
    title="Actual vs Predicted with X Values"
)
fig.add_shape(
    type="line",
    x0=scatter_data["Actual"].min(),
    y0=scatter_data["Actual"].min(),
    x1=scatter_data["Actual"].max(),
    y1=scatter_data["Actual"].max(),
    line=dict(color="Red", dash="dash"),
    name="Perfect Prediction"
)

fig.show()
import plotly.express as px

# Convert X_valid to DataFrame for easier hover display
if hasattr(X_valid, "toarray"):  # Handle sparse matrices
    X_valid_df = pd.DataFrame(X_valid.toarray())
else:
    X_valid_df = pd.DataFrame(X_valid)

# Prepare data for plotting
scatter_data = pd.DataFrame({
    "Actual": Y_valid,
    "Predicted": Y_pred,
    "X Values": X_valid_df.values.tolist()  # List of X values for hover
})

# Create scatter plot with Plotly
fig = px.scatter(
    scatter_data,
    x="Actual",
    y="Predicted",
    hover_data={"X Values": True},  # Show X values in hover
    labels={"x": "Actual Values", "y": "Predicted Values"},
    title="Actual vs Predicted with X Values"
)
fig.add_shape(
    type="line",
    x0=scatter_data["Actual"].min(),
    y0=scatter_data["Actual"].min(),
    x1=scatter_data["Actual"].max(),
    y1=scatter_data["Actual"].max(),
    line=dict(color="Red", dash="dash"),
    name="Perfect Prediction"
)

fig.show()

In [23]:

Copied!





# Get feature importance (coefficients)
feature_importance = model.coef_

# If X is a DataFrame, get the column names, else use generic names
if isinstance(X, pd.DataFrame):
    feature_names = X.columns
else:
    feature_names = [f"Feature {i}" for i in range(X.shape[1])]

# Create a DataFrame for feature importance
importance_df = pd.DataFrame({
    "Feature": feature_names,
    "Importance": feature_importance
}).sort_values(by="Importance", ascending=False)

# Print feature importance
print(importance_df)

# Plot feature importance
fig = px.bar(
    importance_df,
    x="Feature",
    y="Importance",
    title="Feature Importance",
    labels={"Importance": "Coefficient"},
    text="Importance"
)
fig.update_traces(texttemplate='%{text:.2f}', textposition='outside')
fig.show()
# Get feature importance (coefficients)
feature_importance = model.coef_

# If X is a DataFrame, get the column names, else use generic names
if isinstance(X, pd.DataFrame):
    feature_names = X.columns
else:
    feature_names = [f"Feature {i}" for i in range(X.shape[1])]

# Create a DataFrame for feature importance
importance_df = pd.DataFrame({
    "Feature": feature_names,
    "Importance": feature_importance
}).sort_values(by="Importance", ascending=False)

# Print feature importance
print(importance_df)

# Plot feature importance
fig = px.bar(
    importance_df,
    x="Feature",
    y="Importance",
    title="Feature Importance",
    labels={"Importance": "Coefficient"},
    text="Importance"
)
fig.update_traces(texttemplate='%{text:.2f}', textposition='outside')
fig.show()

    Feature  Importance
5        RM    3.641342
3      CHAS    1.807958
8       RAD    0.347263
2     INDUS    0.073697
1        ZN    0.058716
6       AGE    0.008889
11        B    0.008377
9       TAX   -0.015813
0      CRIM   -0.102134
12    LSTAT   -0.518889
10  PTRATIO   -1.005465
7       DIS   -1.523085
4       NOX  -20.377928

In [ ]: