手动实现超参数搜索
手动实现超参数搜索
作者：一只工程狮
以搜索学习率（learning_rate）为例.
代码示例：
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
#为了能在notebook中显示图像
import numpy as np
import sklearn
import pandas as pd
import os
import sys
import time
import tensorflow as tf
from tensorflow import keras
from sklearn.datasets import fetch_california_housing #从sklearn中引用加州的房价数据
housing = fetch_california_housing()
print(housing.DESCR)
print(housing.data.shape)
print(housing.target.shape)
结果：
.. _california_housing_dataset:
California Housing dataset
--------------------------
**Data Set Characteristics:**
:Number of Instances: 20640
:Number of Attributes: 8 numeric, predictive attributes and the target
:Attribute Information:
- MedInc median income in block
- HouseAge median house age in block
- AveRooms average number of rooms
- AveBedrms average number of bedrooms
- Population block population
- AveOccup average house occupancy
- Latitude house block latitude
- Longitude house block longitude
:Missing Attribute Values: None
This dataset was obtained from the StatLib repository.
http://lib.stat.cmu.edu/datasets/
The target variable is the median house value for California districts.
This dataset was derived from the 1990 U.S. census, using one row per census
block group. A block group is the smallest geographical unit for which the U.S.
Census Bureau publishes sample data (a block group typically has a population
of 600 to 3,000 people).
It can be downloaded/loaded using the
:func:`sklearn.datasets.fetch_california_housing` function.
.. topic:: References
- Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,
Statistics and Probability Letters, 33 (1997) 291-297
(20640, 8)
(20640,)
#引用train_test_split对数据集进行拆分
# test_size 控制切分比例，默认切分比例3:1
from sklearn.model_selection import train_test_split
#拆分数据集，加载数据集后返回训练集以及测试集
x_train_all, x_test, y_train_all, y_test = train_test_split(housing.data, housing.target, random_state = 1)
#将训练集进行一次拆分为验证集和测试集
x_train, x_valid, y_train, y_valid = train_test_split(x_train_all, y_train_all, random_state=2)
print(x_train.shape, y_train.shape)
print(x_valid.shape, y_valid.shape)
print(x_test.shape, y_test.shape)
结果：
(11610, 8) (11610,)
(3870, 8) (3870,)
(5160, 8) (5160,)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
#对数据进行归一化处理
#由于transform处理处理数据时二维数组，所以要将数据转化一下
#x_train: [none, 28, 28] -> [none, 784]
#对于使用fit_transform 和transform 请参考我的TensorFlow中的博客
x_train_scaled = scaler.fit_transform(x_train)
x_valid_scaled = scaler.transform(x_valid)
x_test_scaled = scaler.transform(x_test)
#注意在归一化数据后，之后使用的数据要使用新的归一化数据
#使用序贯模型Sequential tf.keras.models.sequential()
# learning_rate: [1e-4, 3e-4, 1e-3, 3e-3, 1e-2, 3e-2]
# W = W + grad * leraning_rate
learning_rate = [1e-4, 3e-4, 1e-3, 3e-3, 1e-2, 3e-2]
histories = []
for lr in learning_rate:
model = keras.models.Sequential([
#keras.layers.Flatten(input_shape = x_train.shape[1:]),如果数据已经展平，真不用再使用flatten。
keras.layers.Dense(30, activation="relu",input_shape = x_train.shape[1:]),
keras.layers.Dense(1),
])
#设置优化函数，使用学习率
optimizer = keras.optimizers.Adam(lr)
#编译compile
model.compile(loss = "mean_squared_error", #损失函数：使用均方根误差(mse)
optimizer = optimizer , #优化函数
)
#使用回调函数
callbacks = [
keras.callbacks.EarlyStopping(patience=5, min_delta=1e-3),
]
#训练模型会，返回一个结果保存在history中
history = model.fit(x_train_scaled, y_train, epochs=50,
validation_data=(x_valid_scaled, y_valid),
callbacks=callbacks) #使用回调函数
histories.append(history)
绘图查看：
def plot_learning_curves(history):
pd.DataFrame(history.history).plot(figsize=(8, 5))
plt.grid(True)
plt.gca().set_ylim(0,1)
plt.show()
for lr, history in zip(learning_rate, histories):
print("Learning_rate: ",lr)
plot_learning_curves(history)
作者：一只工程狮
