#### How we can improve model

# 1. Creating model: add more layers, increase numbers of hidden neurons, change activation functions

# 2. Compiling: change optimizer or its parameters (eg. learning rate)

# 3. Fitting: more epochs, more data

### How?

# from smaller model to larger model

Evaluating models

Typical workflow: build a model -> fit it -> evaulate -> tweak -> fit > evaluate -> ....

Building model: experiment Evaluation model: visualize

What can visualize?

• the data
• model itself
• the training of a model
• predictions

## The 3 sets (or actually 2 sets: training and test set)

tf.random.set_seed(999)

X_train, X_test = tf.split(tf.random.shuffle(X, seed=42), num_or_size_splits=[40, 10])

def plot_predictions(train_data = X_train,
                     train_labels = y_train,
                     test_data = X_test,
                     test_labels = y_test,
                     predictions = y_pred):
  """
  Plots training data, testing_data
  """
  plt.figure(figsize=(10, 7))
  plt.scatter(train_data, train_labels, c="blue", label='Training data')
  plt.scatter(test_data, test_labels, c="green", label="Testing data")
  plt.scatter(test_data, predictions, c="red", label="Predictions")
  plt.legend();

Common regression evaluation metrics

---

keyboard_arrow_down

Introduction

---

For regression problems:

• MAE
  • tf.keras.losses.MAE()
  • tf.metrics.mean_absolute_error()
  • great starter metrics for any regression problem
• MSE
  • tf.keras.losses.MSE()
  • tf.metrics.mean_square_error()
  • when larger errors are more significant that smaller errors
• Huber
  • tf.keras.losses.Huber()
  • combintion of MSE and MAE less sensitive to outliers than MSE

Take away: You should minimize the time between your experiments (that's way you should start with smaller models). The more experiments you do, the more things you figure out that don't work.

Tracking your experiments

One really good habit is to track the results of your experiments. There are tools to help us!

Resource: Try:

• Tensorboard - a component of Tensorflow library to help track modelling experiments
• Weights & Biases

Saving and loading models

Two formats:

• SavedModel format (including optimizer's step)
• HDF5 format

What about TensorFlow Serving format?

# Save the entire model using SavedModel
model_3.save("best_model_3_SavedModel")
# SavedModel is in principle protobuff)pb file

# Save model in HDF5 format:
model_3.save("best_model_3_HDF5.h5")

Load model

loaded_model_SM = tf.keras.models.load_model('/content/best_model_3_SavedModel')
loaded_model_SM.summary()

Getting dataset ready for tensorflow

1. Converting non-numerical columns

For example: Use pandas get_dummies() function

insurance_one_hot = pd.get_dummies(insurance,dtype="int32") #to avoid bool which generate problem with model fitting in TensorFlow
insurance_one_hot

# Create X and y values (features and labels)
y = insurance_one_hot['charges']
X = insurance_one_hot.drop('charges', axis=1)

#y = y.values  # This is not necessary
#X = X.values
#X, y, X.shape, y.shape

# Create training and test datasets
#my way:
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state=42)

Preprocessing data (normalization and standardization)

Preprocessing steps:

1. Turn all data into numbers
2. Make sure your tensors are in the right shape
3. Scale features (normalize or standardize) Neural networks tend to prefer normalization.

Normalization - adjusting values measured on different scales to a notionally common scale

Normalization

# Start from scratch
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf

## Borrow a few classes from sci-kit learn
from sklearn.compose import make_column_transformer
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder
from sklearn.model_selection import train_test_split

#Create column transformer
ct = make_column_transformer((MinMaxScaler(), ['age', 'bmi', 'children']), # turn all values in these columns between 0 and 1
                             (OneHotEncoder(handle_unknown='ignore', dtype="int32"), ['sex', 'smoker', 'region']))

# Create X and y
X = insurance.drop('charges', axis=1)
y = insurance['charges']

# Split datasets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Fit the column transformer on training data and apply to both datasets (train and test)
ct.fit(X_train)

# Transform data
X_train_normalize = ct.transform(X_train)
X_test_normalize = ct.transform(X_test)

Preprocessing data

ct = make_column_transformer((OneHotEncoder(dtype="int32"), ['Sex']), remainder="passthrough") #other columns unchangaed
ct.fit(X_train)

X_train_transformed = ct.transform(X_train)
X_test_transformed = ct.transform(X_test)

Predictions

valuation_predicts = model.predict(X_valuation_transformed)

(array([[ 9.441547],
        [10.451973],
        [10.48082 ],
        ...,
        [10.401164],
        [13.13452 ],
        [ 8.081818]], dtype=float32),
 (60411, 1))

valuation_predicts_squeezed = tf.squeeze(valuation_predicts)

submitt_data = pd.DataFrame({'id': data_test['id'],
                             'Rings': valuation_predicts_squeezed})

#Make sure that Min number of rings = 1
submitt_data.loc[submitt_data['Rings'] < 1, 'Rings'] = 1
submitt_data

Preprocessing data

ct = make_column_transformer((OneHotEncoder(dtype="int32"), ['Sex']), remainder="passthrough") #other columns unchangaed
ct.fit(X_train) 
X_train_transformed = ct.transform(X_train)
X_test_transformed = ct.transform(X_test)