on 05-Jun-2024 (Wed)

As the number of hyperparameters and their range grow, the search space becomes exponentially complex, and tuning the models manually or by grid-search becomes impractical . Bayesian optimization for hyperparameter tuning provides hyperparameters (step 1) iteratively based on previous performance (Shahriari, Swersky, Wang, Adams, & De Freitas, 2015). We use Bayesian optimization to search the hyperparameter space for our model extensively.

Interestingly, the LSTM model beats both benchmark model across the board and performs well at all lift levels.

Below are some common configurations for the batch size:

batch size=1 :
Weights are updated after each sample and the procedure is called stochas- tic gradient descent.

batch size=32 :
weights are updated after a specified number of samples and the procedure is called mini-batch gradient descent. Common values are 32, 64, and 128, tailored to the desired efficiency and rate of model updates. If the batch size is not a factor of the number of samples in one epoch, then an additional batch size of the left over samples is run at the end of the epoch.

batch size=n :
Where n is the number of samples in the training dataset. Weights are updated at the end of each epoch and the procedure is called batch gradient descent

As the number of hyperparameters and their range grow, the search space becomes exponentially complex, and tuning the models manually or by grid-search becomes impractical . Bayesian optimization for hyperparameter tuning provides hyperparameters (step 1) iteratively based on previous performance

We havte to ensure that there is no non-causal association flowing between 𝑋 and 𝑌 . This is true if 𝑋 and 𝑌 are d-separated in the augmented graph where we remove outgoing edges from 𝑋 .

ML engineers

They need just enough data engineering skills to build (and schedule the ETL for) feature datasets for their models, but not to construct a petabyte-scale streaming ingestion framework.