on 06-Jun-2024 (Thu)

Chapter 3 How to Prepare Data for LSTMs

3.0.1 Lesson Goal
The goal of this lesson is to teach you how to prepare sequence prediction data for use with LSTM models. After completing this lesson, you will know: How to scale numeric data and how to transform categorical data. How to pad and truncate input sequences with varied lengths. How to transform input sequences into a supervised learning problem.

ignorability - we can make this assumption realistic by running randomized experiments, which force the treatment to not be caused by anything but a coin toss, so then we have the causal structure shown in Figure 2.2.

Truncated Backpropagation Through Time, or TBPTT, is a modified version of the BPTT training algorithm

Association still flows in exactly the same way in Bayesian networks as it does in causal graphs, though. In both, association flows along chains and forks, unless a node is conditioned on. And in both, a collider blocks the flow of association, unless it is conditioned on.

model amalgamation. In this approach, models are broken into compos- able units that share and adapt components to achieve different objectives with the same initial data. The components are interconnected in a variety of connectivity patterns, in which each component learns communication interfaces between the models through design, without the necessity of a backend application

In addition, model amalgamation can be used to train Internet of Things (IoT) devices for data enrichment, turning IoT sensors from static to dynamically learn- ing devices—a technique called model fusion

enterprise production is moving toward automatic learning for model development

Model development for production continues to be a combination of automatic and hand-designed learning—which is often crucial for proprietary needs or advan- tages. But designing by hand does not mean starting from scratch; typically, you would start with a stock model and make tweaks and adjustments. To do this effectively, you need to know how the model works and why it works that way, the concepts that under- lie its design, and the pros and cons of alternative building blocks you will learn from other SOTA models

formerly state-of-the-art (SOTA) models

advancements in design patterns for structured data evolved with the introduction of the wide-and-deep model pattern, outlined in “Wide & Deep Learning for Recommender Systems” (https://arxiv.org/abs/1606.07792) by Heng- Tze Cheng et. al, at the technology-agnostic research group Google Research, in 2016