on 14-Jul-2022 (Thu)

The minimal building blocks of DAGs consist of chains, forks, immoralities, two unconnected nodes, and two connected nodes.

3) Semisupervised Learning: Semisupervised learning aims to learn a better prediction as opposed to labelled data alone. For a supervised learning algorithm, labels are presented for all the observations in the dataset (i.e., with completely labeled training data), whereas for an unsupervised learning algorithm, labels are not required for the observation of the dataset. Semisupervised learning falls in between supervised or unsupervised learning algorithms. It is an approach that combines a small amount of labeled data with a large amount of unlabeled data during training when the cost of labeling work may render large, fully labeled training sets infeasible, whereas the acquisition of unlabeled data is relatively inexpensive. Generally, typical categories of semisupervised learning algorithms include the generative model method, the low-density separation method, the graph-based method, and the heuristic method. Some of the popular algorithms for semisupervised learning are summarized in Table S1 in the Supplementary Material

RNNs can be applied to predict future consumer behavior in regression and classification settings, for example, to predict interest in fashion brands or consumer lifetime value. We focus on predicting the probability P (o^u | x^u ₁ , . . . , x u T ) of a consumer u to place an order o^u , which we model as a binary classification problem. For instance, we could be interested in orders in general or of specific products. The resulting probability estimates can be used in a recommender system to deliver appropriate product recommendations and webshop contents.

In principle, one could evaluate the logistic regression model at every single time-step in the consumer history to determine the influence of individual events. However, this would involve the inefficient process of re-calculating features for every time-step. Calculations at timesteps t and t − 1 would be highly redundant: features at t represent the complete history until t and not only what happened in between t − 1 and t. Generally speaking, explaining the predictions of vector-based methods is more difficult than often as- sumed. This holds even for linear models like logistic regression. Features are often preprocessed, for example to binarize counts (Sec. 2). Furthermore, they are typically strongly correlated, making it troublesome to interpret individual coefficients [6]. Table 3 shows exemplary features weights in a logistic regression model used to predict order probabilities. If hundreds of features are utilized and are correlated and preprocessed, explaining the impact of consumer actions becomes a complex and confusing task

We have proposed an approach to apply RNNs to predict future consumer behavior in e-commerce. Consumer behavior is inherently sequential which makes RNNs a perfect fit. We are employing RNNs in production now which offers significant advantages over existing methods: reduced feature engineering; improved empirical performance; and better prediction explanations. In the future, predictions on the level of products and individual tastes will be in our focus, enabling sophisticated recommendation products. This will require richer input descriptions at individual time-steps. Likewise, more sophisticated RNN architectures will be promising for future research

ABM is a bottom-up modeling approach in which every agent of the system, theoretically, can be simulated to any level of granularity.

Semisupervised learning falls in between supervised or unsupervised learning algorithms. It is an approach that combines a small amount of labeled data with a large amount of unlabeled data during training when the cost of labeling work may render large, fully labeled training sets infeasible, whereas the acquisition of unlabeled data is relatively inexpensive.

3. Ambiguous: Surveys often fail to reveal the root causes of customer sentiment. In fact, scores can vary based on many outside factors, including geographical bias and industry shocks, making it difficult to perform reliable root-cause analysis using surveys alone

In a bottom-up approach, the individual elements of the system are first specified in detail with a predefined rule of behaviors and agent interactions

Consumer behavior is inherently sequential which makes RNNs a perfect fit.

Consumer behavior is inherently sequential which makes RNNs a perfect fit. We are employing RNNs in production now which offers significant advantages over existing methods: reduced feature engineering; improved empirical performance; and better prediction explanations

6.4 Fit the Model

We can now fit the model on example sequences. The code we developed for the echo sequence prediction problem generates random sequences. We could generate a large number of example sequences and pass them to the model’s fit() function. The dataset would be loaded into memory, training would be fast, and we could experiment with varied number of epochs vs dataset size and number of batches. A simpler approach is to manage the training process manually where one training sample is generated and used to update the model and any internal state is cleared. The number of epochs is the number of iterations of generating samples and essentially the batch size is 1 sample. Below is an example of fitting the model for 10,000 epochs found with a little trial and error.

7.1 The Stacked LSTM

The Stacked LSTM is a model that has multiple hidden LSTM layers where each layer contains multiple memory cells. We will refer to it as a Stacked LSTM here to differentiate it from the unstacked LSTM (Vanilla LSTM) and a variety of other extensions to the basic LSTM model.

[the success of deep neural networks] is commonly attributed to the hierarchy that is introduced due to the several layers. Each layer processes some part of the task we wish to solve, and passes it on to the next. In this sense, the DNN can be seen as a processing pipeline, in which each layer solves a part of the task before passing it on to the next, until finally the last layer provides the output

Additional hidden layers can be added to a Multilayer Perceptron neural network to make it deeper. The additional hidden layers are understood to recombine the learned representation from prior layers and create new representations at high levels of abstraction. For example, from lines to shapes to objects. A sufficiently large single hidden layer Multilayer Perceptron can be used to approximate most functions. Increasing the depth of the network provides an alternate solution that requires fewer neurons and trains faster. Ultimately, adding depth it is a type of representational optimization. Deep learning is built around a hypothesis that a deep, hierarchical model can be exponentially more efficient at representing some functions than a shallow one. — How to Construct Deep Recurrent Neural Networks, 2013

Given that LSTMs operate on sequence data, it means that the addition of layers adds levels of abstraction of input observations over time. In effect, chunking observations over time or representing the problem at different time scales. ... building a deep RNN by stacking multiple recurrent hidden states on top of each other. This approach potentially allows the hidden state at each level to operate at different timescale — How to Construct Deep Recurrent Neural Networks, 2013

7.1. The Stacked LSTM 79 RNNs are inherently deep in time, since their hidden state is a function of all previous hidden states. The question that inspired this paper was whether RNNs could also benefit from depth in space; that is from stacking multiple recurrent hidden layers on top of each other, just as feedforward layers are stacked in conventional deep networks. — Speech Recognition With Deep Recurrent Neural Networks, 2013 In the same work, they found that the depth of the network was more important than the number of memory cells in a given layer to model skill. Stacked LSTMs are now a stable technique for challenging sequence prediction problems. A Stacked LSTM architecture can be defined as an LSTM model comprised of multiple LSTM layers. An LSTM layer above provides a sequence output rather than a single value output to the LSTM layer below. Specifically, one output per input time step, rather than one output time step for all input time steps.