Multidimensional Scaling (MDS) projects data to a higher-dimensional space in order to increasing the distance between data points.

In self-organizing maps the objective is to minimize an energy (error) function.

In locally linear embedding (LLE), each data point is described as a linear combination of its neighbors.

The objective of scaling and projection methods is to project data into a lower-dimensional space, e.g. to allow better model selection.

Generative Topographic Mapping (GTM) is a non-linear latent variable model which maps latent variables to observations.

The objective in t-SNE is minimization of the KL divergence between the similarities of the (high-dimensional) inputs and the (low-dimensional) mapped outputs.

Projection pursuit performs projections such that the extracted signal is as Gaussian as possible.

Isomap uses a geodesic distance matrix, whose largest eigenvectors are the coordinates of the projected space.

Maximizing the negentropy maximizes the mutual information.

In general, ICA components are neither ranked nor orthogonal.

Criteria for independence are mutual information between components and non-Gaussianity of components.

Non-Gaussianity can be measured by the second cumulant.

Infomax maximizes the mutual information between components.

Fast ICA relies on whitening and rotating the data.

The ICA solution is unique.

For ICA, one needs more sources than observations.

In EM, one deals with problems that have hidden variables as well as model parameters.

The entropy of a probability distribution is always larger equal 0 and smaller equal 1.

In the M-step, the hidden variables are updated.

The EM algorithm can be applied to hidden Markov models and mixture models.

The E-step increases the lower bound on the log-likelihood.

In the E-step, the model parameters are updated.

Maximum Entropy requires minimal prior assumptions to assign a-priori-probabilities.

The E-step increases the upper bound on the log-likelihood.