Posts

Information theory essentials: entropy, cross-entropy, joint/conditional entropy, KL divergence, mutual information.

High-dimensional data pitfalls (CoD) and core decision theory: risk, posterior-based rules, reject option.

Change-of-variables for PDFs: scalar and multivariate cases, Jacobian determinant, convolution and CLT.

Bayesian probability: quantifying uncertainty, Bayes’ rule, prior/likelihood/posterior, marginal probability.

Probability fundamentals: rules, PDFs, expectation, variance, covariance, Gaussian distribution.

Polynomial regression from least squares to Bayesian view: closed-form, regularization, predictive uncertainty.

Diffusion Models (DMs) include two processes: forward and backward. Forward process General idea Degrading input data using noise iteratively, forward in time (i.e., $t$ increases). Given image $x_0 \sim q(x_0)$, which called data distribution, forward process gradually adds Gauss noise thru $T$ time steps and produces latent $x_T$. At each time step $t$, we sample Gauss noise that following the distribution $\mathcal{N}(\sqrt{1 - \beta_t} x_{t-1}, \beta_t)$, where the hyper-parameters $0 < \beta_{1:T} < 1$ represent the variance of noise incorporated at each time step....

Determinants, eigenvalues, eigenvectors: geometric meaning, finding methods, and linear transformation essence.

Linear independence, span, basis, dimension: fundamental concepts for vector spaces and subspaces.

Four fundamental subspaces: row space, column space, nullspace, left nullspace with dimensions and relationships.