Prob-Stats

Information theory essentials: entropy, cross-entropy, joint/conditional entropy, KL divergence, mutual information.

Bayesian probability: quantifying uncertainty, Bayes’ rule, prior/likelihood/posterior, marginal probability.

Probability fundamentals: rules, PDFs, expectation, variance, covariance, Gaussian distribution.