mathematics アーカイブ - Data Processing Club

Fisher Information Explained: Python and Visual Illustrations

Definition of Fisher Information The Fisher information is defined as $$\mathrm{FisherInformation}(\theta_0)\stackrel{\text{def}}{=}-\mathbb{E}_{X\sim p(x\mid\theta_0)}\left[\frac{d^2}{d\theta^2}\log p(x\mid\theta)\bigg|_{\theta=\theta_0}\right].$$ Fisher information quantifies how precisely a model parameter can be estimated.A larger Fisher information means the parameter can be estimated more accurately,while a smaller Fisher information indicates that estimation is more difficult. Fisher information admits several equivalent interpretations. Equivalent Expressions $$\begin{align}&\mathrm{FisherInformation}(\theta_0) \\&\stackrel{\text{def}}{=}-\mathbb{E}_{X […]

Fisher Information Explained: Python and Visual Illustrations Read Post »

A Gentle Introduction to Machine Learning Theory

When I first began learning about machine learning, I struggled with the notion that lowering the loss on the training data does not necessarily guarantee performance on the test data, since training data and test data are different. In this article, I will carefully explain why the performance on test data is theoretically guaranteed, aiming

A Gentle Introduction to Machine Learning Theory Read Post »

An Intuitive Proof That Every Real Symmetric Matrix Can Be Diagonalized by an Orthogonal Matrix

It is well known that eigenvalues of a real symmetric matrix are real values, and eigenvectors of a real symmetric matrix form an orthonormal basis. This theorem plays important roles in many fields. For example, the principal component analysis relies on this theorem. Although every textbook on linear algebra contains a proof of this theorem,

An Intuitive Proof That Every Real Symmetric Matrix Can Be Diagonalized by an Orthogonal Matrix Read Post »