Machine learning/Statistics Fundamentals

Point estimation is the process of calculating a statistic from a sample dataset.

• A point estimator is any function for a sample.
• Therefore, any statistic is a point estimator

Methods of finding point estimations

• Method of Moments
• Maximum Likelihood Estimators
• Bayes Estimators
• Maximum a posteriori estimator

Bayes Estimators

Using the Bayesian rule$$\begin{array}{cc}P(\theta |D)& =\frac{P(D|\theta )P(\theta )}{P(D)}\\ & =\frac{P(D|\theta )P(\theta )}{\int P(D|\theta )P(\theta )d\theta }\end{array}$$Where $D=(X_1,X_2,\cdots ,X_n)$ is the sample dataset.

$$P(D|\theta )=\prod _{i=1}^{n}P(X_i|\theta )$$