Parameter Estimation

This chapter systematically studies the basic ideas of parameter estimation, point estimation, interval estimation, the method of moments, the method of maximum likelihood, and the properties of estimators.

Point Estimation and Estimators

Point estimation: using sample statistics to estimate the value of population parameters
Estimator: a statistic used to estimate a parameter
Estimate: the specific value of the estimator

Method of Moments and Method of Maximum Likelihood

Method of moments: set the sample moments equal to the population moments, solve the equations to obtain the estimator
Method of maximum likelihood: construct the likelihood function, maximize it to obtain the estimator

Properties of Estimators

Unbiasedness: $E(\hat{\theta})=\theta$
Efficiency: minimum variance
Consistency: converges to the parameter as the sample size increases

Interval Estimation and Confidence Interval

Interval estimation: gives an interval in which the parameter is likely to fall
Confidence interval: the probability that the interval contains the parameter is the confidence level
Confidence intervals for the mean and variance of a single normal population
Confidence intervals for the difference of means and the ratio of variances of two normal populations

Exercises

Let the population $X \sim N(\mu, \sigma^2)$ , the sample mean is $\overline{X}$ , and the variance is $S^2$ . Write the unbiased estimators for $\mu$ and $\sigma^2$ .
Use the method of moments to estimate the parameter: given a sample $x_1, x_2, \dots, x_n$ from $X \sim Poisson(\lambda)$ .
Use the method of maximum likelihood to estimate the parameters: given a sample $x_1, x_2, \dots, x_n$ from $X \sim N(\mu, \sigma^2)$ .
Write the $1-\alpha$ confidence interval for the mean $\mu$ of a normal population (variance known).
Given two estimators $\hat{\theta}_1, \hat{\theta}_2$ , if $Var(\hat{\theta}_1)<Var(\hat{\theta}_2)$ , which one is more efficient?

Reference Answers

1. Unbiased estimators

The unbiased estimator for $\mu$ is $\overline{X}$ , and for $\sigma^2$ is $S^2 = \frac{1}{n-1}\sum (x_i-\overline{X})^2$

2. Method of moments

$E(X)=\lambda$ , set the sample mean equal to $\lambda$ , so $\hat{\lambda}=\overline{X}$

3. Method of maximum likelihood

The maximum likelihood estimator for $\mu$ is $\overline{X}$ , and for $\sigma^2$ is $\frac{1}{n}\sum (x_i-\overline{X})^2$

4. Confidence interval

$\overline{X} \pm z_{\alpha/2}\frac{\sigma}{\sqrt{n}}$

5. Efficiency

$\hat{\theta}_1$ is more efficient