Cramer-Rao Lower Bound关于克拉美罗界的很好的讲解


Chapter 3 Cramer-Rao Lower Bound

What is the Cramer-Rao Lower Bound
Abbreviated: CRLB or sometimes just CRB

CRLB is a lower bound on the variance of any unbiased estimator:

If θ? is an unbiased estimator of θ , then
2 σθ ? (θ ) ≥ CRLB ? (θ ) ? σ ? (θ ) ≥ CRLB ? (θ )

θ

θ

θ

The CRLB tells us the best we can ever expect to be able to do (w/ an unbiased estimator)

Some Uses of the CRLB
1. Feasibility studies ( e.g. Sensor usefulness, etc.) ? Can we meet our specifications? 2. Judgment of proposed estimators ? Estimators that don’t achieve CRLB are looked down upon in the technical literature 3. Can sometimes provide form for MVU est. 4. Demonstrates importance of physical and/or signal parameters to the estimation problem
e.g. We’ll see that a signal’s BW determines delay est. accuracy ? Radars should use wide BW signals

3.3 Est. Accuracy Consideration
Q: What determines how well you can estimate θ ? Recall: Data vector is x
samples from a random process that depends on an θ ? the PDF describes that dependence: p(x;θ )

Clearly if p(x;θ ) depends strongly/weakly on θ …we should be able to estimate θ well/poorly. See surface plots vs. x & θ for 2 cases: 1. Strong dependence on θ 2. Weak dependence on θ ? Should look at p(x;θ ) as a function of θ for fixed value of observed data x

Surface Plot Examples of p(x;θ )

Ex. 3.1: PDF Dependence for DC Level in Noise
x[0] = A + w[0]
w[0] ~ N(0,σ2)

Then the parameter-dependent PDF of the data point x[0] is:

p (x[0]; A) =

1 2πσ 2

? ( x[0] ? A) 2 ? exp ? ? ? 2 2σ ? ? ? ?

Say we observe x[0] = 3… So “Slice” at x[0] = 3

p(x[0]=3;θ )

3 A x[0]

A

Define: Likelihood Function (LF)
The LF = the PDF p(x;θ ) …but as a function of parameter θ w/ the data vector x fixed

We will also often need the Log Likelihood Function (LLF): LLF = ln{LF} = ln{ p(x;θ )}

LF Characteristics that Affect Accuracy
Intuitively: “sharpness” of the LF sets accuracy… But How??? Sharpness is measured using curvature: ? ? 2 ln p (x ; θ )

2 x = given data θ = true value

Curvature ↑ ? PDF concentration ↑ ? Accuracy ↑ But this is for a particular set of data… we want “in general”: So…Average over random vector to give the average curvature:
2 ? ? ? ln p (x ; θ ? E? 2 ? ? θ ?

)? ?

? ? ?θ

“Expected sharpness of LF”
= true value

E{?} is w.r.t p(x;θ )

3.4 Cramer-Rao Lower Bound
Theorem 3.1 CRLB for Scalar Parameter
Assume “regularity” condition is met: E ? Then σ 2 ≥ θ?
1 ? ? ? 2 ln p (x;θ ) ? ? ? E? ? 2 ? ? ?θ ? ?θ

? ? ln p( x;θ ) ? ? = 0 ?θ ?θ ? ?

= true value

Right-Hand Side is CRLB

E{?} is w.r.t p(x;θ )
2 ? ? 2 ln p (x;θ ) ? ? ln p (x;θ ) ? ? E? p( x;θ )dx ?=∫ 2 2 ? ? ?θ ?θ ? ?

Steps to Find the CRLB
1. Write log 1ikelihood function as a function of θ: ? ln p(x;θ ) 2. Fix x and take 2nd partial of LLF: ? ?2ln p(x;θ )/?θ 2 3. If result still depends on x: ? Fix θ and take expected value w.r.t. x ? Otherwise skip this step 4. Result may still depend on θ: ? Evaluate at each specific value of θ desired. 5. Negate and form reciprocal

Example 3.3 CRLB for DC in AWGN
x[n] = A + w[n], n = 0, 1, … , N – 1
w[n] ~ N(0,σ2) & white

Need likelihood function:
p (x ; A ) =
N ?1



1 2πσ 1
2

n =0

? ? (x [n ] ? A )2 exp ? 2 2 σ ? ?

? ? ? ? ? ? ? ? ? ?

Due to whiteness

=

(2πσ )

N 2 2

? N ?1 2 ? ? ∑ (x [n ] ? A ) exp ? n = 0 ? 2σ 2 ? ?

Property of exp

Now take ln to get LLF:
N? N ?1 ? 1 2 ( ) [ ] ln p ( x; A) = ? ln ? 2πσ 2 2 ? ? x n ? A 2 ∑ ? ? 2σ n =0 ? ? $ !!! #!!! " $!! #!! " ? (~~) =0 ?A ? (~~) =? ?A

(

)

Now take first partial w.r.t. A:
1 ? ln p ( x; A) = ?A σ2
N ?1 n =0

sample mean

∑ (x[n] ? A) = σ 2 (x ? A)
N

N

(!)

Now take partial again:
?2 ?A
2

Doesn’t depend on x so we don’t need to do E{?}

ln p ( x; A) = ?

σ2

Since the result doesn’t depend on x or A all we do is negate and form reciprocal to get CRLB:
CRLB = 1 ? ? ? ? 2 ln p (x;θ ) ? ? E? ? 2 ? ? ?θ ?θ ? =
= true value

σ2
N

?} ≥ var{ A

σ2
N

CRLB For fixed N & σ
2

? Doesn’t depend on A ? Increases linearly with σ 2 ? Decreases inversely with N
A

CRLB For fixed N

CRLB

Doubling Data Halves CRLB! For fixed σ 2

σ2

N

Continuation of Theorem 3.1 on CRLB
There exists an unbiased estimator that attains the CRLB iff:
? ln p ( x;θ ) = I (θ )[g ( x ) ? θ ] ?θ

(!)

for some functions I(θ ) and g(x) Furthermore, the estimator that achieves the CRLB is then given by:

θ? = g ( x )
1 ? θ} = var{ = CRLB with I (θ )

Since no unbiased estimator can do better… this is the MVU estimate!! This gives a possible way to find the MVU: ? Compute ?ln p(x;θ )/?θ (need to anyway) ? Check to see if it can be put in form like (!) ? If so… then g(x) is the MVU esimator

Revisit Example 3.3 to Find MVU Estimate
For DC Level in AWGN we found in (!) that:
? N ln p ( x; A) = 2 (x ? A) ?A σ
N
Has form of I(A)[g(x) – A]

I ( A) =

σ

2

?} = ? var{ A

σ2
N

= CRLB

1 θ? = g ( x ) = x = N

N ?1 n =0

∑ x[n]

So… for the DC Level in AWGN: the sample mean is the MVUE!!

Definition: Efficient Estimator
An estimator that is: ? unbiased and ? attains the CRLB is said to be an “Efficient Estimator” Notes: ? Not all estimators are efficient (see next example: Phase Est.) ? Not even all MVU estimators are efficient
So… there are times when our “1st partial test” won’t work!!!!

Example 3.4: CRLB for Phase Estimation
This is related to the DSB carrier estimation problem we used for motivation in the notes for Ch. 1 Except here… we have a pure sinusoid and we only wish to estimate only its phase Signal Model:
x[n ] = A cos(2πf o n + φo ) + w[n ] $!! ! #!! ! "
s[ n;φo ]

AWGN w/ zero mean & σ 2

Signal-to-Noise Ratio: Signal Power = A2/2 Noise Power = σ 2 Assumptions:
1. 0 < fo < ? ( fo is in cycles/sample)

SNR =

A2 2σ 2

2. A and fo are known (we’ll remove this assumption later)

Problem: Find the CRLB for estimating the phase. We need the PDF:
p (x ; φ ) = 1 ? N ?1 2 ? ? ∑ (x [n ] ? A cos( 2π f o n + φ ) ) exp ? n = 0 ? 2σ 2 ? ? ? ? ? ? ? ?
Exploit Whiteness and Exp. Form

(2πσ )

N 2 2

Now taking the log gets rid of the exponential, then taking partial derivative gives (see book for details):
A ? ln p (x ; φ ) ? A N ?1? ? = 2 ∑ ? x [n ]sin( 2π f o n + φ ) ? sin( 4π f o n + 2φ ) ? 2 ?φ σ n =0 ? ?
2

Taking partial derivative again:
? 2 ln p (x ; φ ) ?φ
2

=

? A

N ?1 n =0

σ

2

∑ (x [n ]cos( 2π f o n + φ ) ? A cos( 4π f o n + 2φ ) )
Still depends on random vector x… so need E{}

Taking the expected value:
2 ? ? A ? ? ln p (x ; φ ) ? ? ? E? ? = E? 2 2 ? ? ?φ ?σ ? ? N ?1

? ∑ (x [n ]cos( 2π f o n + φ ) ? A cos( 4π f o n + 2φ ) )? n =0 ?

=

A

N ?1 n =0

σ

2

∑ (E {x [n ]}cos( 2π f o n + φ ) ? A cos( 4π f o n + 2φ ) )
E{x[n]} = A cos(2π fon + φ )

So… plug that in, get a cos2 term, use trig identity, and get
2 ? A2 ? ? ln p (x ; φ ) ? ? ? E? ?= 2 2 ? ? ?φ ? 2σ ?

? N ?1 ? ∑1? ? ? n =0

N ?1

? NA 2 ∑ cos( 4π f o n + 2φ ) ? ≈ 2σ 2 = N × SNR ? n ?0 ?

=N

<< N if fo not near 0 or ?

N-1

n

Now… invert to get CRLB:

?} ≥ var{φ

1 N × SNR

Non-dB

CRLB

Doubling Data Halves CRLB! For fixed SNR

N

CRLB For fixed N

Doubling SNR Halves CRLB!

Halve CRLB for every 3B in SNR

SNR (non-dB)

Does an efficient estimator exist for this problem? The CRLB theorem says there is only if ? ln p( x;θ ) = I (θ )[g ( x ) ? θ ] ?θ Our earlier result was:
A ? ln p (x ; φ ) ? A N ?1? ? = 2 ∑ ? x [n ]sin( 2π f o n + φ ) ? sin( 4π f o n + 2φ ) ? 2 ?φ σ n =0 ? ?
2

Efficient Estimator does NOT exist!!!
?} → CRLB We’ll see later though, an estimator for which var{φ as N → ∞ or as SNR → ∞

?} var{φ
CRB

N

Such an estimator is called an “asymptotically efficient” estimator (We’ll see such a phase estimator in Ch. 7 on MLE)


相关文档

Chapter 3 Cramer-Rao Lower Bound
§1.7 克拉默法则(Cramer)
Cramer-Rao Lower Bounds for Time Delay and Doppler Shift Estimation
克拉默(Cramer)法则
高动态多普勒频率估计及其Cramer-Rao界
3.2.2-基于克拉美罗不等式的MVU估计
1998 On the Cramer-Rao bound under parametric constraints
1.4克拉默(Cramer)法则
THE CRAMER-RAO BOUND AND ML ESTIMATE FOR DATA-AIDED CHANNEL ESTIMATION IN KSP-OFDM
克拉默_Cramer_法则的推广及Matlab实现
电脑版