MA representation of an AR(2) process

An asymptotically stationary AR(2) process can be built by composing two asymptotically stationary AR(1) processes. Let \((X_t, t\in\mathbb Z)\) be an AR(1) process defined by:

\[ X_t = \rho_1 X_{t-1} + \varepsilon_t \]

where the innovation \(\varepsilon_t\) is a white noise with zero mean and variance \(\sigma_{\varepsilon}^2\), and \(|\rho_1| < 1\). Equivalently, inverting the lag polynomial¹, we have:

\[ X_t = (1-\rho_1L)^{-1}\varepsilon_t \]

We define the process \((Y_t, t\in\mathbb Z)\) as follows:

\[ Y_t = \rho_2 Y_{t-1} + X_t \]

with \(|\rho_2| < 1\). It is easily shown that this process is an asymptotically stationary AR(2) process. We have:

\[ (1-\rho_2 L) Y_t = X_t \]

that is, substituting the definition of \(X_t\) (obtained by inverting the lag polynomial \(1-\rho_1 L\)):

\[ (1-\rho_2 L) Y_t = (1-\rho_1 L)^{-1} \varepsilon_t \]

Multiplying by \(1-\rho_1 L\), we get:

\[ (1-\rho_1 L)(1-\rho_2 L) Y_t = \varepsilon_t \]

\[ \Leftrightarrow\left(1-(\rho_1+\rho_2) L+\rho_1\rho_2 L^2\right) Y_t = \varepsilon_t \]

\[ \Leftrightarrow Y_t = (\rho_1+\rho_2)Y_{t-1} - \rho_1\rho_2 Y_{t-2} + \varepsilon_t \]

This is indeed an AR(2), and the roots of the lag polynomial (\(1/\rho_1\) and \(1/\rho_2\)) are indeed larger than 1 in absolute value. Note that it is not always possible to write an AR(2) as the composition of two real AR(1) processes, simply because the roots of the lag polynomial are not always real, so that one would have to compose complex AR(1) processes.

The MA(\(\infty\)) representation can be obtained by inverting the lag polynomials \(1-\rho_2 L\) and \(1-\rho_1 L\):

\[ (1-\rho_2 L) Y_t = \sum_{i=0}^{\infty} \rho_1^i\varepsilon_{t-i} \]

\[ \Leftrightarrow Y_t = \sum_{j=0}^{\infty}\rho_2^j L^j \sum_{i=0}^{\infty} \rho_1^i\varepsilon_{t-i} \]

\[ \Leftrightarrow Y_t = \sum_{j=0}^{\infty}\sum_{i=0}^{\infty} \rho_2^j\rho_1^i\varepsilon_{t-i-j} \]

\[ \Leftrightarrow Y_t = \sum_{k=0}^{\infty}\left(\sum_{i=0}^{k} \rho_1^i\rho_2^{k-i}\right)\varepsilon_{t-k} \]

The point of this expression is that it shows clearly how the moving average coefficients:

\[ \theta_k = \sum_{i=0}^{k} \rho_1^i\rho_2^{k-i} \]

depend on (the inverses of) the roots of the lag polynomial, that is on the roots of the characteristic polynomial associated with the AR(2). These parameters tend to 0 as \(k\) tends to infinity. One can show that the sequence \((\theta_k)_{k\in\mathbb N}\) is absolutely summable. Indeed, assuming the two roots are distinct (if \(\rho_1=\rho_2=\rho\), the geometric sum simplifies to \(\theta_k=(k+1)\rho^k\) and the conclusion is unchanged):

\begin{equation*} \begin{split} \sum_{k=0}^{\infty} |\theta_k| &= \sum_{k=0}^{\infty} \left|\sum_{i=0}^{k} \rho_1^i\rho_2^{k-i}\right|\\ &= \sum_{k=0}^{\infty} \left|\rho_2^k\frac{1-\left(\frac{\rho_1}{\rho_2}\right)^{k+1}}{1-\frac{\rho_1}{\rho_2}}\right|\\ &= \sum_{k=0}^{\infty} \left|\frac{\rho_2^k-\frac{\rho_1}{\rho_2}\rho_1^k}{1-\frac{\rho_1}{\rho_2}}\right|\\ &= \left|\frac{\rho_2}{\rho_2-\rho_1}\right|\sum_{k=0}^{\infty} \left|\rho_2^k-\frac{\rho_1}{\rho_2}\rho_1^k\right|\\ &\leq \left|\frac{\rho_2}{\rho_2-\rho_1}\right|\sum_{k=0}^{\infty} \left(\left|\rho_2\right|^k+\left|\frac{\rho_1}{\rho_2}\right|\left|\rho_1\right|^k\right) < \infty \end{split} \end{equation*}

since the autoregressive coefficients \(\rho_1\) and \(\rho_2\) are strictly smaller than one in absolute value.

In practice, one does not use this definition of the moving average coefficients \((\theta_k)_{k\in\mathbb N}\) but a recursive definition. To this end, one directly inverts the lag polynomial \(\Phi(L) = 1-(\rho_1+\rho_2)L + \rho_1\rho_2L^2\). Its inverse is a lag polynomial of the form \(\Theta(L) = \sum_{i=0}^{\infty}\theta_iL^i\) such that:

\[ \Phi(L)\Theta(L) = 1 \]

that is, such that:

\[ \left(1-(\rho_1+\rho_2)L + \rho_1\rho_2L^2\right)\left(\sum_{i=0}^{\infty}\theta_iL^i\right) = 1 \]

or equivalently:

\[ \sum_{i=0}^{\infty}\theta_iL^i-(\rho_1+\rho_2)\sum_{i=0}^{\infty}\theta_iL^{i+1} + \rho_1\rho_2\sum_{i=0}^{\infty}\theta_iL^{i+2} = 1 \]

\[ \Leftrightarrow \theta_0 + \Bigl(\theta_1-(\rho_1+\rho_2)\theta_0\Bigr) L + \Bigl(\theta_2 -(\rho_1+\rho_2)\theta_1+\rho_1\rho_2\theta_0\Bigr) L^2 + \sum_{i=3}^{\infty}\Bigl(\theta_i-(\rho_1+\rho_2)\theta_{i-1}+\rho_1\rho_2\theta_{i-2}\Bigr) L^i= 1 \]

By identification², we obtain the following system:

\begin{cases} \theta_0 &= 1\\ \theta_1 - (\rho_1+\rho_2)\theta_0 &= 0\\ \theta_2 - (\rho_1+\rho_2)\theta_1 + \rho_1\rho_2 \theta_0 &= 0\\ &\vdots\\ \theta_k - (\rho_1+\rho_2)\theta_{k-1} + \rho_1\rho_2 \theta_{k-2} &= 0\\ &\vdots \end{cases}

and therefore:

\begin{cases} \theta_0 &= 1\\ \theta_1 &= (\rho_1+\rho_2)\theta_0\\ \theta_2 &= (\rho_1+\rho_2)\theta_1 - \rho_1\rho_2 \theta_0\\ &\vdots\\ \theta_k &= (\rho_1+\rho_2)\theta_{k-1} - \rho_1\rho_2 \theta_{k-2}\\ &\vdots \end{cases}

Note that the recursive definition of the moving average coefficients looks very much like the recursive definition of the autocovariance function of the AR(2). It is of course possible to recover a closed form for \(\theta_k\) from the recurrence equation for the moving average coefficients, in order to compare this recursive expression with the result obtained above. We know that the roots of the characteristic polynomial \(\chi(z) = z^2-(\rho_1+\rho_2)z + \rho_1\rho_2\) are \(\rho_1\) and \(\rho_2\). The general solution³, assuming the two roots are distinct⁴, is therefore of the form:

\[ \theta_k = \alpha \rho_1^k + \beta \rho_2^k \]

We can then identify the constants \(\alpha\) and \(\beta\) from the initial conditions of the recurrence, namely \(\theta_0=1\) and \(\theta_1=\rho_1+\rho_2\). We must therefore have:

\begin{cases} \alpha+\beta &= 1\\ \alpha\rho_1+\beta\rho_2 &= \rho_1+\rho_2 \end{cases}

so that, substituting the first equation into the second:

\[ \alpha\rho_1+(1-\alpha)\rho_2 = \rho_1+\rho_2 \]

\[ \Leftrightarrow \alpha(\rho_1-\rho_2)+\rho_2 = \rho_1+\rho_2 \]

\[ \Leftrightarrow \alpha = \frac{\rho_1}{\rho_1-\rho_2} \]

and hence

\[ \beta = -\frac{\rho_2}{\rho_1-\rho_2} \]

Finally, we obtain:

\[ \theta_k = \frac{\rho_1}{\rho_1-\rho_2} \rho_1^k - \frac{\rho_2}{\rho_1-\rho_2} \rho_2^k \]

One checks that this matches the expression given above:

\begin{equation*} \begin{split} \theta_k &= \sum_{i=0}^{k} \rho_1^i\rho_2^{k-i}\\ &= \rho_2^k \sum_{i=0}^{k} \left(\frac{\rho_1}{\rho_2}\right)^i\\ &= \rho_2^k \frac{1-\left(\frac{\rho_1}{\rho_2}\right)^{k+1}}{1-\frac{\rho_1}{\rho_2}}\\ &= \frac{\rho_2}{\rho_2-\rho_1}\left(\rho_2^k - \frac{\rho_1}{\rho_2}\rho_1^k\right)\\ &= \frac{\rho_2}{\rho_1-\rho_2}\left(\frac{\rho_1}{\rho_2}\rho_1^k - \rho_2^k\right)\\ &= \frac{\rho_1}{\rho_1-\rho_2} \rho_1^k - \frac{\rho_2}{\rho_1-\rho_2} \rho_2^k \end{split} \end{equation*}

The expressions obtained in the two previous sections assume \(\rho_1\neq\rho_2\): the geometric sum of the first approach divides by \(1-\frac{\rho_1}{\rho_2}\), and the general solution of the second is only valid for two distinct roots. Let us then see what becomes of the MA(\(\infty\)) representation when the two roots coincide, \(\rho_1=\rho_2=\rho\) with \(|\rho| < 1\). The process then reads:

\[ (1-\rho L)^2 Y_t = \varepsilon_t \]

\[ \Leftrightarrow Y_t = 2\rho Y_{t-1} - \rho^2 Y_{t-2} + \varepsilon_t \]

that is, the composition of two identical AR(1) processes.

The first approach gets simpler rather than harder: the \(k+1\) terms of the sum defining \(\theta_k\) are all equal, there is no geometric sum left to compute.

\[ \theta_k = \sum_{i=0}^{k} \rho^i\rho^{k-i} = \sum_{i=0}^{k} \rho^k = (k+1)\rho^k \]

The second approach leads to the same result. The characteristic polynomial \(\chi(z) = z^2-2\rho z+\rho^2 = (z-\rho)^2\) has a double root, and the general solution of the recurrence equation then takes the form⁵:

\[ \theta_k = (\alpha + \beta k)\rho^k \]

The initial conditions \(\theta_0=1\) and \(\theta_1=2\rho\) immediately give \(\alpha=1\), then \((\alpha+\beta)\rho = 2\rho\), that is \(\beta=1\). We recover \(\theta_k = (k+1)\rho^k\).

Finally, this result can be seen as a limit. Rewriting the expression obtained for two distinct roots in compact form:

\[ \theta_k = \frac{\rho_1}{\rho_1-\rho_2} \rho_1^k - \frac{\rho_2}{\rho_1-\rho_2} \rho_2^k = \frac{\rho_1^{k+1}-\rho_2^{k+1}}{\rho_1-\rho_2} \]

one recognises the difference quotient of the function \(z\mapsto z^{k+1}\) between \(\rho_2\) and \(\rho_1\). As \(\rho_2\) tends to \(\rho_1=\rho\), this difference quotient tends to the derivative, \((k+1)\rho^k\): the MA(\(\infty\)) representation therefore has no discontinuity at \(\rho_1=\rho_2\).

The sequence \((\theta_k)_{k\in\mathbb N}\) remains absolutely summable, this time with a closed form for the sum:

\[ \sum_{k=0}^{\infty} |\theta_k| = \sum_{k=0}^{\infty} (k+1)|\rho|^k = \frac{1}{\left(1-|\rho|\right)^2} \]

One last remark. Unlike the AR(1) case, where the moving average coefficients decrease from the first lag onwards, the sequence \((|\theta_k|)\) may here start by increasing. Indeed \(|\theta_{k+1}|>|\theta_k|\) if and only if \((k+2)|\rho|^{k+1}>(k+1)|\rho|^k\), that is if and only if:

\[ k < \frac{2|\rho|-1}{1-|\rho|} \]

The impulse response is therefore monotonically decreasing when \(|\rho|\leq\frac{1}{2}\), but hump-shaped when \(|\rho|>\frac{1}{2}\): for \(\rho=0.8\) the maximum is reached at \(k=3\). The shock then takes several periods to produce its maximal effect, which the composition of two AR(1) processes makes intuitive: the effect must first propagate from the first process to the second.

We built the AR(2) by composing two real AR(1) processes, which restricts its generality: as noted in the first section, the roots of the lag polynomial are not always real. Let us therefore take the problem from the other end, starting from an arbitrary AR(2):

\[ Y_t = \varphi_1 Y_{t-1} + \varphi_2 Y_{t-2} + \varepsilon_t \]

The associated characteristic polynomial is \(\chi(z) = z^2-\varphi_1 z - \varphi_2\), whose roots \(\lambda_1\) and \(\lambda_2\) satisfy:

\begin{cases} \lambda_1+\lambda_2 &= \varphi_1\\ \lambda_1\lambda_2 &= -\varphi_2 \end{cases}

When they are real, we recover the previous sections by setting \(\lambda_1=\rho_1\) and \(\lambda_2=\rho_2\). They are complex conjugates if and only if the discriminant is negative:

\[ \Delta = \varphi_1^2+4\varphi_2 < 0 \]

Let us then write \(\lambda_1 = re^{i\omega}\) and \(\lambda_2 = re^{-i\omega}\). The modulus and the argument can be read directly from the relations between coefficients and roots, since \(r^2 = \lambda_1\lambda_2\) and \(2r\cos\omega = \lambda_1+\lambda_2\):

\[ r = \sqrt{-\varphi_2} \qquad \text{and} \qquad \cos\omega = \frac{\varphi_1}{2\sqrt{-\varphi_2}} \]

The process is asymptotically stationary if and only if \(r < 1\), that is if and only if \(\varphi_2>-1\).

The two roots being distinct, since their imaginary part is not zero, the general solution of the recurrence equation keeps the form obtained above and the compact form applies as it stands:

\[ \theta_k = \frac{\lambda_1^{k+1}-\lambda_2^{k+1}}{\lambda_1-\lambda_2} \]

The coefficients \(\theta_k\) are indeed real, as they must be: numerator and denominator are both differences of complex conjugates, hence both purely imaginary. Substituting the polar form:

\begin{equation*} \begin{split} \theta_k &= \frac{r^{k+1}\left(e^{i(k+1)\omega}-e^{-i(k+1)\omega}\right)}{r\left(e^{i\omega}-e^{-i\omega}\right)}\\ &= \frac{r^{k+1}\times 2i\sin\bigl((k+1)\omega\bigr)}{r\times 2i\sin\omega}\\ &= r^{k}\,\frac{\sin\bigl((k+1)\omega\bigr)}{\sin\omega} \end{split} \end{equation*}

One checks that \(\theta_0=1\) and that \(\theta_1 = r\frac{\sin 2\omega}{\sin\omega} = 2r\cos\omega = \varphi_1\), as the recurrence requires.

This expression reads effortlessly: the impulse response is a damped oscillation, with damping factor \(r\) and pseudo-period \(\frac{2\pi}{\omega}\). The moving average coefficients therefore change sign regularly, which cannot happen with two positive real roots. For \(\varphi_1=1.2\) and \(\varphi_2=-0.5\), we get \(r\simeq 0.707\) and \(\omega\simeq 0.558\) radians, that is a pseudo-period of about eleven periods.

The sequence remains absolutely summable, the bound being immediate since \(\left|\sin\bigl((k+1)\omega\bigr)\right|\leq 1\):

\[ \sum_{k=0}^{\infty}|\theta_k| \leq \frac{1}{\left|\sin\omega\right|}\sum_{k=0}^{\infty} r^k = \frac{1}{(1-r)\left|\sin\omega\right|} < \infty \]

There remains the remark of the first section: such a process cannot be written as the composition of two real AR(1) processes. The decomposition does exist, however, but with complex conjugate coefficients, and it is precisely this conjugation that guarantees that the composed process is itself real.