Week 2 — Lecture notes
Signals as vectors, systems as maps
Other formats:
📄 PDFGoals for this week
Make “signal” and “system” precise.
Understand linearity.
How to measure the size of a signal: energy & power.
How to measure the angle between signals.
Useful signals: Dirac delta and friends.
Signals as vectors
We’ve seen signals & systems through examples. In this lecture, we’ll develop a language to represent these things mathematically. At first this may seem quite abstract, but it ends up being extremely powerful, as we’ll see throughout the semester.
We will build on what we know from linear algebra.
A square matrix A maps a vector to another vector of the same dimension:
\underset{\textstyle \mathbb{R}^{n}}{y} \;=\; \underset{\textstyle \mathbb{R}^{n\times n}}{A}\; \underset{\textstyle \mathbb{R}^{n}}{u}
If n = 2, we can visualise A in the plane as scaling, rotating, skewing and reflecting u to produce y:
Scaling and rotating u to produce y = Au (drawn as sketched).
We measure the scaling using a norm:
\|u\| \;=\; \sqrt{u \cdot u} \;=\; \sqrt{u_1^{2} + u_2^{2}}.
We measure the rotation using the dot product:
\cos\theta \;=\; \frac{u \cdot y}{\|u\|\,\|y\|}.
How do we extend these ideas to signals?
Given a continuous signal, we can approximate one period as an infinite dimensional vector.
\dots\; x_{-1}\;\; x_{0}\;\; x_{1}\;\; x_{2}\;\; x_{3}\;\; x_{4}\;\; x_{5}\;\; x_{6}\;\; x_{7}\; \dots
The dot product between these vectors is
\sum_{t=-\infty}^{\infty} u_t\, y_t \;=\; u \cdot y
If we let the sampling become infinitesimally fine, we integrate over the samples, and have an inner product:
\langle u, y\rangle \;:=\; \int_{-\infty}^{\infty} u(t)\,y(t)\,\mathrm{d}t.
For complex signals, we add a complex conjugate:
\boxed{\;\langle u, y\rangle \;:=\; \int_{-\infty}^{\infty} u(t)\,\overline{y}(t)\,\mathrm{d}t\;}
Some useful properties of the inner product:
- \langle u_1 + u_2,\, y\rangle \;=\; \langle u_1, y\rangle + \langle u_2, y\rangle
- \langle \alpha u,\, y\rangle \;=\; \alpha\,\langle u, y\rangle
- \langle u,\, \beta y\rangle \;=\; \overline{\beta}\,\langle u, y\rangle
- \langle u, y\rangle \;=\; \overline{\langle y, u\rangle}
So the inner product is linear in its first argument and conjugate-linear in its second.
The size of a signal: energy & power
We can use the inner product to measure the size of a signal:
\boxed{\;E(u) \;=\; \int_{-\infty}^{\infty} |u(t)|^{2}\,\mathrm{d}t\;}
This is the signal energy. The complex conjugate in \langle u, y\rangle guarantees E(u) is real.
Example:
u(t) = \begin{cases} e^{-t}, & t \geq 0 \\[2pt] 0, & t < 0. \end{cases}
\begin{aligned} E(u) &= \int_{0}^{\infty} e^{-2t}\,\mathrm{d}t \\[4pt] &= \left. \frac{-1}{2}\, e^{-2t} \right|_{0}^{\infty} \\[4pt] &= \frac{1}{2}\, e^{0} \;=\; \frac{1}{2}. \end{aligned}
What if u(t) = A\cos(\omega t)?
\int_{-\infty}^{\infty} A^{2}\cos^{2}(\omega t)\,\mathrm{d}t \;=\; \infty \;\dots
Take the average power:
\text{energy} = \displaystyle\int \text{power}\;\mathrm{d}t
\boxed{\;P(u) \;=\; \lim_{W \to \infty} \frac{1}{2W} \int_{-W}^{W} |u(t)|^{2}\,\mathrm{d}t.\;}
If u(t) = u(t + T) for some T, each period contributes the same, so we get the average.
\begin{aligned} \lim_{W \to \infty} \frac{1}{2W} \int_{-W}^{W} |u(t)|^{2}\,\mathrm{d}t &= \lim_{k \to \infty} \frac{1}{2kT}\left( \cdots \underbrace{\int_{-kT}^{-(k-1)T} |u(t)|^{2}\,\mathrm{d}t}_{\textstyle =\int_{0}^{T}|u(t)|^{2}\,\mathrm{d}t} + \underbrace{\int_{-(k-1)T}^{-(k-2)T} |u(t)|^{2}\,\mathrm{d}t}_{\textstyle =\int_{0}^{T}|u(t)|^{2}\,\mathrm{d}t} + \cdots + \int_{-T}^{0} |u(t)|^{2}\,\mathrm{d}t + \cdots + \int_{(k-1)T}^{kT} |u(t)|^{2}\,\mathrm{d}t \right) \\[10pt] &= \lim_{k \to \infty} \frac{1}{2k} \cdot 2k \cdot \frac{1}{T} \int_{0}^{T} |u(t)|^{2}\,\mathrm{d}t \\[6pt] &= \boxed{\;P(u) = \frac{1}{T} \int_{0}^{T} |u(t)|^{2}\,\mathrm{d}t.\;} \end{aligned}
This is the average power of a periodic signal.
Example: what is the power of u(t) = A\cos(\omega t)
\text{Period} = \frac{2\pi}{\omega}
\begin{aligned} P(u) &= \frac{\omega}{2\pi} \int_{0}^{\frac{2\pi}{\omega}} A^{2}\cos(\omega t)^{2}\,\mathrm{d}t \\[6pt] &= \frac{\omega}{2\pi} \int_{0}^{\frac{2\pi}{\omega}} A^{2}\,\frac{1 + \cos(2\omega t)}{2}\,\mathrm{d}t \\[6pt] &= \frac{\omega}{2\pi} \left[ A^{2}\,\frac{t}{2} \;+\; A^{2}\,\frac{\sin(2\omega t)}{4\omega} \right]_{0}^{\frac{2\pi}{\omega}} \\[6pt] &= \frac{\omega}{2\pi} \left( A^{2}\,\frac{\frac{2\pi}{\omega}}{2} \;+\; A^{2}\,\frac{\overset{\textstyle 0}{\overbrace{\sin\!\left(\tfrac{4\pi}{\omega}\,\omega\right)}}}{4\omega} \right) \\[6pt] &= \frac{A^{2}}{2} \end{aligned}
Example: what is the power of u(t) = \begin{cases} e^{-t}, & t \geq 0 \\[2pt] 0, & t < 0 \end{cases} ?
\begin{aligned} P(u) &= \lim_{W \to \infty} \frac{1}{2W} \int_{-W}^{W} |u(t)|^{2}\,\mathrm{d}t \\[6pt] &= \lim_{W \to \infty} \frac{1}{2W} \int_{0}^{W} e^{-2t}\,\mathrm{d}t \\[6pt] &= \lim_{W \to \infty} \frac{1}{2W} \left[ \frac{-1}{2}\,e^{-2t} \right]_{0}^{W} \\[6pt] &= \lim_{W \to \infty} \underbrace{\frac{1}{2W}}_{\textstyle \to 0} \left[ \underbrace{\frac{1}{2}\,e^{0}}_{\textstyle \frac{1}{2}} \;-\; \underbrace{\frac{1}{2}\,e^{-2W}}_{\textstyle \to 0} \right] \\[6pt] &= 0. \end{aligned}
Through the last two examples, we have discovered two types of signals:
Power signals: infinite energy, finite power. \qquad \cos(\omega t)
Energy signals: finite energy, zero power. \qquad e^{-t}
For periodic signals, we can also define the rms (root mean square):
\boxed{\;u_{\mathrm{RMS}} \;=\; \sqrt{P(u)}\;}
It turns out P comes from a different inner product, which is finite for power signals,
\langle u, y\rangle_{P} \;:=\; \frac{1}{T} \int_{0}^{T} u(t)\, \overline{y}(t)\,\mathrm{d}t.
We then have
\begin{aligned} P(u) &= \langle u, u\rangle_{P}, \\[4pt] u_{\mathrm{RMS}} &= \sqrt{\langle u, u\rangle_{P}} \;=\; \|u\|_{P}. \end{aligned}
Example: the RMS of mains power in Australia is 230 V. What is the peak voltage?
v(t) = V_p \cos(\omega t), \qquad \omega = 2\pi \cdot 50 = 100\pi \;\text{rad/s}, \qquad T = \tfrac{1}{50} = 20\ \text{ms}.
From the earlier example, P(v) = \dfrac{V_p^{2}}{2}, so
v_{\mathrm{RMS}} \;=\; \sqrt{P(v)} \;=\; \frac{V_p}{\sqrt{2}} \qquad\Longrightarrow\qquad V_p \;=\; 230\sqrt{2} \;=\; 325\ \text{V}.
Why do we use v_{\mathrm{RMS}} as the nominal voltage? Let’s calculate the power delivered to a nominal resistive load R, such as a kettle.
Instantaneous power is
p(t) \;=\; v(t)\, i(t) \;=\; \frac{v(t)^{2}}{R} \;=\; \frac{V_p^{2}}{R}\cos^{2}(\omega t).
Average over one period:
\begin{aligned} P_{\mathrm{avg}} &= \frac{1}{T} \int_{0}^{T} \frac{|v(t)|^{2}}{R}\,\mathrm{d}t \;=\; \frac{1}{R}\, P(v) \\[6pt] &= \frac{V_p^{2}}{2R} \\[6pt] &= \frac{v_{\mathrm{RMS}}^{2}}{R}. \end{aligned}
P(v) is a signal power, so its units are \mathrm{V}^{2} — not joules. Dividing by R turns it into a physical power in watts.
This says that v_{\mathrm{RMS}} is the equivalent DC voltage that delivers the same power across a nominal load.
For R = 23\,\Omega,
P_{\mathrm{avg}} = \frac{230^{2}}{23} = 2300\ \text{W}, \qquad i_{\mathrm{RMS}} = \frac{230}{23} = 10\ \text{A}.
The angle between two signals
We can use this inner product to measure the angle between two signals using \cos\theta = \dfrac{\langle u, y\rangle_{P}}{\|u\|_{P}\,\|y\|_{P}}
x_1(t) = 3\cos\omega t, \qquad x_2(t) = 4\cos\!\left(\omega t + \frac{\pi}{4}\right)
\langle x_1(t), x_2(t)\rangle_{P} \;=\; \frac{\omega}{2\pi} \int_{0}^{\frac{2\pi}{\omega}} 12\cos(\omega t)\cos\!\left(\omega t + \frac{\pi}{4}\right)\mathrm{d}t
Plug through to get \;6\cos\!\left(\dfrac{\pi}{4}\right).
\|x_1\|_{P} = \frac{3}{\sqrt{2}}, \qquad \|x_2\|_{P} = \frac{4}{\sqrt{2}} \qquad \text{using the previous example.}
So
\cos\theta \;=\; \frac{6\cos\!\left(\frac{\pi}{4}\right)}{6}, \qquad \theta = \frac{\pi}{4}.
The angle between two sinusoids is their phase shift.
Linear systems, gain and delay
Back to our starting point, that a conformal matrix scales & rotates vectors.
A linear system performs these same operations on signals.
For example, a static gain scales a signal:
If u = \cos(\omega t),
\begin{aligned} (ku)_{\mathrm{RMS}} &= \sqrt{P(ku)} \\[4pt] &= \frac{k}{\sqrt{2}} \qquad \text{from the previous example.} \end{aligned}
u_{\mathrm{RMS}} \;=\; \frac{1}{\sqrt{2}}.
The gain of the system is
\frac{(ku)_{\mathrm{RMS}}}{u_{\mathrm{RMS}}} \;=\; k.
What type of system rotates signals, that is, imparts phase lag?
The pure delay,
y(t) = u(t - \tau).
E.g. if u(t) = \cos(\omega t),
\begin{aligned} y(t) &= \cos\big(\omega(t - \tau)\big) \\[2pt] &= \cos(\omega t - \omega\tau). \end{aligned}
From the previous example,
\cos\theta \;=\; \cos(-\omega\tau) \;=\; \cos(\omega\tau)
\underline{\;\theta = \omega\tau.\;}
The pure delay gives a phase shift of \omega\tau, and gain \underline{1}.
Pure delay and static gain are two simple examples of linear, time-invariant (LTI) systems. We will see many more next week!
Linearity and time invariance
What do linear and time invariant mean?
Linear =
homogeneous (scale invariant): \;A(\alpha u) = \alpha A(u).
and additive: \;A(u_1 + u_2) = A(u_1) + A(u_2). \qquad Add inputs \to add outputs.
Together, we have superposition: \;A(\alpha u_1 + \beta u_2) = \alpha A(u_1) + \beta A(u_2).
Time invariant: behaviour doesn’t depend on starting time:
A(D_\tau u) = D_\tau A(u) \quad \text{for all } \tau. \qquad \text{\small shift input} \to \text{\small shift output.}
Quick quiz: which maps from u to y are linear?
Example: check the delay is linear:
\begin{aligned} D_\tau(\alpha u) &= \alpha\, u(t - \tau) = \alpha\, D_\tau(u) &&\Longrightarrow\; \text{homogeneous.} \\[6pt] D_\tau(u_1 + u_2) &= u_1(t - \tau) + u_2(t - \tau) = D_\tau(u_1) + D_\tau(u_2) &&\Longrightarrow\; \text{additive.} \end{aligned}
Is it time invariant?
D_\tau\big(D_\mu u\big) \;=\; D_\tau\big(u(t - \mu)\big) \;=\; u(t - \mu - \tau) \;=\; D_\mu\big(D_\tau(u)\big) \;\checkmark
Now: some useful signals
We have already seen the most used signal in this course: u(t) = e^{st}, s \in \mathbb{C}, and all its components: e^{-t}, e^{t}, \sin t, \cos t.
Now we’ll introduce three more signals which we will see many more times.
① Dirac delta
u_\varepsilon(t) is the triangle of base \varepsilon and height \tfrac{2}{\varepsilon}, so
\int_{-\infty}^{\infty} u_\varepsilon(t)\,\mathrm{d}t \;=\; \frac{1}{2}\,\varepsilon\,\frac{2}{\varepsilon} \;=\; \varepsilon\,\frac{1}{\varepsilon} \;=\; 1.
The Dirac delta, or impulse, \delta(t), may be defined as
\boxed{\;\delta(t) \;=\; \lim_{\varepsilon \to 0} u_\varepsilon(t)\;}
infinite height, area 1.
The Dirac delta has an extremely important property: the sifting property.
Let y(t) be an arbitrary continuous signal. Then
\boxed{\;\langle y, \delta\rangle \;=\; \int_{-\infty}^{\infty} y(t)\,\bar \delta(t)\,\mathrm{d}t \;=\; y(0)\;}
Sketch of proof:
If \varepsilon is small, y(t) \simeq y(0) on [0, \varepsilon], so
\begin{aligned} \int_{-\infty}^{\infty} \delta(t)\, y(t)\,\mathrm{d}t &= \lim_{\varepsilon \to 0} \int_{0}^{\varepsilon} u_\varepsilon(t)\, y(t)\,\mathrm{d}t \\[6pt] &\simeq y(0) \lim_{\varepsilon \to 0} \int_{0}^{\varepsilon} u_\varepsilon(t)\,\mathrm{d}t \\[6pt] &= y(0). \end{aligned}
(Can be made precise using a continuity argument).
What about \displaystyle \int_{-\infty}^{\infty} \delta(t - \tau)\, y(t)\,\mathrm{d}t ?
Let \sigma = t - \tau. \;\mathrm{d}t = \mathrm{d}\sigma.
Then we have
\int_{-\infty}^{\infty} \delta(\sigma)\, y(\sigma + \tau)\,\mathrm{d}\sigma \;=\; y(\tau).
\boxed{\;\int_{-\infty}^{\infty} \delta(t - \tau)\, y(t)\,\mathrm{d}t \;=\; y(\tau)\;} \qquad \text{\small the sifting property.}
NOTE: The above definition of the Dirac delta is not unique. It can be defined in many ways, for example using different limits, measures or distributions. The important thing (which really defines it) is the sifting property.
NOTE: A Dirac delta only really makes sense inside an integral, so it represents an area of 1, rather than a height.
The scaling property
What is \delta(\alpha t)?
\delta(\alpha t) \;=\; \lim_{\varepsilon \to 0} u_\varepsilon(\alpha t).
Let \sigma = \alpha t. Then \sigma = \varepsilon \;\Rightarrow\; t = \dfrac{\varepsilon}{\alpha}, so on the t axis the triangle has base \left|\dfrac{\varepsilon}{\alpha}\right| and height \dfrac{2}{\varepsilon}, giving
\text{area} \;=\; \frac{1}{2}\left|\frac{\varepsilon}{\alpha}\right|\frac{2}{\varepsilon} \;=\; \frac{1}{|\alpha|}.
The limit is therefore an impulse of area \dfrac{1}{|\alpha|} at the origin:
\boxed{\;\delta(\alpha t) \;=\; \frac{1}{|\alpha|}\,\delta(t)\;}
Example: radar
Radar systems emit a pulse train and measure its reflection to find the distance to objects. We can model this as a sequence of deltas.
Total path length: 2r. Round trip time: \dfrac{2r}{c} = t_d. So
r \;=\; \frac{c\,t_d}{2}.
Delay \Rightarrow distance. E.g. a 1\,\mu\text{s} delay gives
r = \tfrac{1}{2}\,(3\times10^{8})(10^{-6}) = 150\ \text{m}.
If there are N targets at ranges r_i with reflectivities a_i, the reflected signal is
y(t) \;=\; \sum_{i=1}^{N} a_i\, \delta\!\left(t - \frac{2 r_i}{c}\right).
Unambiguous range. Suppose we transmit radar pulses at a frequency f = \dfrac{1}{T}:
Suppose we measure an echo at time t_d since the most recent pulse. Which pulse is it an echo of? The distance could be
\frac{c\,t_d}{2}, \qquad \frac{c\,(t_d + T)}{2}, \qquad \frac{c\,(t_d + 2T)}{2}, \qquad \dots
The range is only known up to
R_u \;=\; \frac{cT}{2},
the unambiguous range.
Higher T: larger range, but slower updates.
How to deal with this? Transmit at two frequencies — the range has to be consistent with both measurements.
② The unit step
Another important function is the unit step or Heaviside function:
\begin{aligned} H(t) &:= \int_{-\infty}^{t} \delta(\varsigma)\,\mathrm{d}\varsigma \\[6pt] &= \begin{cases} 0, & t < 0 \\[2pt] 1, & t \geq 0. \end{cases} \end{aligned}
③ The pulse
Finally, the pulse function p_\tau(t):
p_\tau(t) := \begin{cases} 1, & -\dfrac{\tau}{2} \leq t \leq \dfrac{\tau}{2} \\[6pt] 0, & \text{otherwise.} \end{cases}