It is known that a Matched Filter is optimal under additive white Gaussian noise (AWGN) conditions. A matched filter is one whose impulse response is the time inverted version of waveform of the actual signal that is to be decoded. In the case of 45.45 baud RTTY, the match filter's impulse response is therefore a rectangular function that is 22 milliseconds long.
The bandwidth of a matched filter is very wide. The envelope of the transfer function in the frequency domain falls only at the rate 1/f, which is not surprising since the envelope of the keying sidebands of a single RTTY bit falls as 1/f. To counter interference, something narrower is desired.
Frank Gaudé K6IBE suggested in his September 1963 RTTY bulletin article "Two-Tone, Shifts And Filters," that a narrow linear phase filter with a bandwidth of 60 Hz tends to work well.
It is however left to Victor Poor K3NIO to present the real definitive requirements (the Nyquist Criterion) for any filter to perform optimally for filtering RTTY data so that individual data bits do not interfere with one another (so called "inter-symbol interference" or ISI).
As touched on earlier, Mr. Poor gave two necessary conditions that an optimal data filter must meet. The two conditions govern location of the "sample" and "slicing" points of the filtered data waveform, as shown in the figure below:
For the filter to work
optimally, the points where a waveform are sampled need to
all have an identical constant amplitude, regardless of
whether the data stream consists of alternating bits (such
as a Baudot RYRY sequence) or if there are two or more
consecutive Mark or Space bits.
The second necessary condition is that the output waveform from the filter has to pass though zero at locations which are halfway in between the sample points.
Mr. Poor went on to show exemplary filters whose eye patterns satisfy the condition for the sampling point.
Notice that all waveforms in
the above figure pass thorough the same sample points (the
six small open circles that are just visible in the
diagram). However, the waveforms differ in between the
sampling points, therefore they cannot all meet the second
condition which he gave.
The following is his example of waveforms from filters that meet the slicing point condition (the two small circles), but again, naturally they cannot all meet the sample point condition:
Finally, Poor shows waveforms
from three exemplary filters that satisfy both of his
Mr. Poor calls the above "eye
pictures." "Eye patterns" is a more common name for
these diagrams today.
As emphatically stated by Mr. Poor, the proper filter has nothing to do with being "wide enough to let the third (or fifth or umpteenth) harmonic pass." For any design bandwidth, the optimal filter has to meet his two conditions.
The matched filter is optimal when the noise is Gaussian and has constant power at all frequencies. When noise power is not constant, or if there is an interfering signal nearby, a filter with narrower passband is more optimal, but the filter still has to meet the two conditions laid out by Mr. Poor.
Restating Victor Poor's main point in modern filter nomenclature, the two conditions are satisfied if
(1) to preserve the mid-bit sample points, the frequency response shape must be a brick wall filter plus a shape with odd symmetry (and an additional even symmetry in the imaginary component) about the frequency that is equal to one-half the bit rate,
(2) to preserve the slicing points, the frequency response must be cosine function from DC to the frequency that is equal to one half the bit rate, plus a shape with even symmetry (and an additional shape with odd symmetry in the imaginary component), about the frequency that is equal to one-half the bit rate.
In Nyquist's 1928 paper the two conditions above are presented as Figure 2 (condition 1) and Figure 3 (condition 2):
Victor Poor had only mentioned
the real parts of the filter (the top figures in Figure 2
and bottom figures in Figure 3 in Nyquist's paper) because
he was working with inductors and capacitors on a real
signal. Today, with software modems, we can use in-phase
and quadrature (I and Q) signals and we have greater
liberty to exploit Harry Nyquist's original model. Of the
imaginary components, Nyquist stated, "Each one of [the
imaginary shape factor] may of course be combined with any
one of [the real shape factor]."
There are an infinite number of filters that meet condition (1) but not condition (2), and similarly, there are also an infinite number of filters that meet condition (2) but not condition (1). We need a filter that meets both conditions.
For a 45.45 baud RTTY signal, the filter that has the narrowest bandwidth and still meets the above two conditions is the Raised-cosine Filter whose cutoff is 22.7 Hz. A raised-cosine filter is a Nyquist filter, i.e., it has a response of zero at sampling points other than its own sampling point, thus satisfying Victor Poor's sample point criterion.
A general raised-cosine filter is actually not a single filter, but a family of filters with a roll-off parameter β which takes a value between 0 and 1.
The raised-cosine filter with β of 0 has the sharpest roll-off and is the same as a rectangular brick-wall transfer function that passes no signal beyond 22.7 Hz on either side of the center frequency. It will be virtually impossible to properly tune a signal through such a transfer function, and a pulse will also ring for an infinite time.
The raised-cosine filter with β of 1 has the widest roll-off and its transfer function is a cosine function between -π and +π that is raised by a DC term that is equal to the amplitude of the cosine (the properties which give the filter its name). In the case of 45.45 baud RTTY, the β = 1 raised-cosine filter has a non-zero response that extends up to 45.45 Hz on either side of the center frequency. It also has the shortest ringing in the time domain of all the raised-cosine family and appears to be the "raised-cosine" that Victor Poor chose to use.
Notice that the optimal raised-cosine filter (irrespective of the β) will pass only the fundamental of the keying sideband.
When there is no nearby interference, widening the filter bandwidth by using other forms of the Nyquist filter that pass more keying sidebands can improve the signal to noise ratio. This is because an RTTY signal is itself quite wide. In the extreme case, when there is no bandwidth limitation, the filter that has the best SNR is a Matched Filter (which by the nature of its definition, is a Nyqust filter).
An AWGN matched filter is one which connects the sample points in the eye pattern with straight lines. (i.e., the eye pattern looks like triangular waves). Victor Poor understood this since he says this about a matched filter:
"Maximum signal-to-noise ratio in the case of RTTY does not come with minimum bandwidth. Maximum signal-to-noise ratio comes with what is called a "matched filter."
Mr. Poor goes on to discuss good approximations to the optimal Raised-cosine filter. For practical reasons, he suggests using a third order Butterworth approximation. With today's software modems, it is easy to get a much better approximation to the Raised-cosine function than a third order Butterworth function.
Single Tuned Filters
In their "Current RTTY Receiving Techniques" article in the December 1964 issue of the RTTY bulletin, Irv Hoff and Keith Petersen published the following plot made by Tom Lamb K8ERV, showing filters that are constructed with 88 mH toroids at different Q and comparing them to an ideal Raised Cosine transfer function and Victor Poor's 3rd order Butterworth approximation:
Notice that with a Q of about
50, a single tuned circuit provides a good match to the
raised cosine, down to about -10 dB. However, it flares out
faster than both the ideal filter and Victor Poor's 3rd
order Butterworth, thus it will pass through more noise and
be more susceptible to interference. With a Q of 25, a
single tuned circuit will let through too much noise and
interference, and with a Q of 100, a single tuned circuit
will be too narrow to properly pass single bit transitions
(e.g., an RYRY pattern) from a weak signal.
When using a modem that already includes an optimal filter, you should not use a filter in the receiver that is too narrow since doing so will cause the combined receiver filter and modem filter to no longer meet the Nyquist conditions that are laid down by Victor Poor's article.
With a software demodulator that includes a matched filter or a raised cosine filter that matches the baud rate of the RTTY signal, the receiver's filter should be only as narrow as needed to keep interference from clipping the sound card.
Please remember this each time you are tempted to use a narrow crystal filter. If you are still not convinced, read Victor Poor's article.