Listen to data broadcasts with Software Defined Radio

Introduction

Software Defined Radio (SDR) replaces traditional hardware components of wireless communication technology with software algorithms. Affordable SDR dongles enables users to explore and manipulate a broad spectrum of radio frequencies using just general purpose computers. Applications of SDR are radio monitoring, signal analysis, and experimentation.

In Europe the frequencies at 434MHz and 868MHz serve as a communication spectrum for a diverse array of devices, including sensors, remote controls, and automation systems for applications ranging from home automation to industrial monitoring. Radio signals at other frequencies with music or voice are not subject of this article.

Radio signal data broadcasts involve modulation through three primary methods: amplitude modulation, frequency modulation, and pulse timing encoding. In the 868MHz band, frequency modulation like Frequency Shift Keying (FSK) is the prevalent method, wherein digital data is encoded by manipulating the carrier frequency. Pulse time encoding which is like morse code with shorter and longer pulses, is also observed, and more often in the 434MHz band. Reason may be that both this modulation technique and the 434MHz frequency band have a longer history.

As I did not succeed to apply available software to decode Frequency Shift Keying (FSK) signals reliable with rtl_433 or GNU Radio, I took the journey to discover the best method to decode FSK myself and code the solution. This article documents what I learnt.

General signal observations

When considering radio signals, it is essential to shift our perspective from seconds to the scales of milliseconds (ms) and microseconds (µs). The duration of a pulse, being the duration of the high or low state, can be short (25 µs) or long (600 µs), depending on the length of the message resulting in a short signal (15ms) or long signal (75ms).

The Frequency Shift Keying (FSK) signals observed apply Gaussian FSK with smooth transitions and not sudden Binary FSK signals. The decoding method for both variants is the same. I did not observe modulation with more than two frequencies or the compact Minimum-shift keying method.

For decoders it is hard to decode signals with long pulses on the high or low frequency representing a series of ‘0’ or ‘1’ bits. Manchester encoding translates a bit to a high-low or low-high pulse, doubling the pulses but removing the uncertainty of how many bits are represented by a very long pulse and ensuring the decoder whether the signal is still at the high or low frequency. This last remark is related to the rtl_433 decoder: with each data point the code adjusts the center frequency a little. With longer pulses or a higher sample rate like 2.048k compared to the 250K rtl_433 used as reference, it stops after 1024 data points on the same frequency.

A common convention is to use a preamble or a synchronization sequence at the beginning of the transmission which helps receivers synchronize with the incoming signal.

Locate signals in time

The method is to calculate the power in the signal for smaller time periods, like 0,5ms, for multiple frequencies, and presume a signal if the power at one frequency is minimal 10 times greater than the median power, representing the average noise level. The data can thus be divided in multiple time periods with or without a signal, and each signal can be processed seperately.

Typically 250 to 3.072 data points per ms are available and through advanced mathematics, the power at that many frequencies can be calculated. Using 2.048.000 samples per second and taking 0,5ms of data, the method will calculate the power at 1.024 different frequency bins.

Rules are in place to include some extra data, like 1ms, in front and after the signal to ensure catching the complete signal needed to decode the data. And it is preferred to neglect very short signals (2 or 3ms) or signal breaks (1 ms). This approach rescues bad recordings, but at the end bad recordings often not result in successful data decodings, so not bother too much.

Locate the frequencies used

To identify the two frequencies employed in the FSK-signal, we utilize the Welch method, calculating the average power per frequency across the entire signal length. The parameters are tuned to create 1kHz frequency bins.

Disparities in power between high and low frequency may lead to neighboring frequencies of the highest peak having more power than the second frequency we are searching for. To address this, we consider only local peaks. We select the frequency bins with both neighboring bins exhibiting lower power as candidates and select the top 2.

Choose the granularity in frequency vs time

Determining the appropriate granularity involves a balance between short pulses corresponding to small data segments (e.g. 16 data points) and longer pulses related to larger segments (e.g. 256 data points). Short pulses can lead to coarse frequency bins, where both the high and low frequency of the FSK share the same bin, resulting in information loss. Conversely, longer data segments solve the frequency bin problem but may mix multiple pulses, leading to information loss as well.

This trade-off is also relevant for the sending device, so very short pulses with very narrow bandwidth is not to be expected. The granuality can be chosen depending on the bandwidth measured and we may expect longer pulses if the bandwidth is narrow.

But that is not enough. Second measure is to calculate with 4 times more data points than each step in time taken, also formulated as the window-size being 4 times the hop-size. The greater window-size creates smaller frequency bins and the smaller hop-size creates shorter pulse measurements.

The goal is to get at least a multiple of three or four measurements for each pulse to distinguish between noise and data and secondly calculate the length of the pulse.

The difference in power between the two frequencies is used to decide whether the “hop” is LOW or HIGH. As the power of the frequency bins may differ as described above, the threshold is not set at zero, but set at the average difference.

Two examples

The bandwidth, the distance between low and high frequency, of a signal is 40kHz and the pulses are multiples of 26µs. With 2.048.000 samples per second, 64 data points leads to 32kHz wide frequency bins, and lowering the hopsize to 16 data points results to 7,8µs long measurements (compared to 31,3µs), just enough to determine the pulses.
The bandwidth of another signal is 9kHz and the pulses are multiples of 300µs. With 2.048.000 samples per second, 256 data points leads to 8kHz wide frequency bins, and lowering the hopsize to 64 data points, this results to 31,2µs long measurements.
In both cases the low and high frequency are in different frequency bins and enough data is available to define the pulse and the pulse length.

Measure the exact timings

To convert LOW and HIGH sequences into one or more bits, we estimate the sequence length per bit. Our assumption is that the most frequently occurring sequence length corresponds to 1 bit. However, relying solely on the most common sequence length for determining the size of 1 bit lacks precision, especially for dealing with sequences of longer lengths. To address this, we incorporate both the 1 lower and 1 higher sequence lengths, allowing us to calculate a weighted sequence length per bit. This approach proves highly effective in determining the right number of bits, particularly when dealing with longer sequence lengths.

Decode the high/low phases to bits

A preliminary step may precede the interpretation of bits. Frequently, Manchester encoding is employed, as previously mentioned. Anticipated are sequences of ’10’s and ’01’s, initiating either at an odd or even position among the retrieved bits. Both ’11’ and ’00’ configurations are treated as errors, and a Manchester-decoded signal devoid of such errors is considered interpretable.

Decode the bits

This article focuses on signal decoding, with an emphasis on understanding the significance of transmitted bits. The interpretation of these bits varies depending on the specific device in question. To effectively engage in this process, it is crucial to first identify the device by physically exploring the surroundings and observing the signals being emitted. Additionally, a key initial step involves accumulating a substantial number of recordings for in-depth analysis of signal frequency and the patterns in bit behavior associated with the device’s status. It is noteworthy that the initial bytes in the signal may serve as an identifier for the particular device under scrutiny.