Update (21 Jan 2017) : Tutorial updated to comply with new API in liquid-dsp 1.3.0 .
In the previous error correction and phase-locked loops tutorials we have created only the basic building blocks for wireless communication. This tutorial puts them all together by introducing a very simple framing structure for sending and receiving data over a wireless link. In this context "framing" refers to the encapsulation of data into a modulated time series at complex baseband to be transmitted over a wireless link. Conversely, "packets" refer to packing raw message data bytes with forward error-correction and data validity check redundancy.
Problem Statement∞
For this tutorial we will be using the framegen64 and framesync64 objects in liquid . As you might have guessed framegen64 is the frame generator object on the transmit side of the link and framesync64 is the frame synchronizer on the receive side. Together these objects realize a a very simple frame which encapsulates a 8-byte header and 64-byte payload within a frame consisting of 600 symbols at complex baseband. Conveniently the frame generator interpolates these symbols with a matched filter to produce a 1440-sample frame at complex baseband, ready to be up-converted and transmitted over the air. This frame has a nominal spectral efficiency of 0.8 bits/second/Hz (512 bits from 64 payload bytes assembled in 640 symbols).
Note: For simplicity this computation of spectral efficiency neglects any excess bandwidth of the pulse-shaping filter.
This means that if you transmit with a symbol rate of 10kHz you should expect to see a throughput of 8kbps if all the frames are properly decoded. On the receiving side, raw samples at complex baseband are streamed to an instance of the frame synchronizer which picks out frames and invokes a user-defined callback function. The synchronizer corrects for gain, carrier, and sample timing offsets (channel impairments) in the complex baseband samples with a minimal amount of pre-processing required by the user. To help with synchronization, the frame includes a special preamble which can be seen in the figure below.
After up-conversion (mixing up to a carrier frequency) the frame is transmitted over the link where the receiver mixes the signal back down to complex baseband. The received signal will be attenuated and noisy and typically degrades with distance between the two radios. Also, because receiver's oscillators run independent of the transmitter's, this received signal will have other impairments such as carrier and timing offsets. In our program we will be operating at complex baseband and will add the channel impairments artificially.
The frame synchronizer's purpose is to correct for all of these impairments (within limitations, of course) and attempt to detect the frame and decode its data. The framing preamble assists the synchronizer by introducing special phasing sequences before any information-bearing symbols which aids in correcting for carrier and timing offsets. Without going into great detail, these sequences significantly increase the probability of frame detection and decoding while adding a minimal amount of overhead to the frame; a small price to pay for increased data reliability!
Setting up the Environment∞
As with the other tutorials I assume that you are using gcc to compile your programs and link to appropriate libraries. Create a new file framing.c and include the headers stdio.h , stdlib.h , math.h , complex.h , and liquid/liquid.h . Add the int main() definition so that your program looks like this:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#include <liquid/liquid.h>
int main() {
printf("done.\n");
return 0;
}
Compile and link the program using gcc :
$ gcc -Wall -o framing framing.c -lm -lc -lliquid
The flag " -Wall " tells the compiler to print all warnings (unused and uninitialized variables, etc.), " -o framing " specifies the name of the output program is " framing ", and " -lm -lc -lliquid " tells the linker to link the binary against the math, standard C, and liquid DSP libraries, respectively. Notice that the above command invokes both the compiler and the linker collectively.
If the compiler did not give any errors, the output executable framing is created which can be run as
$ ./framing
and should simply print " done. " to the screen. You are now ready to add functionality to your program.
Creating the Frame Generator∞
The particular framing structure we will be using accepts a 8-byte header and a 64-byte payload and assembles them into a frame consisting of 1440 samples. These sizes are fixed and cannot be adjusted for this framing structure.
.. footnote
Alternatively, the `flexframegen` and `flexframesync`
objects implement a dynamic framing structure which has many more
options than the `framegen64` and `framesync64` objects.
See [section-framing] for details.
The purpose of the header is to conveniently allow the user a separate control channel to be packaged with the payload. For example, if your application is to send a file using multiple frames, the header can include an identification number to indicate where in the file it should be written. Another application of the header is to include a destination node identifier for use in packet routing for ad hoc networks. Both the header and payload are assembled with a 16-bit cyclic redundancy check (CRC) to validate the integrity of the received data and encoded using the Hamming(12,8) code for error correction. (see [ref:section-fec] for more information on error detection and correction capabilities in liquid ). The encoded header and payload are modulated with QPSK and encapsulated with a BPSK preamble. Finally, the resulting symbols are interpolated using a square-root Nyquist matched filter at a rate of 2 samples per symbol. This entire process is handled internally so that as a user the only thing you will need to do is call one function.
The framegen64 object can be generated with the framegen64_create() method which does not accept any arguments. Historically the frame generator accepted parameters for the filter response; however for simplicity these values are now static. To begin, create a frame generator:
framegen64 fg = framegen64_create();
As with all structures in liquid you will need to invoke the corresponding destroy() method when you are finished with the object. Now allocate memory for the header and payload data arrays, remembering that they have lengths 8 and 64, respectively. Raw "message" data are stored as arrays of type unsigned char in liquid .
unsigned char header[8];
unsigned char payload[64];
Finally you will need to create a buffer for storing the frame samples. For this framing structure you will need to allocate a number of samples of type float complex , viz
float complex buf[LIQUID_FRAME64_LEN];
where LIQUID_FRAME64_LEN is the number of samples in the static frame (1440 samples, at the time of writing this). Initialize the header and payload arrays with whatever values you wish. All that is needed to generate a frame is to invoke the frame generator's execute() method:
framegen64_execute(fg, header, payload, buf);
That's it! This completely assembles the frame complete with interpolation and is ready for up-conversion and transmission. To generate another frame simply write whatever data you wish to the header and payload buffers, and invoke the framegen64_execute() method again as done above. If you wish, print the first few samples of the generated frame to the screen (you will need to separate the real and imaginary components of each sample).
for (i=0; i<10; i++)
printf("%3u : %12.8f + j*%12.8f\n", i, crealf(buf[i]), cimagf(buf[i]));
Your program should now look similar to this:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#include <liquid/liquid.h>
int main() {
// allocate memory for arrays
unsigned char header[8]; // data header
unsigned char payload[64]; // data payload
unsigned int buf_len = LIQUID_FRAME64_LEN; // length of frame
float complex buf[buf_len]; // sample buffer
// create frame generator
framegen64 fg = framegen64_create();
framegen64_print(fg);
// initialize header, payload
unsigned int i;
for (i=0; i<8; i++)
header[i] = i;
for (i=0; i<64; i++)
payload[i] = rand() & 0xff;
// execute generator and assemble the frame
framegen64_execute(fg, header, payload, buf);
// print a few of the generated frame samples to the screen
for (i=0; i<10; i++)
printf("%3u : %12.8f + j*%12.8f\n", i, crealf(buf[i]), cimagf(buf[i]));
// destroy objects
framegen64_destroy(fg);
printf("done.\n");
return 0;
}
Compile the program as before, creating the executable " framing ." Running the program should produce an output similar to this:
framegen64 [m=7, beta=0.30]:
preamble/etc.
* ramp/up symbols : 7
* p/n symbols : 64
* ramp\down symbols : 7
* zero padding : 12
payload
* payload len, uncoded : 64 bytes
* payload len, coded : 150 bytes
* modulation scheme : qpsk
* payload symbols : 600
* pilot symbols : 30
summary
* total symbols : 720
* spectral efficiency : 0.8000 b/s/Hz
0 : -0.00074591 + j* -0.00074591
1 : 0.00077571 + j* 0.00077571
2 : 0.00262754 + j* 0.00411935
3 : -0.00289224 + j* -0.00444366
4 : -0.00565739 + j* -0.01091247
5 : 0.00825994 + j* 0.01404442
6 : 0.00978434 + j* 0.01960731
7 : -0.01882642 + j* -0.03379488
8 : -0.01267161 + j* -0.02698521
9 : 0.03836738 + j* 0.07023574
done.
You might notice that the imaginary component of the samples in the beginning of the frame are zero. This is because the preamble of the frame is BPSK which has no imaginary component at complex baseband.
Creating the Frame Synchronizer∞
As stated earlier the frame synchronizer's purpose is to detect the presence of a frame, correct for the channel impairments, decode the data, and pass it back to the user. In our program we will simply pass to the frame synchronizer the samples we generated in the previous section with the frame generator. Furthermore, the hardware interface might pass the baseband samples to the synchronizer in blocks much smaller than the length of a frame (512 samples, for instance) or even blocks much larger than the length of a frame (4096 samples, for instance). How does the synchronizer relay the decoded data back to the program without missing any frames? The answer is through the use of a callback function.
What is a callback function? Put quite simply, a callback function is a function pointer (a designated address in memory) that is invoked during a certain event. For this example the callback function given to the framesync64 synchronizer object when the object is created and is invoked whenever the synchronizer finds a frame. This happens irrespective of the size of the blocks passed to the synchronizer. If you pass it a block of data samples containing four frames|several thousand samples|then the callback will be invoked four times (assuming that channel impairments haven't corrupted the frame beyond the point of recovery). You can even pass the synchronizer one sample at a time if you wish.
The framesync64 object can be generated with the framesync64_create() method which accepts two pointers as arguments:
framesync64 framesync64_create(framesync64_callback _callback,
void * _userdata);
- _callback is a pointer to your callback function which will be invoked each time a frame is found and decoded.
- _userdata is a void pointer that is passed to the callback function each time it is invoked. This allows you to easily pass data from the callback function. Set to NULL if you don't wish to use this.
The framesync64 object has a callback function which has seven arguments and looks like this:
int framesync64_callback(unsigned char * _header,
int _header_valid,
unsigned char * _payload,
unsigned int _payload_len,
int _payload_valid,
framesyncstats_s _stats,
void * _userdata);
The callback is typically defined to be static and is passed to the instance of framesync64 object when it is created.
- _header is a pointer to the 8 bytes of decoded header data. This pointer is not static and cannot be used after returning from the callback function. This means that it needs to be copied locally for you to retain the data.
- _header_valid is simply a flag to indicate if the header passed its cyclic redundancy check (" 0 " means invalid, " 1 " means valid). If the check fails then the header data most likely has been corrupted beyond the point that the internal error-correction code can recover; proceed with caution!
- _payload is a pointer to the 64 bytes of decoded payload data. Like the header, this pointer is not static and cannot be used after returning from the callback function. Again, this means that it needs to be copied locally for you to retain the data.
- _payload_len is an integer specifying the length of the payload. For the frame64 structure this will always be 64 ; however the same callback is used for other framing structures and can be dynamic.
- _payload_valid is simply a flag to indicate if the payload passed its cyclic redundancy check (" 0 " means invalid, " 1 " means valid). As with the header, if this flag is zero then the payload most likely has errors in it. Some applications are error tolerant and so it is possible that the payload data are still useful. Typically, though, the payload should be discarded and a re-transmission request should be issued.
- _stats is a synchronizer statistics construct that indicates some useful PHY information to the user. We will ignore this information in our program, but it can be quite useful for certain applications. For more information on the framesyncstats_s structure, see[ref:section-framing-framesyncstats_s] .
- _userdata Remember that void pointer you passed to the create() method? That pointer is passed to the callback and can represent just about anything. Typically it points to another structure and is the method by which the decoded header and payload data are returned to the program outside of the callback.
This can seem a bit overwhelming at first, but relax! The next version of our program will only add about 20 lines of code.
Putting it All Together∞
First create your callback function at the beginning of the file, just before the int main() definition; you may give it whatever name you like (e.g. mycallback() ). For now ignore all the function inputs and just print a message to the screen that indicates that the callback has been invoked, and return the integer zero ( 0 ). This return value for the callback function should always be zero and is reserved for future development. Within your main() definition, create an instance of framesync64 using the framesync64_create() method, passing it a NULL for the first and third arguments (the properties and userdata constructs) and the name of your callback function as the second argument. Print the newly created synchronizer object to the screen if you like:
framesync64 fs = framesync64_create(mycallback,NULL);
framesync64_print(fs);
After your line that generates the frame samples (" framegen64_execute(fg, header, payload, buf); ") invoke the synchronizer's execute() method, passing to it the frame synchronizer object you just created ( fs ), the pointer to the array of frame symbols ( buf ), and the length of the array ( buf_len ):
framesync64_execute(fs, buf, buf_len);
Finally, destroy the frame synchronizer object along with the frame generator at the end of the file. That's it! Your program should look something like this:
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#include <liquid/liquid.h>
// user-defined static callback function
static int mycallback(unsigned char * _header,
int _header_valid,
unsigned char * _payload,
unsigned int _payload_len,
int _payload_valid,
framesyncstats_s _stats,
void * _userdata)
{
printf("***** callback invoked!\n");
return 0;
}
int main() {
// allocate memory for arrays
unsigned char header[8]; // data header
unsigned char payload[64]; // data payload
unsigned int buf_len = LIQUID_FRAME64_LEN; // length of frame
float complex buf[buf_len]; // sample buffer
// create frame generator
framegen64 fg = framegen64_create();
framegen64_print(fg);
// create frame synchronizer using default properties
framesync64 fs = framesync64_create(mycallback, NULL);
framesync64_print(fs);
// initialize header, payload
unsigned int i;
for (i=0; i<8; i++)
header[i] = i;
for (i=0; i<64; i++)
payload[i] = rand() & 0xff;
// execute generator and assemble the frame
framegen64_execute(fg, header, payload, buf);
// execute synchronizer and receive the entire frame at once
framesync64_execute(fs, buf, buf_len);
// destroy objects
framegen64_destroy(fg);
framesync64_destroy(fs);
printf("done.\n");
return 0;
}
Compile and run your program as before and verify that your callback function was indeed invoked. Your output should look something like this:
framegen64 [m=7, beta=0.30]:
preamble/etc.
* ramp/up symbols : 7
* p/n symbols : 64
* ramp\down symbols : 7
* zero padding : 12
payload
* payload len, uncoded : 64 bytes
* payload len, coded : 150 bytes
* modulation scheme : qpsk
* payload symbols : 600
* pilot symbols : 30
summary
* total symbols : 720
* spectral efficiency : 0.8000 b/s/Hz
framesync64:
frames detected : 0
headers valid : 0 ( 0.0000 %)
payloads valid : 0 ( 0.0000 %)
bytes received : 0
***** callback invoked!
done.
As you can see, the framesync64 object has a long list of modifiable properties pertaining to synchronization; the default values provide a good initial set for a wide range of channel conditions. Duplicate the line of your code that executes the frame synchronizer. Recompile and run your code again. You should see the " ***** callback invoked! " printed twice.
Your program has only demonstrated the basic functionality of the frame generator and synchronizer under ideal conditions: no noise, carrier offsets, etc. The next section will add some channel impairments to stress the synchronizer's ability to decode the frame.
Final Program∞
In this last section we will add some channel impairments to the frame after it is generated and before it is received. This will simulate non-ideal channel conditions. Specifically we will introduce carrier frequency and phase offsets, channel attenuation, and noise. We will also add a frame counter and pass it through the userdata construct in the frame synchronizer's create() method to be passed to the callback function when a frame is found. Finally, the program will split the frame into pieces to emulate non-contiguous data partitioning at the receiver.
To begin, add the following parameters to the beginning of your main() definition with the other options:
unsigned int frame_counter = 0; // userdata passed to callback
float phase_offset = 0.3f; // carrier phase offset
float frequency_offset= 0.02f; // carrier frequency offset
float SNRdB = 10.0f; // signal-to-noise ratio [dB]
float noise_floor = -40.0f; // noise floor [dB]
The frame_counter variable is simply a number we will pass to the callback function to demonstrate the functionality of the userdata construct. Make sure to initialize frame_counter to zero.
If you completed the tutorial on phase-locked loop design you might recognize the phase_offset and frequency_offset variables; these will be used in the same way to represent a carrier mismatch between the transmitter and receiver.
The channel gain and noise parameters are a bit trickier and are set up by the next two lines. Typically the noise power is a fixed value in a receiver; what changes is the received power based on the transmitter's power and the gain of the channel; however because theory dictates that the performance of a link is governed by the ratio of signal power to noise power, SNR is a more useful than defining signal amplitude and noise variance independently. The SNRdB and noise_floor parameters fully describe the channel in this regard. The noise standard deviation and channel gain may be derived from these values as follows:
float nstd = powf(10.0f, noise_floor/20.0f);
float gamma = powf(10.0f, (SNRdB+noise_floor)/20.0f);
Add to your program (after the framegen64_execute() line) a loop that modifies each sample of the generated frame by introducing the channel impairments.
$$ \textup{buf}[i] \leftarrow \gamma \textup{buf}[i] e^{j(\theta + i\omega)} + \sigma n $$where\(\textup{buf}[i]\) is the frame sample at index \(i\) ( buf[i] ),\(\gamma\) is the channel gain defined above ( gamma ),\(\theta\) is the carrier phase offset ( phase_offset ),\(\omega\) is the carrier frequency offset ( frequency_offset ),\(\sigma\) is the noise standard deviation defined above ( nstd ), and\(n\) is a circular Gauss random variable. liquid provides the randnf() methods to generate real random numbers with a Gauss distribution; a circular Gauss random variable can be generated from two regular Gauss random variables \(n_i\) and \(n_q\) as \(n = (n_i + jn_q)/\sqrt{2}\) .
buf[i] *= gamma;
buf[i] *= cexpf(_Complex_I*(phase_offset + i*frequency_offset));
buf[i] += nstd * (randnf() + _Complex_I*randnf())*0.7071;
Check the program listed below if you need help.
Now modify the program to incorporate the frame counter. First modify the piece of code where the frame synchronizer is created: replace the last argument (initially set to NULL ) with the address of our frame_counter variable. For posterity's sake, this address will need to be type cast to void* (a void pointer) to prevent the compiler from complaining. In your callback function you will reverse this process: create a new variable of type unsigned int* (a pointer to an unsigned integer) and assign it the _userdata argument type cast to unsigned int* . Now de-reference this variable and increment its value. Finally print its value near the end of the main() definition to ensure it is being properly incremented. Again, check the program below for assistance.
The last task we will do is push one sample at a time to the frame synchronizer rather than the entire frame block to emulate non-contiguous sample streaming. To do this, simply remove the line that calls framesync64_execute() on the entire frame and replace it with a loop that calls the same function but with one sample at a time.
The final program is listed below.
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <complex.h>
#include <liquid/liquid.h>
// user-defined static callback function
static int mycallback(unsigned char * _header,
int _header_valid,
unsigned char * _payload,
unsigned int _payload_len,
int _payload_valid,
framesyncstats_s _stats,
void * _userdata)
{
printf("***** callback invoked!\n");
printf(" header (%s)\n", _header_valid ? "valid" : "INVALID");
printf(" payload (%s)\n", _payload_valid ? "valid" : "INVALID");
framesyncstats_print(&_stats);
// type-cast, de-reference, and increment frame counter
unsigned int * counter = (unsigned int *) _userdata;
(*counter)++;
return 0;
}
int main() {
// options
unsigned int frame_counter = 0; // userdata passed to callback
float phase_offset = 0.3f; // carrier phase offset
float frequency_offset= 0.02f; // carrier frequency offset
float SNRdB = 10.0f; // signal-to-noise ratio [dB]
float noise_floor = -40.0f; // noise floor [dB]
// allocate memory for arrays
unsigned char header[8]; // data header
unsigned char payload[64]; // data payload
unsigned int buf_len = LIQUID_FRAME64_LEN; // length of frame
float complex buf[buf_len]; // sample buffer
// create frame generator
framegen64 fg = framegen64_create();
framegen64_print(fg);
// create frame synchronizer using default properties
framesync64 fs = framesync64_create(mycallback, (void*)&frame_counter);
framesync64_print(fs);
// initialize header, payload
unsigned int i;
for (i=0; i<8; i++)
header[i] = i;
for (i=0; i<64; i++)
payload[i] = rand() & 0xff;
// execute generator and assemble the frame, storing samples in buffer
framegen64_execute(fg, header, payload, buf);
// add channel impairments (attenuation, carrier offset, noise)
float nstd = powf(10.0f, noise_floor/20.0f); // noise std. dev.
float gamma = powf(10.0f, (SNRdB+noise_floor)/20.0f);// channel gain
for (i=0; i<buf_len; i++) {
buf[i] *= gamma;
buf[i] *= cexpf(_Complex_I*(phase_offset + i*frequency_offset));
buf[i] += nstd * (randnf() + _Complex_I*randnf())*M_SQRT1_2;
}
// EXECUTE synchronizer and receive the frame in one block
framesync64_execute(fs, buf, buf_len);
// destroy objects
framegen64_destroy(fg);
framesync64_destroy(fs);
printf("received %u frames\n", frame_counter);
printf("done.\n");
return 0;
}
Compile and run the program as before. The output of your program should look something like this:
framegen64 [m=7, beta=0.30]:
preamble/etc.
* ramp/up symbols : 7
* p/n symbols : 64
* ramp\down symbols : 7
* zero padding : 12
payload
* payload len, uncoded : 64 bytes
* payload len, coded : 150 bytes
* modulation scheme : qpsk
* payload symbols : 600
* pilot symbols : 30
summary
* total symbols : 720
* spectral efficiency : 0.8000 b/s/Hz
framesync64:
frames detected : 0
headers valid : 0 ( 0.0000 %)
payloads valid : 0 ( 0.0000 %)
bytes received : 0
***** callback invoked!
header (valid)
payload (valid)
EVM : -12.49366951 dB
rssi : -30.30548477 dB
carrier offset : 0.02010066 Fs
num symbols : 600
mod scheme : qpsk (2 bits/symbol)
validity check : crc24
fec (inner) : none
fec (outer) : g2412
received 1 frames
done.
Play around with the initial options, particularly those pertaining to the channel impairments. Under what circumstances does the synchronizer miss the frame? For example, what is the minimum SNR level that is required to reliably receive a frame? the maximum carrier frequency offset? The "random" noise generated by the program will be seeded to the same value every time the program is run. A new seed can be initialized on the system's time (e.g. time of day) to help generate new instances of random numbers each time the program is run. To do so, include the <time.h> header to the top of your file and add the following line to the beginning of your program's main() definition:
srand(time(NULL));
This will ensure a unique simulation is run each time the program is executed. For a more detailed program, see examples/framesync64_example.c in the main liquid directory. /doc/framing/ describes liquid 's framing module in detail.
While the framing structure described in this section provides a simple interface for transmitting and receiving data over a channel, its functionality is limited and isn't particularly spectrally efficient. liquid provides a more robust framing structure which allows the use of any linear modulation scheme, two layers of forward error-correction coding, and a variable preamble and payload length. These properties can be reconfigured for each frame to allow fast adaptation to quickly varying channel conditions. Furthermore, the frame synchronizer on the receiver automatically reconfigures itself for each frame it detects to allow as simple an interface possible. The frame generator and synchronizer objects are denoted flexframegen and flexframesync , respectively, and are described here . A detailed example program examples/flexframesync_example.c is available in the main liquid directory.