Re: LF: Coherent BPSK on LF using EbNaut

[Markus Vester <markusvester@aol.com>]

Thanks very much to Domenico IZ7SLZ for informing the group about our LF EbNaut-PSK activities! I am glad that this has raised some interest, and I'm sure that trying Paul's software is well worth the effort for anyone wishing to communicate with really weak signals, very near to the ultimate Shannon-Hartley limit.

 

In this mail, I will try to provide some information regarding the receive procedure. As mentioned before, we have used SpecLab as a receiving frontend to generate an IQ wav file, which can be read by EbNaut-RX. Wolf has provided a dedicated riggered audio recording function, which is able to save decimated and precisely timestamped IQ data, starting at specified GPS-controlled times.

 

As I am already continuously running a SpecLab instance for the opds decoder, I decided to go down a slightly different route, by reusing the exported ascii spectrum-data files for EbNaut as well. One advantage of this procedure is that the LO used in the downconversion for the spectrogram already includes samplerate tracking, which avoids the need for CPU-intensive samplerate interpolation. If GPS-1pps is used for reference, the absolute LO phase will even be recovered after a soundcard dropout. Timestamping is derived from NTP, which is slightly less accurate but perfectly adequate for the symbol durations down to 0.2 s that were  used in our tests.

 

So all one has to do is to select a MMDDhhmm.txt file from the spectrum\data\ directory which contains the received sequence, and convert it back to time domain using an inverse FFT. In an email to Domenico (pasted beneath), I briefly described the file conversion and the frequency-time alignment process used with EbNaut. An updated version of the utilities can be found at
http://df6nm.bplaced.net/VLF/fec_tests/df6nm_ebnaut_utilities.zip (184 kB).

The SpecLab config-file _opds32_150924.usr for opds has been slightly updated, hopefully making life a bit it easier for first-time opds and EbNaut users. The analyzed audio frequency band has been centered on 1500 Hz. You will still have to calibrate your samplerate and receiver frequency. Note that for opds and EbNaut postprocessing the number of exported FFT bins (262144) always needs to be half the FFT length (524288).

 

The inverse-FFT utility is ebnaut_ifft3.exe. Dragging and dropping a .txt data file over it's icon will let it produce an equally named .wav file which can then be read by ebnaut-rx software. Note that for correct time and frequency information in the wav header, the ifft configuration file sr.txt needs to contain your accurate soundcard samplerate, FFT decimation factor and length, and RX frequency.

 

Shortly after starting ebnaut-rx and before it enters the decoding process, it writes the symbol amplitudes and phases to a logfile called rawsyms.txt. This can be used to assist the determination of the correct frequency and time offset. Clicking on the show_rawsyms3.exe utility will output a window with three different diagrams and two lines of text information.

 

The white dots at the top are a time domain representation of the symbol phases. With a strong signal, these should ideally be gathered around two horizontal lines, 180° apart. For the example screenshot, the frequency offset had been made 1 mHz too negative, resulting in gradually increasing symbol phases.

 

The red spectrum is a Fourier transform of the symbol amplitudes. The Nyquist frequency range is identical to the symbol rate (ie +- 0.5 Hz for 1 second symbols), with a zoomed section on the right. The spectrom will show a peak if a test carrier has been sent, and the freq offset in EbNaut can then be adjusted iteratively to bring it to the center. However no red peak will be appear from a PSK data transmission.  

 

The blue spectrum is calculated from the squared complex amplitudes. As taking the square eliminates the signal polarity, this spectrum will also show a peak from a PSK modulated sequence. Unfortunately the squared spectrum fails to produce a peak for very weak signals where the SNR of the individual symbols becomes too low. If he peak is present, the frequency error (e.g. offset needs to be increased by 0.97 mHz) can be read in the text beneath.

 

Unfortunately there is no direct indication to guide you towards the correct bit timing. One way is to iteratively adjust the fractional time offset for least scatter of dots or highest blue peak. In the bottom line, an estimate of the time error magnitude (nominally 0 to 0.5) is given, which should be small if the correct symbol timing has been found. This is calculated by a normalized autocorrelation with a one-symbol shift, exploiting the property of the sequence that the likelyhood of a symbol transition should always be 50%, independent of a prior transition. Again, this method will not work if the SNR is low.
 
For testing purposes, you can try the data in
http://df6nm.bplaced.net/VLF/fec_tests/ebnaut_data_09200300.zip (750 kB).
If you carefully inspect the timestamp of the successful decode, you may note that in this case an extra one-second delay was required. Domenico confirmed that it was caused by a temporary GPS-derived clock error on the transmit PC.

Hope this helps... so go ahead and have fun!

 

Thanks to Paul and Wolf for their original ideas and invaluable software, and to Domenico for surmounting all obstacles to bring this to life on 137 kHz.

 

Best 73,
Markus

-----Ursprüngliche Mitteilung----- 
Von: IZ7SLZ <iz7slz.domenico@gmail.com>
An: rsgb_lf_group <rsgb_lf_group@blacksheep.org>
Verschickt: Mi, 23 Sept 2015 2:06 am
Betreff: LF: Coherent BPSK on LF using EbNaut

Hello LF,

I am pleased to report a successful "Coherent BPSK" QSO between DF6NM
and IZ7SLZ on LF using EbNaut program of Paul Nicholson.
QSO took place in the night between Friday 2015-09-18 and Saturday
2015-09-19 .

I think it was the first time that anyone tried this on 137 kHz !

Frequencies used were 137510.00 Hz (DF6NM) and 137502.12 (IZ7SLZ).

We exchange messages of 16 characters in coherent BPSK with UT
synchronous symbols, using a symbol period of 1 second and 8K19A scheme
coding.

Software used were the Windows' versions of EbNaut-TX and EbNaut-RX (tnx
to P. Nicholson: your software is working without problems!).
Also Spectrum Lab (tnx Wolf !) was used to export FFT data, in a similar
way of the Markus' OPDS setup.
Other programs, written by Markus, were used: one to convert FFT
exported data for the EbNaut entry and another to search for the best
frequency and time offsets.

There was not much phase drift during the qso and in other test
transmissions, since we reach a good frequency stability at both ends
thanks to Rubidium Reference Oscillator (Markus side) and a good quality
OCXO from my side.

You can see reported symbol / carrier EbN0 and also the messages
exchanged, in the attached screen-shots taken at both ends.

We hope now to joint other stations that want to challenge the timing
and frequency accuracy, to have a QSO in LF close to the Shannon limit.

Markus DF6NM, will report soon all particulars and his comments of the
experiment. Thanks in advance Markus!

73, Domenico (iz7slz) 
 
 
Re: EbNaut - success again!
Von: Markus Vester <markusvester@aol.com>
An: iz7slz.domenico <iz7slz.domenico@gmail.com>
Cc: paul <paul@abelian.netcom.co.uk>
Datum: Mi, 19 Aug 2015 4:58 pm 

Hi Domenico,

 

here's some guidance towards setting up EbNaut for receiving, using your opds SpecLab instance.

 

1. Generating a stable test transmission

For accurate transmissions, it would be better to avoid the SSB and audio carrier method as it is prone to soundcard glitches. You may rather want to generate the carrier directly at RF (eg from your AD9850 DDS), and apply the DBM modulation there. In your setup, you might need to make a small predriver amplifier to feed the PA.

 

2. Decoder dry test

First try a dry test of the software using the Sunday 23:14 data file. Download the conversion program
http://df6nm.bplaced.net/VLF/fec_tests/ebnaut_ifft2.zip
(including my calibration data file sr.txt) and the data
http://df6nm.bplaced.net/VLF/fec_tests/08162314.zip .
Open a command shell and type
 ebnaut_ifft2 08162314
which will output a couple of lines and generate a 08162314.wav file. Load this into ebnaut-rx, using parameters as shown in attached screenshot (start offsets 248.5s, freq offset 2.0369 Hz). This should reproduce the perfect decode.

 

3. Setting up the opds SpecLab instance

As you have successfully received opds no big changes are nessary. PC time needs to be accurate to 0.2 s using NTP or GPStime. I'd recommend to activate samplerate tracking, using either a tone or 1 pps signal.

 

4. Saving data files

If opds detector software is running in parallel, disable the file deletion in cleanup bat, so that the *.txt files will not be removed from the c:/spectrum/data directory, e.g. by outcommenting the second and third lines
 @copy headline.txt + DETECTED.TXT + detecold.txt c:\dropbox\public\detected.txt >nul
 ::@del c:\spectrum\data\*.txt 1>nul 2>nul
 ::@copy dirw.txt dirdone.txt >nul
Note that changing cleanup.bat will take effect immediately, without having to restart opds.

A new data file will be created every 10 minutes, with the filename corresponding to minute of the middle of the aquisition period. Note that in the long run this may accumulate a large amount of data on your disk. You can reduce filesize by selecting the "use INTEGER numbers only" option in the "File - Export.. - Export of FFT results" menu. In that case please check the data file output, if it consists of many zeroes or single digit numbers, you'll need to increase the audio level.

 

5. Opds calibration

You should then perform an accurate frequency calibration. Read the samplerate from the samplerate detector. The RF center frequency can best be checked using Loran lines. In the mediteranean area, lines from Soustons should be detectable in daytime at exact multiples of 50000/6731 = 7.428316743 Hz, eg. 137505.5712 Hz (see https://dl.dropboxusercontent.com/u/26404526/Loran_lines_in_LF_slots.txt ). Use the cursor to read their display frequency and adjust the Options - ..part 2.. - Radio Freq Offset entry according to the error. You can then then enter use the calibration  in opds.ini.

 

6. Converting to a wav file

Then you need to select a data file containing an ebnaut signal or a test carrier, and convert it to a wav file. To use the ifft program, you should first edit sr.txt and enter your RX calibration result. This file contains four numbers, which are the soundcard samplerate, the FFT decimation factor, the FFT length (usually 524288), and the center frequency. Copy the data file from spectrum/data and run ebnaut_ifft2 as before.

 

7. Searching frequency offset

Start Paul's ebnaut-rx software and open a wav file which contains a pure and reasonbly strong carrier. Enter the difference to the calibrated center frequency (positive for signals above center), and start a decode using 1 sec symbol duration for 1 Hz search range, and eg. 16 for message length. Within a couple of seconds, ebnaut will have preprocessed the audio and written a rawsyms.txt file. The decode attempt can then be aborted by pressing Stop.

We now have to inspect rawsys.txt, which contains the complex baseband signal at one sample per symbol. We want to plot the contents of the fourth column showing the phase. I am using the attached MathCad script for this, but you might as well employ Excel, gnuplot, or any other software. Ideally the data points should be gathered along a horizontal line, indicating constant phase and zero frequency error. Usually the line will have a slope, and you will have to adjust the Freq offset field and repeat the procedure. Finally you should do a fine-adjust using a longer symbol duration like 10 seconds.

Unfortunately this is a lengthy procedure, and it becomes very difficult or impossible when the signal is weak. This is the reason why it's so important to have a stable and repeatable frequency. At LF, a simple TCXO usually has too much variation, and we really want a Rubidioum or GPS-controlled oscillator, or a very stable OCXO.

 

8. Searching time offset

Now we can go about decoding a wav containing a real ebnaut sequence. The remaining task isto determine the time offset between the beginning of the file and the sequence start. Theoretically it can be obtained from the difference of the displayed file start (e.g. 22:55:52.610) and the known transmission (e.g. 23:00:00, in this case 4 minutes and 7.39 seconds, i.e. 247.39 seconds). Unfortunately, there usually seems to be a small extra offset (around 0 to 2 seconds) which is initially unknown. So we will need a little trial and error.

Now start ebnaut with the theoretical start offset, and look at rawsyms.txt again. If the signal is strong enough, the dots should now gathered around  dots around two lines, 180 degrees apart. Most likely there will a number of dots scattered elsewhere, due to a time error which leads to sampling during symbol transitions.

Then try to repeat the procedure, varying the start offset in steps of 0.2 symbol periods (eg. 0.2 sec), trying to minimize the number of scattered points between the lines. If the signal is well above the noise you will find a clear optimum. If the two lines are still sloping, adjust the frequency offset in small steps (+-0.0002 Hz).

Let Ebnaut run a couple of minutes. If you are lucky you will already be getting a decode.

If no decode happens, the start time may still be wrong by an integer number of symbols periods. Then try to increase time offset in steps of one symbol duration. With one of these offsets, the signal should be decoded successfully.

 

You can still play around with the time offset to reduce symbol errors and maximize EbN0. Take a note of the best start offset. Assuming that the opds spectrogram is running continuously, a new file will be written exactly every 10 minutes. Thus the start offset can be reused for the next transmission, e.g. one hour later. After a SpecLab restart, the file start times will have changed arbitrarily, but the empirical extra offset will likely still be valid.

Maybe we can do a test from me to you on the weekend? However before going on air, I want to insert a simple RC shaping network before the modulator, to avoid splattering up Stefan's grabber and opds screen too much.

 

All the best,
Markus