Re: LF: Smart noise cancelling?!?

[Andy Talbot <andy.g4jnt@gmail.com>]

But surely, when narrowband filtering is in place - as any narrow band mode will of necessity be doing internally - any wideband non Gaussian or bursty noise when applied to this narrow filter will eventually become Gaussian IN THAT BANDWIDTH

We first ealised this in teh original QRSS tests with G3PLX back in the 1990's.   73kHz was full of spikes and 'crud' from teh then existing Decca signals and other stuff.   But when Peter examined the output from the narrow filters, (the FFT bins)  it lookdd like, and appeared to show itself to be Gaussian.   He said it passed the tests for Gaussian noise

A mathematician could probably prove that any random non Gaussian signal if filtered sufficiently narrow in comparison to itsnature, would end up Gaussian in the filtered bandwidth. 

In fact, to end up non-Gaussian, it would have to have definite components repeating at rates very close to the reciprocal of the bandwidth of the filter.

Andy  G4JNT

On 3 November 2016 at 19:24, Alan Melia <alan.melia@btinternet.com> wrote:

This is an interesting idea. We often said in the early days of LF that LF "noise" was not Gaussian.
I do remember attending a lecture at work by one of the resident maths team. (Pitching for customers) He was considering telecom "noise" not radio. But he demonstrated the standard processes, which assume Gaussian noise were not effective on his example. He then introduced a "new" parameter he called "burstiness". He was using Burst Error Rate as a metric, I think, rather than S/N but the ideas were quite interesting. He was also investigating different error-detection/correction algorithms against the "burstness" index.. I am afraid it was mid 1980s and I dont remember much detail (or even a name to follow for references) A few faces in the audience were quite blank, one or two were leaning forward and some sprawled backward as if to say "What's the problem, if its noisy, close the door" :-))

Alan
G3NYK

----- Original Message ----- From: "DK7FC" <selberdenken@posteo.de>
To: <rsgb_lf_group@blacksheep.org>
Sent: Thursday, November 03, 2016 6:37 PM
Subject: Re: LF: Smart noise cancelling?!?

Hi Peter,

That sounds interesting and looks convincing. But it only works for 'compressed' spectra covering several kHz and hours, right?

As far as i understand this method can't be done for weak signal detections because you 'need' all the energy available from the weak signal to fill the bin (one pixel) and just throw away what causes a reduction of the SNR, so just the stronger QRN bursts are blanked and most of the signal is coming through whereas your method selects just a small fraction of the incoming energy (?).

My idea/question came from the consideration that different kind of QRN has different optimum blanker settings.

Am 02.11.2016 21:25, schrieb Paul Nicholson:

Even more aggressive sferic blanking raises
the decode to Eb/N0 +1.7 dB BER 38.2%
S/N 16.10 dB in 25.4 uHz.

So, if propagation changes, the optimal blanker settings will change. So they vary all the time. If a post-processing of a transmission/recording taking several hours and night/day changes, it could be useful to dynamically vary the blanker settings. So these blanker setting levels would have to be determined/calculated by the incoming signal and then applied in a next step.
This may be CPU-load intensive, i don't know, it's behind my current skills. But the idea is there...


73, Stefan


Am 03.11.2016 18:55, schrieb Peter:

Hi Stefan et al.,

On 03.11.2016 14:22, DK7FC wrote:

...
Last night i thought a bit about noise cancelling on LF/VLF.

> ...

Running an experimental receiver at VLF/LF for SID-detection (on RPi 3) I chose an almost non-parametric procedure running in frequency domain. It works as follows:
Do a windowed FFT, compute the median (by sorting) from the power spectrum. Store all spectra and corresponding median values. Next choose a time period (let's say 100ms), pick the spectrum with the lowest median, plot it, drop all the others. The key is that "Median values" are more robust to outliers compared to other averaging procedures.

See what happened when switching from simple averaging to median selection algorithm (~16:50 utc):
http://lf-radio.de/cgi-bin/test/show_wf.cgi?date=16-10-02

I know that this won't work in case of searching for coherent signal detection, or would be hard to implement. But using this method I'm detecting such very weak signals from far east like NDI or RTZ on a regular basis.

Drawbacks? Yes; it's throwing away a lot of information which may be useful. Another pitfall has to be mentioned: using 1 sec. as a selection window strong time service transmitters nearly vanished since the algorithm will unerringly choose the gaps [^_^]. Therefore I'm using only the spectral part between 15 and 50 kHz for computing the median values.

Peter, df3lp