MIME-Version: 1.0
In-Reply-To: <1173910789.20161013103634@chriswilson.tv>
References: <1585067730.20161012155231@chriswilson.tv> <B2DEB8B3AB3E4CFA94B0685D61917F18@AGB>
 <CAA8k23RYdWYQ9p4BXBVnqEtkJ5E_Wj24KBouMmEUKyzdfSGEFQ@mail.gmail.com> <1173910789.20161013103634@chriswilson.tv>
From: Andy Talbot <andy.g4jnt@gmail.com>
Date: Thu, 13 Oct 2016 13:16:50 +0100
Message-ID: <CAA8k23RUtkOBVt64qhgP9sDg1E4OL0tDgcbMEXNx4EKZoAA_WQ@mail.gmail.com>
To: rsgb_lf_group@blacksheep.org
Subject: Re: Re[2]: LF: Running 2 WSPR transmissions, how to stop them TX'ing
 at the same time??
Content-Type: multipart/alternative; boundary=001a114021ec9c9b8d053ebe15c4
Sender: owner-rsgb_lf_group@blacksheep.org
Precedence: bulk
Reply-To: rsgb_lf_group@blacksheep.org
Content-Length: 14084
Status: O

--001a114021ec9c9b8d053ebe15c4
Content-Type: text/plain; charset=UTF-8

No 41dB is NOT a real S/N figure and is due solely to your artificially
restricting the noise bandwidth gon gin !    If you use a receiver
bandwidth any narrower than the full SSB bandwidth then the S/N reporting
will be wrong.

With just one exception (*) there is no benefit at all to be gained from
using a narrow receiver filter.  It does absolutely nothing to improve
signal reception and just makes your noise measurement wrong.  The reasons
are as follows :

All filtering of the signal is done in the DSP software, where for WSPR it
is bandpass filtered to a bank of 1.46Hz wide filters, called bins (as in
dustbin) - this being the tone spacing and the speed of a WSPR signal.
There may be some incidental pre-filtering on the way to getting down to
this narrow bandwidth, but it is not relevant to the process.  ONLY the
final 1.46Hz bandwidth is the one that matters.

Measuring the signal level  (the S part of the S/N) is straightforward.
Your four tones fall in four of the 1.46Hz wide adjacent parallel filter
banks corresponding to each of the four tones transmitted.   The sum of the
power in the tones is then the signal in the four adjacent bins  added
together and  is just a relative number.

But to get S/N we now have to make a noise measurement and here is where it
gets difficult.  There are many ways of  estimating the noise and one of
the simplest is to take everything going into the soundcard / software and
assume that as it is so wide, a few weak signals present  won't affect the
average too much; ie.  that it is all noise.
An assumption has to be made about the input now, and since SSB radios are
used, the assumption is made that the noise is bandpass filtered at this
point to 2.5kHz width - this being the bandwidth of a typical SSB
filter.    By measuring the signal over this full bandwidth a value for N
in 2.5kHz can be obtained.

As the WSJT standard for S/N is to express it in a 2.5kHz bandwidth, simply
dividing the previously measured S value by this N (for 2.5kH input)   will
give the final S/N value.   If DSP prefiltering has been included in the
software, the noise will have been measured over just this pre-filtered
bandwidth and the lower resulting N is then scaled up to what it would be
in 2.5kHz.  Noise power is proportional to bandwidth, so a 250Hz prefilter
means the measured total noise is now 10dB lower and this correction has ot
be added on

Simples !   Except it is all too easy to go very wrong.

First of all, if the input noise is externally filtered (say by a narrow CW
filter) its total power will be lower by the proportion of this BW to
2.5kHz.  So the value of N will be less.    S stays the same (it is
filtered in software to 4 * 1.46Hz)  so same S, lower N  results in a
higher S/N  when the input is bandpass  filtered.

This is clearly why a certain F station consistently reports ridiculously
high values of S/N for everyone.

Next is what happen when strong signals come up inside the SSB filter width
but outside the WSPR band.   These can get lumped into the overall
measurement and give an artificially high value of N.   They are some way
away from the signal of interest so don't actually affect the decoding
process, but to lead to a falsely low value for S/N.     This is probably
happening for stations who consistently report lower than expected S/N for
many stations.

There are many ways that more accurate estimates of noise can be obtained,
using statistics and probability distributions to separate out genuine
noise from signals.   Others just by looking for the presence of signals
and adding everything else up that doesn't fit that criteria of 'signal'.
All are used in the electronic Warfare and Communications Intercept
business to automatically detect and then classify signals.
But it is a complicated process just to obtain an incidental measurement
value for information only, that does nothing to help the actual decoding
process.

(*) The one exception, where a narrow input filter will help is if a very
strong local signal appears in the SSB filter passband that is sufficient
to depress the AGC.    This can degrade the wanted signal.   However, if
the receiver is properly designed, the S/N of the wanted signal in the now
depressed audio should still be the same.   A soundcard has typically 80dB
dynamic range or more, so unless the unwanted signal  is say  60 to 70dB
above the WSPR signal of interest, it should still cope.  Although the N
measurement will be wrong for the reasons detailed above

For a description of how signals can be detected in noise, take a look at
the  RadCom  Data columns for  April and June 2008.  There's also a
description on page 47  my book "Command,  Computers, Microcontrollers and
DSP for the Radio Amateur".   RSGB still have a few copies - I saw it on
the stand at the Convention!  (Although several sections in there are
embarrassingly out of date by now).

Andy  G4JNT


On 13 October 2016 at 10:36, Chris Wilson <chris@chriswilson.tv> wrote:

> Hello Andy,
>
> Thursday, October 13, 2016
>
> Thanks for the detailed explanation Andy. Good copy thoughout the
> night on your WSPR2 and 15 signals. Early this morning I greatly
> reduced the filter width to just encompass the needed section of the
> band to hear your WSPR2 signal and the level reported from WSPR went
> mad high. I read a paper you wrote on S/N levels in WSPR and filter
> width but it rather went over my head. I assume +41 SNR is not a
> "real" figure when filter width is reduced right down?
>
>
> Best regards,
>  Chris  2E0ILY                          mailto:chris@chriswilson.tv
>
>  My part time LF grabber is at http://www.chriswilson.tv/grabber.html
>
>
> > Correct.   I have a PIC that reads timing data from a GPS and at
> > the right timing uses a set of pre-stored WSPR symbols to calculate
> > the frequencies for the tones and sends the resulting data to an
> > AD9852 DDS.  Different symbol sets (for the two messages) and
> > frequency data is stored for each of the two modes.
>
> <SNIP>
>
>
>

--001a114021ec9c9b8d053ebe15c4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>No 41dB is NOT a real S/N figure and is due solely to=
 your artificially restricting the noise bandwidth gon gin !=C2=A0=C2=A0=C2=
=A0 If you use a receiver bandwidth any narrower than the full SSB bandwidt=
h then the S/N reporting will be wrong.</div><div><br></div><div>With just =
one exception (*) there is no benefit at all to be gained from using a narr=
ow receiver filter.=C2=A0 It does absolutely nothing to improve signal rece=
ption and just makes your noise measurement wrong.=C2=A0 The reasons are as=
 follows :</div><div><br></div><div>All filtering of the signal is done in =
the DSP software, where for WSPR it is bandpass filtered to a bank of 1.46H=
z wide filters, called bins (as in dustbin)=C2=A0- this being the tone spac=
ing and the speed of a WSPR signal.=C2=A0=C2=A0=C2=A0 There may be some inc=
idental pre-filtering on the way to getting down to this narrow bandwidth, =
but it is not relevant to the process.=C2=A0 ONLY the final 1.46Hz bandwidt=
h is the one that matters.=C2=A0=C2=A0 </div><div><br></div><div>Measuring =
the signal level=C2=A0 (the S part of the S/N) is straightforward.=C2=A0=C2=
=A0 Your four tones fall in four of the 1.46Hz wide adjacent parallel filte=
r banks corresponding to each of the four tones transmitted.=C2=A0=C2=A0 Th=
e sum of the power in=C2=A0the=C2=A0tones is then the signal in the four ad=
jacent=C2=A0bins =C2=A0added together and=C2=A0 is just a relative number.<=
/div><div><br></div><div>But to get S/N we now have to make a noise measure=
ment and here is where it gets difficult.=C2=A0 There are many ways of=C2=
=A0 estimating the noise=C2=A0and one of the=C2=A0simplest is to take every=
thing going into the soundcard=C2=A0/ software and assume that as it is so =
wide, a few weak signals present =C2=A0won&#39;t affect the average too muc=
h; ie.=C2=A0=C2=A0that it is all noise.=C2=A0=C2=A0 </div><div>An assumptio=
n has to be made about the input now, and since SSB radios are used, the as=
sumption is made that the noise is bandpass filtered at this point to 2.5kH=
z width - this being the bandwidth of a typical SSB filter.=C2=A0=C2=A0=C2=
=A0 By measuring the signal over this full bandwidth a value for N in 2.5kH=
z=C2=A0can be obtained.</div><div><br></div><div>As the WSJT standard for S=
/N is to express it in a 2.5kHz bandwidth, simply dividing the previously m=
easured=C2=A0S value by this N (for 2.5kH input)=C2=A0=C2=A0 will give the =
final S/N value.=C2=A0=C2=A0=C2=A0If DSP prefiltering has been included in =
the software, the noise will have been=C2=A0measured over just this pre-fil=
tered bandwidth and the lower resulting N is then scaled up to what it woul=
d be in 2.5kHz.=C2=A0 Noise power is proportional to bandwidth, so a 250Hz =
prefilter means the measured total noise is now 10dB lower and this correct=
ion has ot be added on</div><div><br></div><div>Simples !=C2=A0=C2=A0 Excep=
t it is all too easy to go very wrong.</div><div><br></div><div>First of al=
l, if the input noise is externally filtered (say by a narrow CW filter) it=
s total power will be lower by the proportion of this BW to 2.5kHz.=C2=A0 S=
o the value of N will be less.=C2=A0=C2=A0=C2=A0 S stays the same (it is fi=
ltered in software to 4 * 1.46Hz)=C2=A0 so same S, lower N=C2=A0 results in=
 a higher S/N=C2=A0 when the input is bandpass =C2=A0filtered.</div><div><b=
r></div><div>This is clearly why a certain F station consistently reports r=
idiculously high values of S/N for everyone.</div><div><br></div><div>Next =
is what happen when strong signals come up inside the SSB filter width but =
outside the WSPR band.=C2=A0=C2=A0 These can get lumped into the overall me=
asurement and give an artificially high value of N.=C2=A0=C2=A0 They are so=
me way away from the signal of interest so don&#39;t actually affect the de=
coding process, but to lead to a falsely low value for S/N.=C2=A0=C2=A0=C2=
=A0=C2=A0 This is probably happening for stations who consistently report l=
ower than expected S/N for many stations.</div><div><br></div><div>There ar=
e many ways that more accurate estimates of noise can be obtained, using st=
atistics and probability distributions to separate out genuine noise from s=
ignals.=C2=A0=C2=A0 Others just by looking for the presence of signals and =
adding everything else up that doesn&#39;t fit that criteria of &#39;signal=
&#39;.=C2=A0=C2=A0 All are used in the electronic Warfare and Communication=
s Intercept business to automatically detect and then classify signals.=C2=
=A0=C2=A0</div><div>But it is a complicated process just to obtain an incid=
ental measurement value for information only, that does nothing to help the=
 actual decoding process.</div><div><br></div><div>(*) The one exception,=
=C2=A0where a narrow input filter will help is if a very strong local signa=
l appears in the SSB filter passband that is sufficient to depress the AGC.=
=C2=A0=C2=A0=C2=A0 This can degrade the wanted signal.=C2=A0=C2=A0 However,=
 if the receiver is properly designed, the S/N of the wanted signal=C2=A0in=
 the now depressed audio should still be the same.=C2=A0=C2=A0 A soundcard =
has typically 80dB dynamic range or more,=C2=A0so unless the unwanted signa=
l =C2=A0is say=C2=A0 60 to 70dB above the WSPR signal of interest, it shoul=
d still cope.=C2=A0 Although the N measurement will be wrong for the reason=
s detailed above</div><div><br></div><div>For a description of how signals =
can be detected in noise, take a look at the =C2=A0RadCom=C2=A0 Data column=
s for=C2=A0 April and June 2008.=C2=A0 There&#39;s also a description=C2=A0=
on page 47 =C2=A0my book &quot;Command,=C2=A0 Computers, Microcontrollers a=
nd DSP for the Radio Amateur&quot;.=C2=A0=C2=A0 RSGB still have a few copie=
s - I saw it on the stand at the Convention!=C2=A0 (Although several sectio=
ns in there are embarrassingly out of date by now).</div><div><br></div><di=
v>Andy=C2=A0 G4JNT</div><div><br></div><div><br></div><div><br></div><div><=
br></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On 13 Oc=
tober 2016 at 10:36, Chris Wilson <span dir=3D"ltr">&lt;<a href=3D"mailto:c=
hris@chriswilson.tv" target=3D"_blank">chris@chriswilson.tv</a>&gt;</span> =
wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8=
ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1p=
x;border-left-style:solid">Hello Andy,<br>
<br>
Thursday, October 13, 2016<br>
<br>
Thanks for the detailed explanation Andy. Good copy thoughout the<br>
night on your WSPR2 and 15 signals. Early this morning I greatly<br>
reduced the filter width to just encompass the needed section of the<br>
band to hear your WSPR2 signal and the level reported from WSPR went<br>
mad high. I read a paper you wrote on S/N levels in WSPR and filter<br>
width but it rather went over my head. I assume +41 SNR is not a<br>
&quot;real&quot; figure when filter width is reduced right down?<br>
<br>
<br>
Best regards,<br>
=C2=A0Chris=C2=A0 2E0ILY=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mailto:<a href=3D"mailto:chris@ch=
riswilson.tv">chris@chriswilson.tv</a><br>
<br>
=C2=A0My part time LF grabber is at <a href=3D"http://www.chriswilson.tv/gr=
abber.html" target=3D"_blank" rel=3D"noreferrer">http://www.chriswilson.tv/=
<wbr>grabber.html</a><br>
<br>
<br>
&gt; Correct.=C2=A0=C2=A0 I have a PIC that reads timing data from a GPS an=
d at<br>
&gt; the right timing uses a set of pre-stored WSPR symbols to calculate<br=
>
&gt; the frequencies for the tones and sends the resulting data to an<br>
&gt; AD9852 DDS.=C2=A0 Different symbol sets (for the two messages) and<br>
&gt; frequency data is stored for each of the two modes.<br>
<br>
&lt;SNIP&gt;<br>
<br>
<br>
</blockquote></div><br></div></div>

--001a114021ec9c9b8d053ebe15c4--