DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=U6kkvKHd3837pLMAdCtT0YsI5exotxG3FZn9GMZ+mdjUNBQ9uSJbWO0Z9zhn67sS9r
         C0JyLsyW/IMFVBcRf3X98/fcXo1+6XTUPMaqBZxxif+S376lGTqgztmx7yTGdQa8xeBy
         nN1Z8rnAOhwnSEOX1Dh1+MbqXtWwuQ7lX+Apk=
MIME-Version: 1.0
In-Reply-To: <FAEE5C1C09A4415EBB6C38E663C4EEDC@JimPC>
References: <00f801c98235$a558af20$a402a8c0@Inspiron>
	 <FAEE5C1C09A4415EBB6C38E663C4EEDC@JimPC>
Date: Thu, 29 Jan 2009 21:51:05 +0000
Message-ID: <fc7eccec0901291351q203d6e63p52284a7844ddee48@mail.gmail.com>
From: Andy Talbot <andy.g4jnt@googlemail.com>
To: rsgb_lf_group@blacksheep.org
DomainKey-Status: good (testing) 
Subject: Re: LF: Re: WSPR Timing issue
Content-Type: multipart/alternative; boundary=0016363b9922da4fc40461a6155b
Sender: owner-rsgb_lf_group@blacksheep.org
Precedence: bulk
Reply-To: rsgb_lf_group@blacksheep.org


--0016363b9922da4fc40461a6155b
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I haven't really been following this thread, and have deleted many of the
relevent messages, unfortunately.  But you can quite safely assume that all
timing, apart from the ititial even minute kick-off, WILL most definitely be
all derived via the soundcard.   The whole beauty behind the WSPR (and the
JT65 and FSK441) protocols is that the tone spacings and signalling
intervals are coherent.  If they weren't, you couldn't have such tone narrow
spacing with that symbol interval.

The WSPR tone spacing is 12000Hz / 8192 ~~ 1.48Hz.  The signalling interval
is the reciprocal of this, or 0.68 seconds.   (there are 162 symbols, and
162 * 8192 / 12000 = 110.592 seconds - just under two minutes, the frame
interval.)   This exact relationshihp allows semi-coherent processing to be
performed.

Timing needs to be sufficient such that after 162 symbols, the final one is
still reasonably aligned.  When I was writing the JT65 PIC code eventually
used on GB3VHF and 'RAL  beacons, I mentioned this aspect to Joe K1JT, as I
knew the PIC generated timing would be in error by several tens of ppm. to
ask his opinion.  He reckoned that provided the final symbol was aligned to
better than about 50% of its length, the decoding wouldn't suffer too much.
So my tens of ppm was more than adequate.   50% actually sounds quite
generous;  I would have thought a 10% alignment was more like that required
for no serious degradation.

If the same rule applies to WSPR, this suggests a soundcard timing error of
0.68 / 2 = 340ms after 110 seconds, or 0.3%.  So a sampling rate error of
+/-37Hz at 12kHz is permitted.

I think there was a reference to 1.5 seconds error in a previous posting on
this matter - this is very bad if it refers to the run-time and not just a
timing error..
Now this is pure speculation on my part, but could, just, explain why weak
signals could be copied, and strong ones not...
If copy is good, the receiver will try to decode all the symbols, which will
not match but it won't know which to reject so teh mesasge fails in
totality.
With a weaker signal, the decoding algorithm may just look at a portion of
the stream, which may be good enough and have not drifted too much in timing
- so will generate a valid decode.

I suggest you study thoroughly the JT65 description on Joe's website, and
understand the principles of that coding - as WSPR hasn't yet been
written-up.    WSPR uses similar coding and synchronisation principles to
JT65, although obviously different physical properties such as number of
tones, spacing and time interval;  even base soundcard rate.

And yes, in Windoze programming multiple wave threads can run in
parallel.  You can open multiple buffers and they all concatenate
automatically, passing from one to another seamlessly as soon as each one
empties.   In fact, for coherent, continuous, reading or writing of wave
data, ping-pong buffers have to be set up, so that one can be processed
while the other is read from or written to,  to avoid missing samples.
Many software writers open multiple buffers within a programme so as
to allow seamless interupt processing and allow user I/O.
Its not connected to the problem in hand.


Andy  G4JNT
www.g4jnt.com


2009/1/29 James Moritz <james.moritz@btopenworld.com>

> Dear Lee, Graham, LF Group,
>
> I did some experiments at M0LMH's suggestion (see below)... I think this is
> what he meant. Perhaps the results are also of interest to  people in the
> Group who have experience of writing DSP software for the PC.
>
> The top trace in the attachment has the same WSPR beacon message as last
> night with normal windows clock settings.
>
> The middle trace shows the effect of retarding the windows clock by roughly
> one minute, while transmitting. The current sequence continues until its
> normal ending, but when the even minute occurs on the re-set clock, a new
> sequence begins as well, so for a while there are two overlapping frames.
> After that, frames are generated normally, according to the RTC timing. At
> the moment you make the clock adjustment, the time display in WSPR stops
> incrementing until the windows clock has "caught up", when it starts
> incrementing again normally.
>
> When the clock is advanced by one minute roughly (lower trace), the current
> frame continues and terminates normally. There is then a gap until the next
> even minute of about 1 minute, when the next frame is generated according to
> the RTC time setting.
>
> I suppose this shows that transmission of a frame is initiated by the real
> time clock reaching an even minute, but once started, the generation and
> timing of the bits in the frame is independent of the RTC, although it does
> not tell us how that timing is achieved. Also, the fact that the time
> display in WSPR stops when you retard the clock, and that a new frame can be
> started in the middle of an existing frame, would seem to show WSPR is
> continuously looking at the RTC data.
>
> The really funny thing is that two frames can be occuring at the same time.
> That would seem to suggest that more than one instance of the tone
> generation algorithm is in existence at once! Curious, but perhaps not
> connected to the problem in hand...
>
> Cheers, Jim Moritz
> 73 de M0BMU
>
> ----- Original Message ----- From: "Lee Hudson" <
> hudsonville@btinternet.com>
> To: "'James Moritz'" <james.moritz@btopenworld.com>; "'Graham'" <
> g8fzk@g8fzk.fsnet.co.uk>
> Sent: Thursday, January 29, 2009 5:18 PM
> Subject: WSPR Timing issue
>
>
> Hi Jim, Graham,
>
> I've been having more thoughts on Grahams timing issue, and there are a few
> possibilities.
>
> Firstly assuming Graham has been receiving reports that he is on his
> frequency from trusted sources then we can rule out sample rate error.
> The crystal on the sound card would have to be a long way out to cause such
> a problem.
> I believe Graham tried two different sound cards with no change.
>
> The second possibility is the the RTC on his PC is fast, again looking at
> the reports from last night there does seem to be some drift but definately
> not enough do be worried over.
> And also I'm sure the time of transmission start was the same from the even
> minute epoch.
>
> This leaves a third posibility I think from recalling PC architecture and
> that there is a Timer chip, this from memory is clocked from a separate
> crystal to the RTC and certainly remote from the sound card. Some PCs use
> this crystal also for producing the multiplied up CPU clocks etc.
>
> Point being here is that the Timer is what dictates any functions requiring
> sub second interval timing, and is used by windows for task switching etc.
>
> It all boils down to how the WSPR software derrives its timing to decide
> when the transition of a bit occurs.
> I looked carefully at Jim comparision image, to first check if a bit had
> actually been dropped somewhere, and I can't see one.
>
> If I had written the software I would have used the RTC time to decide when
> to start the transmission.
> Then I would have used the sample rate as the main source of timing from
> that point on until the end, even if compensating for a sample rate error.
>
> In my opinion I think the RTC is used to start the transmission, and no
> real
> error here.
> The frequency of transmission I'm assuming is also correct, and no real
> error from the sound card side.
>
> However if the timing mechanism used to decide when a bit is over and shift
> frequency is related to the Timer, this is where the issue may really lie.
> Especially if this is how the software is written.
> Using Windows Timers can yeild incremental timing issues too, and this may
> be a manifestation also.
>
> One final thought that crossed my mind, is that Graham my be using a GPS
> signal to keep his PC RTC updated or possibly an NTP server.
> From reading deep and dark Microsoft helpfiles this does also affect how
> windows maintains it's internal time derived from the Timer, this it keeps
> separate from the RTC, but occasionally makes an alignment change.
>
> If this is the case and Graham is updating his Clock say every second from
> a
> GPS, then this could be enough to cause the incremental error.
> Simple test here would be to disconnect any external time influence and see
> if things improove.
>
> My gut feeling here is that it is really a software issue, and not related
> to the hardware as such.
> Some may be caused by Windows itself not being overly well designed for
> timing purposes.
> Second the WSPR software not using the most reliable method for timing
> things.
>
> It could be worth another test for Jim to see what happens when you locally
> generate a signal and then wind the clock forward say 1 minute.
> Record one test normally, then anothor test with the time step introduced.
> Does the data output as before and complete a full 2 minute cycle
> Or does it truncate the data period skipped and finish on scheduled time
> according to the clock.
> Or does it truncate the data at the end.
> This will give some clue as to how the data bits themselves are timed.
> Be intresting also to time step backwards too!!
>
> Cheers,
> Lee.
>
>

--0016363b9922da4fc40461a6155b
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>I haven&#39;t really been following this thread, and have deleted many=20=
of the relevent messages, unfortunately.&nbsp; But you can quite safely assu=
me that all timing, apart from the ititial even minute kick-off,&nbsp;WILL m=
ost definitely&nbsp;be all derived via the soundcard.&nbsp;&nbsp; The whole=20=
beauty behind the WSPR (and the JT65 and FSK441) protocols is that the tone=20=
spacings and signalling intervals are coherent.&nbsp; If they weren&#39;t, y=
ou couldn&#39;t have such tone narrow spacing with that symbol interval.&nbs=
p; </div>

<div>&nbsp;</div>
<div>The WSPR tone spacing is 12000Hz / 8192&nbsp;~~ 1.48Hz.&nbsp; The signa=
lling interval is the reciprocal of this, or 0.68 seconds.&nbsp;&nbsp; (ther=
e are 162 symbols, and 162 * 8192 / 12000 =3D 110.592 seconds - just under t=
wo minutes, the frame interval.)&nbsp;&nbsp; This exact relationshihp allows=
 semi-coherent processing to be performed.</div>

<div>&nbsp;</div>
<div>Timing needs to be sufficient such that after 162 symbols, the final on=
e is still reasonably aligned.&nbsp; When I was writing the JT65 PIC code ev=
entually used on GB3VHF and &#39;RAL&nbsp; beacons, I mentioned this aspect=20=
to Joe K1JT, as I knew the PIC generated timing would be in error by several=
 tens of ppm. to ask his opinion.&nbsp; He reckoned that provided the final=20=
symbol was aligned to better than about 50% of its length, the decoding woul=
dn&#39;t suffer too much.&nbsp; So my tens of ppm was more than adequate.&nb=
sp;&nbsp; 50% actually sounds quite generous; &nbsp;I would have&nbsp;though=
t a 10% alignment was more like that required for no serious degradation.</d=
iv>

<div>&nbsp;</div>
<div>If the same rule applies to WSPR, this suggests a soundcard timing erro=
r of 0.68 / 2 =3D 340ms after 110 seconds, or 0.3%.&nbsp; So a sampling rate=
&nbsp;error&nbsp;of +/-37Hz at 12kHz is permitted.</div>
<div>&nbsp;</div>
<div>I think there was a reference to 1.5 seconds error in a previous postin=
g on this matter - this is very bad if it refers to the run-time and not jus=
t a timing error..</div>
<div>Now this is pure speculation on my part, but could, just, explain why w=
eak signals could be copied, and strong ones not...</div>
<div>If copy is good, the receiver will try to decode all the symbols, which=
 will not match but it won&#39;t know which to reject so teh mesasge fails i=
n totality.</div>
<div>With a weaker signal, the decoding algorithm may just look at a portion=
 of the stream, which may be good enough and have not drifted too much in ti=
ming - so will generate a valid decode.</div>
<div>&nbsp;</div>
<div>I suggest you study thoroughly the JT65 description on Joe&#39;s websit=
e, and understand the principles of that coding - as WSPR hasn&#39;t yet bee=
n written-up.&nbsp;&nbsp;&nbsp; WSPR uses similar coding and synchronisation=
&nbsp;principles to JT65, although obviously different physical properties s=
uch as number of tones, spacing and time interval;&nbsp; even base soundcard=
 rate.</div>

<div>&nbsp;</div>
<div>And yes, in Windoze programming&nbsp;multiple wave threads can run in p=
arallel.&nbsp;&nbsp;You can open multiple buffers and they all concatenate a=
utomatically, passing from one to another seamlessly as soon as&nbsp;each on=
e empties.&nbsp;&nbsp; In fact, for coherent, continuous, reading or writing=
 of wave data, ping-pong buffers have to be set up, so that one can be proce=
ssed while the other is read from or written to, &nbsp;to avoid missing samp=
les.&nbsp;&nbsp; Many software writers open multiple buffers within a progra=
mme so as to&nbsp;allow seamless interupt processing and allow user I/O. </d=
iv>

<div>Its not connected to the problem in hand.</div>
<div>&nbsp;</div>
<div><br clear=3D"all">Andy &nbsp;G4JNT<br><a href=3D"http://www.g4jnt.com/"=
 target=3D"_blank">www.g4jnt.com</a><br><br><br></div>
<div class=3D"gmail_quote">2009/1/29 James Moritz <span dir=3D"ltr">&lt;<a h=
ref=3D"mailto:james.moritz@btopenworld.com" target=3D"_blank">james.moritz@b=
topenworld.com</a>&gt;</span><br>
<blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0p=
x 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">Dear Lee, Graham, LF Group,<br><br=
>I did some experiments at M0LMH&#39;s suggestion (see below)... I think thi=
s is what he meant. Perhaps the results are also of interest to &nbsp;people=
 in the Group who have experience of writing DSP software for the PC.<br>
<br>The top trace in the attachment has the same WSPR beacon message as last=
 night with normal windows clock settings.<br><br>The middle trace shows the=
 effect of retarding the windows clock by roughly one minute, while transmit=
ting. The current sequence continues until its normal ending, but when the e=
ven minute occurs on the re-set clock, a new sequence begins as well, so for=
 a while there are two overlapping frames. After that, frames are generated=20=
normally, according to the RTC timing. At the moment you make the clock adju=
stment, the time display in WSPR stops incrementing until the windows clock=20=
has &quot;caught up&quot;, when it starts incrementing again normally.<br>
<br>When the clock is advanced by one minute roughly (lower trace), the curr=
ent frame continues and terminates normally. There is then a gap until the n=
ext even minute of about 1 minute, when the next frame is generated accordin=
g to the RTC time setting.<br>
<br>I suppose this shows that transmission of a frame is initiated by the re=
al time clock reaching an even minute, but once started, the generation and=20=
timing of the bits in the frame is independent of the RTC, although it does=20=
not tell us how that timing is achieved. Also, the fact that the time displa=
y in WSPR stops when you retard the clock, and that a new frame can be start=
ed in the middle of an existing frame, would seem to show WSPR is continuous=
ly looking at the RTC data.<br>
<br>The really funny thing is that two frames can be occuring at the same ti=
me. That would seem to suggest that more than one instance of the tone gener=
ation algorithm is in existence at once! Curious, but perhaps not connected=20=
to the problem in hand...<br>
<br>Cheers, Jim Moritz<br>73 de M0BMU<br><br>----- Original Message ----- Fr=
om: &quot;Lee Hudson&quot; &lt;<a href=3D"mailto:hudsonville@btinternet.com"=
 target=3D"_blank">hudsonville@btinternet.com</a>&gt;<br>To: &quot;&#39;Jame=
s Moritz&#39;&quot; &lt;<a href=3D"mailto:james.moritz@btopenworld.com" targ=
et=3D"_blank">james.moritz@btopenworld.com</a>&gt;; &quot;&#39;Graham&#39;&q=
uot; &lt;<a href=3D"mailto:g8fzk@g8fzk.fsnet.co.uk" target=3D"_blank">g8fzk@=
g8fzk.fsnet.co.uk</a>&gt;<br>
Sent: Thursday, January 29, 2009 5:18 PM<br>Subject: WSPR Timing issue<br><b=
r><br>Hi Jim, Graham,<br><br>I&#39;ve been having more thoughts on Grahams t=
iming issue, and there are a few<br>possibilities.<br><br>Firstly assuming G=
raham has been receiving reports that he is on his<br>
frequency from trusted sources then we can rule out sample rate error.<br>Th=
e crystal on the sound card would have to be a long way out to cause such<br=
>a problem.<br>I believe Graham tried two different sound cards with no chan=
ge.<br>
<br>The second possibility is the the RTC on his PC is fast, again looking a=
t<br>the reports from last night there does seem to be some drift but defina=
tely<br>not enough do be worried over.<br>And also I&#39;m sure the time of=20=
transmission start was the same from the even<br>
minute epoch.<br><br>This leaves a third posibility I think from recalling P=
C architecture and<br>that there is a Timer chip, this from memory is clocke=
d from a separate<br>crystal to the RTC and certainly remote from the sound=20=
card. Some PCs use<br>
this crystal also for producing the multiplied up CPU clocks etc.<br><br>Poi=
nt being here is that the Timer is what dictates any functions requiring<br>=
sub second interval timing, and is used by windows for task switching etc.<b=
r>
<br>It all boils down to how the WSPR software derrives its timing to decide=
<br>when the transition of a bit occurs.<br>I looked carefully at Jim compar=
ision image, to first check if a bit had<br>actually been dropped somewhere,=
 and I can&#39;t see one.<br>
<br>If I had written the software I would have used the RTC time to decide w=
hen<br>to start the transmission.<br>Then I would have used the sample rate=20=
as the main source of timing from<br>that point on until the end, even if co=
mpensating for a sample rate error.<br>
<br>In my opinion I think the RTC is used to start the transmission, and no=20=
real<br>error here.<br>The frequency of transmission I&#39;m assuming is als=
o correct, and no real<br>error from the sound card side.<br><br>However if=20=
the timing mechanism used to decide when a bit is over and shift<br>
frequency is related to the Timer, this is where the issue may really lie.<b=
r>Especially if this is how the software is written.<br>Using Windows Timers=
 can yeild incremental timing issues too, and this may<br>be a manifestation=
 also.<br>
<br>One final thought that crossed my mind, is that Graham my be using a GPS=
<br>signal to keep his PC RTC updated or possibly an NTP server.<br>From rea=
ding deep and dark Microsoft helpfiles this does also affect how<br>windows=20=
maintains it&#39;s internal time derived from the Timer, this it keeps<br>
separate from the RTC, but occasionally makes an alignment change.<br><br>If=
 this is the case and Graham is updating his Clock say every second from a<b=
r>GPS, then this could be enough to cause the incremental error.<br>Simple t=
est here would be to disconnect any external time influence and see<br>
if things improove.<br><br>My gut feeling here is that it is really a softwa=
re issue, and not related<br>to the hardware as such.<br>Some may be caused=20=
by Windows itself not being overly well designed for<br>timing purposes.<br>
Second the WSPR software not using the most reliable method for timing<br>th=
ings.<br><br>It could be worth another test for Jim to see what happens when=
 you locally<br>generate a signal and then wind the clock forward say 1 minu=
te.<br>
Record one test normally, then anothor test with the time step introduced.<b=
r>Does the data output as before and complete a full 2 minute cycle<br>Or do=
es it truncate the data period skipped and finish on scheduled time<br>
according to the clock.<br>Or does it truncate the data at the end.<br>This=20=
will give some clue as to how the data bits themselves are timed.<br>Be intr=
esting also to time step backwards too!!<br><br>Cheers,<br>Lee.<br><br>
</blockquote></div><br>

--0016363b9922da4fc40461a6155b--