Maker Pro
Maker Pro

What Nyquist Didn't Say

J

John Larkin

Jan 1, 1970
0
["Followup-To:" header set to sci.electronics.design.]
On Fri, 29 Sep 2006 13:55:32 -0700,
in Msg. said:
Don't you go knocking whiteboards. A big board with a nice fresh set
of markers will multiply my IQ by about 1.3 or so.

But only with the smelly markers. The water-based stuff doesn't do it
for me.

robert

Absolutely. And the solvent smell is part of the experience.

John
 
T

Tim Wescott

Jan 1, 1970
0
Robert said:
["Followup-To:" header set to sci.electronics.design.]
On Sat, 30 Sep 2006 09:22:43 -0700,
in Msg. said:
I have to say that this paper rather turned into a monster while I was
writing it. I thought it was going to be between 2000 and 3000 words,
with almost no math and very little real work. Instead it's about 5500
words, and I've got about a man-week into all those pretty charts and
graphs (I should publish an appendix with "the making of..." along with
all the math underneath).

As a reaction to this I haven't done my usual stage of letting it rest
and getting back to it -- I was afraid I'd never do the "getting back to
it" step. Instead I've put it outside without giving it time to get
it's coat and boots on. If I can figure out how to tighten it up I
certainly will -- assuming that I don't run away screaming at the
thought of doing even _more_ work on it.


Let me tell you that this article is so high-class from an educational
point of view that it /deserves/ coat and boots. Next time anybody comes
up to me and wants something explained about Nyquist I can just point
them at your page, and I'm sure many others will do likewise.

The only thing I'd want is the whole thing as a pretty, nicely printable
PDF document. But don't listen to me.

robert

Thanks for the comment. The motivation for the article came about
because I'd see these 15 word questions ("I need to monitor a power
line, so I can sample it at 120Hz, right?"). Unfortunately, when I
tried to answer them I'd get up to a few hundred words I'd give up,
because there's just too dang much background that needs to be given --
so now I can just point to the article, maybe with a "see section X.x",
and cover the question.

As far as pdf documents go -- I have been doing them in HTML out of fear
that folks will just print out the PDF document and never come back to
my web site. It's a royal PITA. Now I'm thinking that maybe I'll be
better off to go ahead and print them as PDF documents, with embedded
hyperlinks and all that cool stuff so that folks can print it out and
_still_ get pointed back to my website.

Hmm.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
J

Jerry Avins

Jan 1, 1970
0
Robert said:
["Followup-To:" header set to sci.electronics.design.]
On Fri, 29 Sep 2006 22:35:46 GMT,
in Msg. said:
In the first sentence of your papere, you say that if the signal is band
limited to fo or less, then a sample frequency of 2fo or more is adequate to
contain completely all the information necessary to recreate the signal. My
understanding from water cooler conservation is that the signal bandwidth must
be strictly less than fo for a sample frequency of 2fo and that the signal
must be infinite in extent to allow perfect reconstruction.

I see it that way as well. A pure sine signal of exactly f_0, sampled at
exactly 2f_0, might come out as all-zero, or as a pulse train with
alternating polarity and an amplitude of anything between 0 and V_p.
There is not enough information to reconstruct the signal in this case.

However, if you sample the same signal at 2f_0+f_e (f_e being very
small, think epsilon), it will come out as a pulse train with
alternating polarity and the amplitude modulated by a beat frequency
f_e. This would seem to be not enough information to reconstruct the
signal, but in fact that's not true: A f_0 sine is the only possible
input signal because the modulated pulse train contains frequency
components greater than f_0 which couldn't have been in the input signal
(which, as a prerequsitite to Nyquist, is brick-walled at f_0).

So, the way I see it is that the sampling frequency must be strictly
greater than the highest frequency in the input.

You're correct, but there's more to it than that. Strictly greater is
necessary for reconstruction, but equal is good enough to avoid
aliasing. Moreover, while it's true that sampling at 2f_0 + f_e allows
reconstruction, you need a significant fraction of 1/f_e seconds of
signal in order to do it. It's the same as trying to distinguish f_e/2
from DC.

Jerry
 
P

PeteS

Jan 1, 1970
0
Tim said:
Robert said:
["Followup-To:" header set to sci.electronics.design.]
On Sat, 30 Sep 2006 09:22:43 -0700,
in Msg. said:
I have to say that this paper rather turned into a monster while I was
writing it. I thought it was going to be between 2000 and 3000 words,
with almost no math and very little real work. Instead it's about 5500
words, and I've got about a man-week into all those pretty charts and
graphs (I should publish an appendix with "the making of..." along with
all the math underneath).

As a reaction to this I haven't done my usual stage of letting it rest
and getting back to it -- I was afraid I'd never do the "getting back to
it" step. Instead I've put it outside without giving it time to get
it's coat and boots on. If I can figure out how to tighten it up I
certainly will -- assuming that I don't run away screaming at the
thought of doing even _more_ work on it.


Let me tell you that this article is so high-class from an educational
point of view that it /deserves/ coat and boots. Next time anybody comes
up to me and wants something explained about Nyquist I can just point
them at your page, and I'm sure many others will do likewise.

The only thing I'd want is the whole thing as a pretty, nicely printable
PDF document. But don't listen to me.

robert

Thanks for the comment. The motivation for the article came about
because I'd see these 15 word questions ("I need to monitor a power
line, so I can sample it at 120Hz, right?"). Unfortunately, when I
tried to answer them I'd get up to a few hundred words I'd give up,
because there's just too dang much background that needs to be given --
so now I can just point to the article, maybe with a "see section X.x",
and cover the question.

As far as pdf documents go -- I have been doing them in HTML out of fear
that folks will just print out the PDF document and never come back to
my web site. It's a royal PITA. Now I'm thinking that maybe I'll be
better off to go ahead and print them as PDF documents, with embedded
hyperlinks and all that cool stuff so that folks can print it out and
_still_ get pointed back to my website.

Hmm.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

I read the page a couple of times, and I like it. Apart from some very
minor readability issues it gets to the meat of the subject without
hashing it to death mathematically - I've seen the comment about
possibly removing all the formulae, but I am not sure it could be done.

As a practical example (that I am sure most here have seen), I'll see
if I can dig out a picture I took of a digital sampling scope with
terrible aliasing in certain frequency / resolution bands (i.e. the
problem would only show up at certain timebase settings) which can be
very confusing to the beginner - certainly it's a known issue with
sampling scopes.

So kudos to you for finishing it off.

Cheers

PeteS
 
Ban said:
There are differences between analog and digital filters. Digital means an
approximation of the desired characteristic in the passband, but the poles
and zeros are modified to compensate for the modulation effects.

Modulation effect? All you have are two different domains (s and z).

aliasing is
always happening, but it can be reduced below the noise floor with the
analog input filter.
Linear phase has a very undesirable side effect, it rings *before and after*
the step, supposed to be more audible.

Ring before the signal arrives? That sound non-causal to me.
 
Mike said:
Ban,

Thanks for the clarification. In most references, the Bessel is considered
to have low, negligible, or no overshoot, especially when compared to
Butterworth, Chebyshev and other types of filters. Your numbers confirm
this.

In practise, it is difficult to obtain the exact component values needed
for the theoretical performance. Not only are the values non-standard, but
it may be difficult to get the inductor "Q" values used in most
calculations. So we can assume there will be some deviation from the
theoretical performance, and the overshoot will probably increase slightly.
However, it is still low enough to be difficult to measure, and the terms
"low", "neglible" or "no overshoot" are quite descriptive.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm

For audio signal processing, the filters are generally active, so no
inductors are used. [speaker crossovers excepted.]

In many applications, ringing cannot be tolerated. Scales for instance.
 
M

martin griffith

Jan 1, 1970
0
Ring before the signal arrives? That sound non-causal to me.
ISTR Roger Lagadec at Studer (and Sony) pointing this out in the early
1980's, after listening tests on early digital systems with piano
music showed that something was amiss. Probably in AES and IEEE
archives.




martin
 
T

Tim Wescott

Jan 1, 1970
0
Modulation effect? All you have are two different domains (s and z).

aliasing is



Ring before the signal arrives? That sound non-causal to me.
Ring before the main part of the signal arrives. FIR filter responses
are often shown as if zero delay were the middle of the filter,
effectively subtracting out the constant delay added by the filter.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
C

CBFalconer

Jan 1, 1970
0
Ban wrote:
.... snip ...

Ring before the signal arrives? That sound non-causal to me.

Don't think of a single signal. An impulse (or a step) has a wide
range of frequency components. Some are delayed more than others
when passing through the filter.

--
Some informative links:
<<http://www.geocities.com/nnqweb/>
<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html>
<http://www.netmeister.org/news/learn2quote.html>
<http://cfaj.freeshell.org/google/>
 
J

Jerry Avins

Jan 1, 1970
0
Ring before the signal arrives? That sound non-causal to me.

Please read more carefully. The filter rings before the main part of the
output step *emerges* but after the step arrives at the input. The
filter's inherent delay makes that quite possible.

Jerry
 
S

Steve Underwood

Jan 1, 1970
0
Ring before the signal arrives? That sound non-causal to me.

Its only non-causal if it arrives before it was sent. :)

There is no reason why the main signal should not arrive latter than
some of the crud which accompanies it. Its perfectly normal in a
dispersive medium.

Steve
 
S

Steve Underwood

Jan 1, 1970
0
Hi Tim,

Tim said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

May I ask what software you used to render the maths on that page? It
looks clearer than the stuff I produce. MathML is getting into browsers
now, but the rendering of that looks so bad with anything I have tried,
that inserted images in HTML pages still seems the only practical approach.

I'd still like to see a web page I can point people to when they say a
10kHz sine wave on a CD will come out as a square wave/triangular
wave/some other weird notion.

Steve
 
Jerry said:
Please read more carefully. The filter rings before the main part of the
output step *emerges* but after the step arrives at the input. The
filter's inherent delay makes that quite possible.

Jerry
--
"The rights of the best of men are secured only as the
rights of the vilest and most abhorrent are protected."
- Chief Justice Charles Evans Hughes, 1927
¯¯¯¯¯

¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Here is the line verbatim:
"Linear phase has a very undesirable side effect, it rings *before and
after*
the step, supposed to be more audible."

Nothing wrong with my reading. Now if you are somehow looking at the
output to interpret where the large transition occurred, that is a
different story. However, any filter where the impulse response goes
negative will have such ringing, be it linear phase or not. You need
to visualize the convolution.
 
T

Tim Wescott

Jan 1, 1970
0
Steve said:
Hi Tim,



May I ask what software you used to render the maths on that page? It
looks clearer than the stuff I produce. MathML is getting into browsers
now, but the rendering of that looks so bad with anything I have tried,
that inserted images in HTML pages still seems the only practical approach.
SciLab

I'd still like to see a web page I can point people to when they say a
10kHz sine wave on a CD will come out as a square wave/triangular
wave/some other weird notion.

Well, get writing!

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
B

Ban

Jan 1, 1970
0
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Here is the line verbatim:
"Linear phase has a very undesirable side effect, it rings *before and
after*
the step, supposed to be more audible."

Nothing wrong with my reading. Now if you are somehow looking at the
output to interpret where the large transition occurred, that is a
different story. However, any filter where the impulse response goes
negative will have such ringing, be it linear phase or not. You need
to visualize the convolution.

It's called *pre-ringing* and it appears because the chunks are processed
forward and backward in a row, so a unity pulse will have an identical
rising and falling edge. If the filter is of the ringing type, thus the
ringing occurrs twice.
You are right in saying it's impossible, but only in an analog world.
Digital filters do have a latency which will always be longer than the delay
of the corresponding analog filter; with linear phase it will be twice the
FIR size plus twice the conversion time and more than double than the analog
counterpart.
Well done analog filters are of the *minimum phase* type, having just the
lowest possible delay for that shape of output response. This is possible to
realize digitally with IIR filters only.
And do not think that even a Gauss filter has only positive
FIR-coefficients. This would be only true for a filter of infinite length,
which apparently isn't that desirable at all. For practicable sizes the
location of the poles and zeros has to be modified and one might get even
negative coefficients, depending on the ratio of sampling- and filter
frequency and filter length.
 
Ban said:
It's called *pre-ringing* and it appears because the chunks are processed
forward and backward in a row, so a unity pulse will have an identical
rising and falling edge. If the filter is of the ringing type, thus the
ringing occurrs twice.
You are right in saying it's impossible, but only in an analog world.
Digital filters do have a latency which will always be longer than the delay
of the corresponding analog filter; with linear phase it will be twice the
FIR size plus twice the conversion time and more than double than the analog
counterpart.
Well done analog filters are of the *minimum phase* type, having just the
lowest possible delay for that shape of output response. This is possibleto
realize digitally with IIR filters only.
And do not think that even a Gauss filter has only positive
FIR-coefficients. This would be only true for a filter of infinite length,
which apparently isn't that desirable at all. For practicable sizes the
location of the poles and zeros has to be modified and one might get even
negative coefficients, depending on the ratio of sampling- and filter
frequency and filter length.

The Gaussian to which I refer is S domain. If you mapped it to Z
domain, it would have to be IIR, not FIR.
 
J

Jeroen Belleman

Jan 1, 1970
0
Ban said:
It's called *pre-ringing* and it appears because the chunks are processed
forward and backward in a row, so a unity pulse will have an identical
rising and falling edge. If the filter is of the ringing type, thus the
ringing occurrs twice.

Even steep linear phase analogue filters will exhibit pre-ringing.
If you were to linearise the phase response of, say, a Butterworth
filter, by adding one or more all-pass sections, its impulse
repsonse will ring before and after the main output. Of course,
the overall delay must go up for the filter to remain causal.

Jeroen Belleman
 
G

glen herrmannsfeldt

Jan 1, 1970
0
Joerg wrote:
(someone wrote)
Nyquist published his paper about the minimum required sample rate in
1928. Shannon was a kid of 12 years back then. The paper wasn't about
ADCs or sampling in today's sense but about how many pulses per second
could be passed through a telegraph channel of a given bandwidth.
(and be distinguished on the other end).

The important point being that the math is the same even though the
goal is different. I suppose, then, the sample rate should be
a lemma to Nyquist's telegraph channel theorem.

By the way, Gauss published the first paper on the FFT.

-- glen
 
J

Jerry Avins

Jan 1, 1970
0
Ban wrote:

...
Well done analog filters are of the *minimum phase* type, having just the
lowest possible delay for that shape of output response. This is possible to
realize digitally with IIR filters only.

Minimum-phase (or nearly minimum) FIRs are possible, just not symmetric
FIRs. You can make maximum-phase FIRs too. Then *all* the ringing is on
the leading edge.

Jerry
 
Top