Searching \ for 'Don't use Microchip's FFT' in subject line. ()
Make payments with PayPal - it's fast, free and secure! Help us get a faster server
FAQ page: www.piclist.com/techref/microchip/math/filter.htm?key=fft
Search entire site for: 'Don't use Microchip's FFT'.

Truncated match.
PICList Thread
'Don't use Microchip's FFT'
2000\03\30@095254 by Steve Thackery

flavicon
face
*No* disrespect to the guys who wrote it, but it doesn't work.

Over the last several months I've been developing a product for my
employer which uses a fast fourier transform as part of its functionality.
The product is based upon a 17C44, by the way.

I was foolish enough to plan my timescales assuming that the Microchip FFT
would be pretty well "plug and play".  After all, I do tend to trust the
stuff they publish.  How wrong I was!

I won't bore you with the details, but Microchip's FFT is seriously
defective.  I can only guess that it didn't receive much testing.  In
fact, reading between the lines in AN542 it becomes clear that it was
primarily written as a *benchmarking* exercise rather than a reliable FFT
module for use in our projects.

Basically, if you test it with the waveform shown in the AN, it works
perfectly.  I should have been suspicous, even so.  The waveform shown is
of very low frequency, and has a whole number of cycles in the sampling
window.  Alter that frequency by even a small amount, so that there isn't
a whole number of cycles in the window and the whole thing goes haywire!
It generates artefacts in the FFT output all over the place, both above
and below the "real" frequency.  Similarly, test it at higher frequencies
with sine or square waves, and you get a complete spectrum of rubbish at
the output.

Incidentally, it wasn't made clear in the AN that permissible input range
of values is between 0 and 3FFF.  In other words, you need to DC-shift
your sound sample so its centred around 1FFF, and adjust its gain.  That
caught us out for a couple of days!

We assumed it must be something we were doing wrong and spent ages poring
over the AN to see if we were doing something stupid.  We also tried
working through the code itself, but it is almost inpenetrable.  We tried
everything, including using very small amplitude signals, but to no avail.
The output is rubbish *except* with very low frequency square waves with a
whole number of cycles in the window.

I can only imagine that there is a scaling problem somewhere in the maths.
It looks like there is some sort of overflow or fold-back going on.

This cost me two months of development time and made me very unpopular
with my employer and customer!

In the end we gave up and looked elsewhere.  This is where Robert Lacoste
comes into the picture.  He developed his own FFT module from scratch and
incorporated it into his PIC'Spectrum product.  I contacted him and asked
if he would be willing to let us use his FFT module - for prototyping and
test purposes only - in our own product.  He was willing and incredibly
helpful, answering all our questions with clarity.

It took us two days to unplug the Microchip FFT module and plug in the
Lacoste module.  Here's what happened.  It worked PERFECTLY!  The output
is classic, text-book perfect under all circumstances.  Furthermore, it
runs at least ten times as fast as the Microchip module.

Let me state publicly my most grateful thanks to Robert.  If you need to
include an FFT in any of your products, contact him on
spam_OUTrobert_lacosteTakeThisOuTspamyahoo.fr  He allowed us free use in our prototypes, and
suggested a very reasonable royalty on our production models.  I expect he
might extend a similar deal to others.

In summary, then: BEWARE the Microchip FFT, and THANK YOU Robert Lacoste!

Steve

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\03\30@113431 by Dan Michaels

flavicon
face
At 01:06 PM 3/30/00 +0100, you wrote:
.....
>Over the last several months I've been developing a product for my
>employer which uses a fast fourier transform as part of its functionality.
>The product is based upon a 17C44, by the way.
....
>In the end we gave up and looked elsewhere.  This is where Robert Lacoste
>comes into the picture.  He developed his own FFT module from scratch and
>incorporated it into his PIC'Spectrum product.  I contacted him and asked
.....
>Steve Thackery
>Suffolk, England.
>Web Site: http://www.btinternet.com/~stevethack/
>

Among other things, the pittance of addressable RAM in PICs [and
Scenix] would appear to severely limit their capability to do
serious DSP.

How large an FFT were you finally able to get to work?
Does Lacoste have a website?

best regards,
- Dan Michaels

2000\03\30@160935 by Steve Thackery

flavicon
face
Dan Michaels asked

> How large an FFT were you finally able to get to work?
> Does Lacoste have a website?

Robert's FFT is set up to do 256 samples only.  However, I believe it would be
simple to extend it to do higher or lower powers of 2.  I've considered asking
him if he would be willing to make the alterations to his code, but I don't
want to prevail upon him to do stuff just as a favour to me.

I suggest you email him if you are seriously interested.

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\03\31@032152 by Robert Lacoste

flavicon
face
>Among other things, the pittance of addressable RAM
in >PICs [and
>Scenix] would appear to severely limit their
capability to do
>serious DSP.
>
>How large an FFT were you finally able to get to
work?
>Does Lacoste have a website?
>
>best regards,
>- Dan Michaels

Hi everybody,
First thank you very much Steve for your fantastic
feedback.
Dan, regarding RAM size, my code was originaly
developped for the 17C756 (>900 bytes of RAM) and was
adequate for a 128 words FFT, calculated in 50ms.
Could be of course extended with external RAM.
And yes I have a (very small) web site. My FFT code is
not available there, but you will find my freeware
17C756 programmer : //http://www.geocities.com/robert_lacoste

Cheers,
Robert.

___________________________________________________________
Do You Yahoo!?
Achetez, vendez! @ votre prix! Sur http://encheres.yahoo.fr

2000\03\31@113118 by Dan Michaels

flavicon
face
Robert Lacoste wrote:
.....
>Dan, regarding RAM size, my code was originaly
>developped for the 17C756 (>900 bytes of RAM) and was
>adequate for a 128 words FFT, calculated in 50ms.
>Could be of course extended with external RAM.
>And yes I have a (very small) web site. My FFT code is
>not available there, but you will find my freeware
>17C756 programmer : //http://www.geocities.com/robert_lacoste
>
>Cheers,
>Robert.
>

Robert, thanks for responding.

Let's see, 128 bytes/input, plus 2 x [256 for real/imaginary]
= 640 bytes, or 768 if 12-bit input data. Doesn't leave any
left for double buffering or output buffering, if you wanted
to do real-time pass-through. And this on a 68-pin, $16 chip
with hardware multiplier. Guess that's why I never tried it
on a PIC before [but always wanted to].

However, the new 18C252 is faster, cheaper, with fewer pins
(28-p) and more RAM with linear addressing. With 1536 RAM bytes,
you can just squeeze in a 256 FFT: 256 + 2*[256+256] = 1280
bytes, if 8-bit input data. Probably could not utilize the full
10-bits of the '252 A/D here. But could probably do 128 FFT with
real-time double-buffering, and pass-thru.

Now, we need someone with a freeware 18C252/452 programmer, and
DSP code library personalized for it.

Thanks much for the info, and best regards,
- Dan Michaels
Oricom Technologies
http://www.sni.net/~oricom
==========================


'Don't use Microchip's FFT'
2000\04\01@102944 by Scott Dattalo
face
flavicon
face
On Fri, 31 Mar 2000, Dan Michaels wrote:

>
> Let's see, 128 bytes/input, plus 2 x [256 for real/imaginary]
> = 640 bytes, or 768 if 12-bit input data. Doesn't leave any
> left for double buffering or output buffering, if you wanted
> to do real-time pass-through. And this on a 68-pin, $16 chip
> with hardware multiplier. Guess that's why I never tried it
> on a PIC before [but always wanted to].

I haven't seen Robert's code, but whenever I've needed to do an embedded FFT
(which I've never needed to do in a pic), I've used the Discrete Hartley
transform and more specifically, the Fast Hartley Transform or FHT. Now it
really does depend on your application. Depending on the situation, the real FFT
as (opposed to the complex FFT) can be faster than an FHT. However, if you're
interested in the power at each freqeuncy, the FHT is faster. What's nice about
the FHT too is that for one additional pass through the processed data, you can
obtain the complex Fourier coefficients!

Furthermore, the FHT does not require a complex kernel. Thus the memory
requirements are half of the FFT's (although, a clever manipulation of indices
can reduce the kernel memory requirements for the FFT by 1/2 - and if you're
really clever, you can take additional advantages of trigonometric identies to
reduce this again by 1/4 (easy) or 1/8 (slightly harder).

But I have to ask, why would one need to compute an FFT in a PIC?

Scott

2000\04\01@122536 by Dan Michaels

flavicon
face
Scott Dattalo wrote:
>
>I haven't seen Robert's code, but whenever I've needed to do an embedded FFT
>(which I've never needed to do in a pic), I've used the Discrete Hartley
>transform and more specifically, the Fast Hartley Transform or FHT. Now it
>really does depend on your application. Depending on the situation, the
real FFT
>as (opposed to the complex FFT) can be faster than an FHT. However, if you're
>interested in the power at each freqeuncy, the FHT is faster. What's nice about
>the FHT too is that for one additional pass through the processed data, you can
>obtain the complex Fourier coefficients!
>

Well, if the only tool you have is a hammer (ie, FFT), then .....
Unfortunately, cinder-block inertia dictates the way some of us attack
problems. Never tried FHT, but will have to look into it.
===========

>
>Furthermore, the FHT does not require a complex kernel. Thus the memory
>requirements are half of the FFT's (although, a clever manipulation of indices
>can reduce the kernel memory requirements for the FFT by 1/2 - and if you're
>really clever, you can take additional advantages of trigonometric identies to
>reduce this again by 1/4 (easy) or 1/8 (slightly harder).
>

Off-line, Robert said:

       The main algorithm is a complex radix-1 FFT with 16
       bits calculation (16 bits real+16 bits imaginary), 128
       input values / 128 frequencies (so 512 bytes of
       input/output buffer).

       I've developped also a surounding "real mode FFT"
       algorithm, packing two real samples per complex value
       and "descrambling" the resulting complex FFT. So I got
       in fact a 256 points 16 bits real FFT.

       The 50ms figure is for the resulting real mode FFT
       (complex FFT + some overhead).

So Robert had already taken some of the optimizing methods into account.
Obviously, doing 16-bit arithmetic on an 8-bit cpu is a major setback
with PICs, even with h.w. multiplier.

[OT] For anyone interested there is an old book called "DFT/FFT and
Convolution Algorithms" by C.S. Burrus and T.W. Parks, Wiley 1985,
that discusses in detail ways to trim fat off FFT cycles, although
the discussion is geared to TMS320.
==============

>But I have to ask, why would one need to compute an FFT in a PIC?
>Scott
>

Good question. One could always use an actual DSP chip, but PICs
are smaller, cheaper, more convenient, known territory, easy to
interface, plus you can add it to on-hand embedded systems, also
for fun, wheel spinning, doing things the hard way, tacking on
recursive costs for the boss [Wouter's favorite], on and on.
Also, perhaps most important, trying to figure out how far you
can push a PIC. For future reference.

I never actually tried it before on a PIC, mainly because of too
little RAM and too slow. Scenix could burn rubber, but the RAM
problem is horrific (enough for maybe 16-pt FFT). But with the
new 18C242, with 10 MIPs, 28-pin, 10-bit A/D, 1536 bytes RAM,
linear RAM addressing, and h.w. multiplier, it might be more
feasible. [if it only had a DAC].

best regards,
- Dan Michaels
==============

2000\04\01@171445 by Steve Thackery

flavicon
face
Scott Dattalo asks:

> But I have to ask, why would one need to compute an FFT in a PIC?

Why not?  For the same reason you have to compute it in any other micro,
surely?  Am I missing out on an alternative?

I can't give full details of the project I'm working on, but in overview it's
a tool for testing decay in telegraph poles by analysing the vibration
spectrum from the pole when it is stimulated with an impulse.  The FFT is one
stage in a fairly complex algorithm which produces a good/decayed indication.
The algorithm was developed on a PC using Delphi, and then ported into a
low-cost hand-held device.  Hence the use of a PIC.

I'm not being sarcastic when I said "Why not?" above.  Would you have used
different approach?  You are clearly much more experienced than I and I'd love
to learn!

Best wishes,

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\04\01@185805 by Sean Breheny

face picon face
Hi Steve,

I think that Scott is saying that there is a real temptation to go for a
"full" version of an algorithm when either a partial algorithm or something
totally different would suffice.

For example, let's say that I want to determine signal power in 7 narrow
bands. If I did this with an FFT, and ran it on 1024-sample blocks, I would
actually get the amplitude in 1024 bands, and I would have to select out
the 7 bands I wanted. Lots of computation and RAM. Instead, I could use
Goertzel's Algorithm (a method of computing an FFT for only a few
frequencies) to get only the information I need. In addition, it requires
significantly less RAM than a full FFT.

Another option, if you want to analyze signal power in large bands, would
be to use several band-pass filter algorithms (optimized to share commonly
computed data), which (I think) would take less computing power than an FFT.

So, in your case, do you actually analyize the WHOLE spectrum, or only
portions of it? I think it is rare in a microcontroller application where
one needs the full amount of information that an FFT gives you.

Sean

At 05:59 PM 4/1/00 +0100, you wrote:
{Quote hidden}

love
>to learn!
>
>Best wishes,
>
>Steve Thackery
>Suffolk, England.
>Web Site: http://www.btinternet.com/~stevethack/
>
|
| Sean Breheny
| Amateur Radio Callsign: KA3YXM
| Electrical Engineering Student
\--------------=----------------
Save lives, please look at http://www.all.org
Personal page: http://www.people.cornell.edu/pages/shb7
.....shb7KILLspamspam@spam@cornell.edu ICQ #: 3329174

2000\04\02@045824 by Steve Thackery

flavicon
face
Sean writes:

> So, in your case, do you actually analyize the WHOLE spectrum, or only
> portions of it? I think it is rare in a microcontroller application where
> one needs the full amount of information that an FFT gives you.

Actually I do use the whole spectrum, in my case.  However, the points you
explain are well made and I appreciate that very much.  I didn't know about
Goertzel's Algorithm or the existence of band-pass filter algorithms.

Thanks for being gentle with a newbie!

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\04\02@101447 by Scott Dattalo

face
flavicon
face
On Sat, 1 Apr 2000, Sean Breheny wrote:

{Quote hidden}

Sean, you hit the head right on the nail. Let me give a few more examples...

First, Sean alludes to the Goertzel algorithm. This is nothing more than an
efficient way to compute the Discrete Fourier Transform (DFT). However, I'd be
careful using the Goertzel algorithm because it is subject to accumulative
roundoff errors.

If you have 2^N samples and wish to run these through an FFT then you can
resolve these into 2^N (or perhaps 2^(N-1)) frequencies. If you're only
interested in N or less of those frequency components then in general it's more
efficient to compute the N DFT's. The DFT's can be thought of as narrow band
filters (specifically FIR or finite impulse response filters) that pluck out
those frequencies of interest.  Now this is just a rule-of-thumb approximation
based on the computational complexity of the two algorithms. If you're
interested in the narrow frequency bands then there are more efficient ways then
a DFT to go about obtaining them (such as an IIR filter).

Often times, like your's Steve, the interest is in bands of frquencies. If the
frequency bands are evenly spaced, then an FFT algorithm can still be used.
However, most often the banding is logarithmic. I recall seeing an FFT-type
algorithm that will do this logarithmic banding, but I don't recall the details.
A better approach in my opinion, is to implement filter banks for decomposing
the signal into arbitrary frequency bands. In the worst case, you can design N
band pass filters and execute each in succession. A much more difficult but also
much more efficient approach is to use Multirate Signal Processing and something
called Polyphase Decomposition. In conceptual terms, this approach takes
advantage of the results of one filter in implementing another filter. For
example, suppose you have four freqeuncy bands. You can divide these into two
low bands and into two high bands. This first split is implemented with one
filter. Now if you further split the low frequency band into two more bands then
you can take advantage of the fact that much of the filtering has already been
performed. In addition, this requires only one filter. The same is true for
splitting the two high bands. In all, there are three filters that are required
as opposed to four if you took a brute force approach.


What's good about this approach is that the signal can be split into arbitrary
frequency bands fairly efficiently. What's bad about it is that it's rather
difficult to design (although fairly straight forward to implement).

A decent book on the subject is "Advanced Digital Signal Processing" by Zelniker
and Taylor, ISBN 0-8247-9145-2 .

As a final comment, it's been my experience that FFT's are primarily used to
provide a human a quantitative representation of the frequency content of a
signal. In other words, you do an FFT so you can show the results to someone. I
haven't run across an algorithm that depended on or utilized an FFT.  (That's
not to say there isn't one.)


I hope this doesn't totally confuse your design decisions. If you've got
something that's working with the FFT and your boss and (more importantly) your
customers are happy then by all means stick with the FFT! But for the next time,
you've got another tool you can potentially use.

Scott

2000\04\02@103807 by Alan B Pearce

face picon face
>I can't give full details of the project I'm working on, but in overview it's
>a tool for testing decay in telegraph poles by analysing the vibration
>spectrum from the pole when it is stimulated with an impulse.  The FFT is one

Is it not a case where you just record the sound and analyse it later in the
office? does it really need to be analysed on site. I would have thought a sound
recording with pole ID# would have been sufficient. with high performance
analyses later.

2000\04\02@112239 by Steve Thackery

flavicon
face
Alan writes:

> Is it not a case where you just record the sound and analyse it later in the
> office? does it really need to be analysed on site. I would have thought a
sound
> recording with pole ID# would have been sufficient. with high performance
> analyses later.

Ooops, we're drifting off topic here!  Even so, I'm sure no-one will mind a
brief reply.  Basically, yes, it could work that way.  At the moment our pole
testers test several poles a day, marking the defective ones appropriately,
and the next day they move on and test another batch.  Off-line processing
would require a change to this working practice, with the likelihood of having
to return to some of the poles the next day to mark them defective.

This wouldn't be much of a problem if it should prove necessary.  But you know
how resistant workers and unions can be to change!  If we can achieve it, we
simply want to provide the testers with another tool in their toolkit to help
them test a pole better.  If we can't achieve that, perhaps because of the
complexities of the signal processing, then maybe we could go to something
like you suggest.

Thanks for all your comments, folks.

Steve

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\04\02@112243 by Steve Thackery

flavicon
face
Scott,

Thank you for such a helpful contribution to the debate.  This sort of thing
is what makes the PICLIST so damn good.

Magic!  And thanks again.

Steve

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\04\02@114803 by Alan B Pearce

face picon face
>> Is it not a case where you just record the sound and analyse it later in the
>> office? does it really need to be analysed on site. I would have thought a
sound
>> recording with pole ID# would have been sufficient. with high performance
>> analyses later.

>Ooops, we're drifting off topic here!  Even so, I'm sure no-one will mind a
>brief reply.  Basically, yes, it could work that way.  At the moment our pole
>testers test several poles a day, marking the defective ones appropriately,

Well I was assuning you would use a PIC for the data collection. However I can
now see why you would want to do it the way you propose. I would also guess that
it is a case of not needing to make it absolutely micro miniature as it could be
connected to a car battery in 99.9999% of cases, at least for recharging most of
the time. from this I would take it that you could happily use the PIC with the
largest RAM etc.

Just how accurate does the FFT need to be? I do not have expertise in this area,
but if it was to show as a "bargraph" on a graphics LCD (spectrum analyser
style) does this save you anything? I assume that the good/bad decision would be
made by someone looking at the picture to decide if the "out of band"
resonance's/artefacts are sufficient to condemn the pole.

2000\04\02@123446 by Dan Michaels

flavicon
face
Steve Thackery wrote:
>>
>> But I have to ask, why would one need to compute an FFT in a PIC?
>
>Why not?  For the same reason you have to compute it in any other micro,
>surely?  Am I missing out on an alternative?
>
>I can't give full details of the project I'm working on, but in overview it's
>a tool for testing decay in telegraph poles by analysing the vibration
>spectrum from the pole when it is stimulated with an impulse.  The FFT is one
>stage in a fairly complex algorithm which produces a good/decayed indication.
>The algorithm was developed on a PC using Delphi, and then ported into a
>low-cost hand-held device.  Hence the use of a PIC.
>

Steve,

For the particular application you describe, you might actually be
farther ahead using a small notebook PC, possibly even a palmheld
using the (dreaded Microsoft) WindowsCE OS, with an A/D module
attached. The PC can perform all the complex algorithm processing
you will ever require, plus be able to permanently save data on HD
from 1000s of telephone poles for later analysis/archiving.

Plus, you can write s.w. in a high-level language on the PC that
will execute many times faster than highly-optimized assembler on
a PIC.

Seriously, the PIC may not be the best approach in this case. Its
limited internal RAM and inability to easily address external RAM,
plus its marginal computational power for DSP, are a serious
drawback to its use here.

best regards,
- Dan Michaels

2000\04\02@134447 by picxpert

picon face
I'd suggest not using CE, but a PalmPilot. They can be had for really cheap,
and last a good long while on a pair of AAAs. Plus, they have an
oh-so-easy-to-use serial port, which is, from what I hear, well documented
in the API docs. Plus, what A/D capable PIC doesn't have a UART? You could
use a PIC to interface with sensors, and then send off the data to the Palm
to do processing (which is what I think Mr. Michaels was suggesting.)

My observation is that Windows CE / Palm PC are only around because
Microsoft is too stubborn to discontinue support for it. Palm Computing
holds the lion's share of the market - they have the brightest future, and
perhaps some of the cheapest handhelds (A Palm IIIe is only $149US!)

-Randy Glenn
PICxpertspamKILLspamtechie.com - http://i.am/PICxpert

Ineptitude is a sure indicator of intellect - except in the case of @Home
techs.

===========
To unsubscribe, send a message containing the text "unsubscribe PICLIST" to
.....LISTSERVKILLspamspam.....MITVMA.MIT.EDU

{Original Message removed}

2000\04\02@154952 by Steve Thackery

flavicon
face
Alan asks:

> Well I was assuning you would use a PIC for the data collection. However I
can
> now see why you would want to do it the way you propose. I would also guess
that
> it is a case of not needing to make it absolutely micro miniature as it
could be
> connected to a car battery in 99.9999% of cases, at least for recharging
most of
> the time. from this I would take it that you could happily use the PIC with
the
> largest RAM etc.

That's right.  It's a small hand-held box that plugs into the car's cigarette
lighter to recharge the battery.  We use a 17C44 with 64K of external RAM.
Once you've set up the macros/subroutines for talking to the external RAM it's
no problem at all.

> Just how accurate does the FFT need to be? I do not have expertise in this
area,
> but if it was to show as a "bargraph" on a graphics LCD (spectrum analyser
> style) does this save you anything? I assume that the good/bad decision
would be
> made by someone looking at the picture to decide if the "out of band"
> resonance's/artefacts are sufficient to condemn the pole.

Highly accurate, unfortunately.  It does not show the spectrum to the user.
Instead a complicated algorithm is run on the spectrum which weights certain
frequency bands, applies a statistical function to the weighted spectrum, and
extracts from that a good/bad indication which is given to the user on a
simple text display.  I'm not allowed to give details about that bit because
it's the subject of a patent application and my employer would have my balls
for a bowtie!!

Steve Thackery
Suffolk, England.
Web Site: http://www.btinternet.com/~stevethack/

2000\04\02@162452 by Dan Michaels

flavicon
face
Randy Glenn wrote:
>I'd suggest not using CE, but a PalmPilot. They can be had for really cheap,
>and last a good long while on a pair of AAAs. Plus, they have an
>oh-so-easy-to-use serial port, which is, from what I hear, well documented
>in the API docs. Plus, what A/D capable PIC doesn't have a UART? You could
>use a PIC to interface with sensors, and then send off the data to the Palm
>to do processing (which is what I think Mr. Michaels was suggesting.)
>
>My observation is that Windows CE / Palm PC are only around because
>Microsoft is too stubborn to discontinue support for it. Palm Computing
>holds the lion's share of the market - they have the brightest future, and
>perhaps some of the cheapest handhelds (A Palm IIIe is only $149US!)
>

Palm Pilot would be fine too, just as long as you could code a
hi-speed FFT, along with other serious DSP algorithms on it, store
MBytes worth of data, display hi resolution data at 10-15 screen
updates/sec, and easily transfer over to a PC for archiving and
further processing. It would also be nice if the Palm executable
were interchangeable with the PC, for obvious reasons. I had no
idea the Palm was built for such serious number crunching
- [but good to know].

Actually, if I were attempting the original problem, I'd try
a notebook or mini-notebook as first choice, and later look at
porting it to a palmheld. It's just so much easier to develop
DSP algorithms in C (for example) on a PC, with its math coP
and nice graphics. [at least it used to be before Windows].

2000\04\03@105337 by Don Hyde

flavicon
face
I've been through this one, with power meters as one instance.  Sending
someone out to the pole is expensive.  In the case of power meters, it costs
more than the meter itself.

If you save a trip out, especially one that involves a several-man crew in
an expensive specialized vehicle, you can save at least $50.

The big payoff is if you can have a handheld gizmo that tells you right now,
while you're already there with the right stuff, whether you should replace
that pole or not.  We're already here for something else, let's thump those
old poles and see if we can save another trip later.

> {Original Message removed}

2000\04\03@124145 by Tom Handley

picon face
  Scott, thanks for the excellent insight. I have a PIC-based DAQ project
as well as an `on-going' Logic Analyzer/DSO project. I've been very
interested in this subject as I will want to implement FFTs and a spectrum
analyzer amongst other things. I'm working from two old books; "The Fourier
Transform and it's Applications" 2nd edition, Ronald Bracewell. And;
"Digital Signal Processing", Alan Oppenheim / Ronald Schafer.

  I would like to get your thoughts on a general purpose spectrum analyzer
where the frequency can vary from a few Hz to 40MHz. I can do it via the
`brute strength and ignorance' method using classic math. Note, my hardware
includes frequency synthesis/stimulus. The hardware is based on a PIC16F877
with a custom CPLD that interfaces to a 512K NVSRAM (Dallas DS1647) with a
RTCC and Lithium Battery. The CPLD provides the SRAM interface as well as
external chip selects and 8 dedicated SPI bus chip selects.

  I should mention that post processing will be done on a PC though I have
some potential applications such as low frequency vibration analysis on
machinery, using an accelerometer, where your comments apply directly to the
problem. In this case, the math would be implemented in a PIC. Thanks,

  - Tom

At 09:13 AM 4/2/00 -0500, Scott Dattalo wrote:
[snip]
>Sean, you hit the head right on the nail. Let me give a few more examples...
>
>First, Sean alludes to the Goertzel algorithm. This is nothing more than an
>efficient way to compute the Discrete Fourier Transform (DFT). However,
I'd be
>careful using the Goertzel algorithm because it is subject to accumulative
>roundoff errors.
>
>If you have 2^N samples and wish to run these through an FFT then you can
>resolve these into 2^N (or perhaps 2^(N-1)) frequencies. If you're only
>interested in N or less of those frequency components then in general it's
more
>efficient to compute the N DFT's. The DFT's can be thought of as narrow band
>filters (specifically FIR or finite impulse response filters) that pluck out
>those frequencies of interest.  Now this is just a rule-of-thumb
approximation
>based on the computational complexity of the two algorithms. If you're
>interested in the narrow frequency bands then there are more efficient
ways then
>a DFT to go about obtaining them (such as an IIR filter).
>
>Often times, like your's Steve, the interest is in bands of frquencies. If
the
>frequency bands are evenly spaced, then an FFT algorithm can still be used.
>However, most often the banding is logarithmic. I recall seeing an FFT-type
>algorithm that will do this logarithmic banding, but I don't recall the
details.
>A better approach in my opinion, is to implement filter banks for decomposing
>the signal into arbitrary frequency bands. In the worst case, you can
design N
>band pass filters and execute each in succession. A much more difficult
but also
>much more efficient approach is to use Multirate Signal Processing and
something
>called Polyphase Decomposition. In conceptual terms, this approach takes
>advantage of the results of one filter in implementing another filter. For
>example, suppose you have four freqeuncy bands. You can divide these into two
>low bands and into two high bands. This first split is implemented with one
>filter. Now if you further split the low frequency band into two more
bands then
>you can take advantage of the fact that much of the filtering has already
been
>performed. In addition, this requires only one filter. The same is true for
>splitting the two high bands. In all, there are three filters that are
required
>as opposed to four if you took a brute force approach.
>
>
>What's good about this approach is that the signal can be split into
arbitrary
>frequency bands fairly efficiently. What's bad about it is that it's rather
>difficult to design (although fairly straight forward to implement).
>
>A decent book on the subject is "Advanced Digital Signal Processing" by
Zelniker
>and Taylor, ISBN 0-8247-9145-2 .
>
>As a final comment, it's been my experience that FFT's are primarily used to
>provide a human a quantitative representation of the frequency content of a
>signal. In other words, you do an FFT so you can show the results to
someone. I
>haven't run across an algorithm that depended on or utilized an FFT.  (That's
>not to say there isn't one.)
>
>
>I hope this doesn't totally confuse your design decisions. If you've got
>something that's working with the FFT and your boss and (more importantly)
your
>customers are happy then by all means stick with the FFT! But for the next
time,
>you've got another tool you can potentially use.
>
>Scott


------------------------------------------------------------------------
Tom Handley
New Age Communications
Since '75 before "New Age" and no one around here is waiting for UFOs ;-)

2000\04\05@085836 by Scott Dattalo

face
flavicon
face
On Mon, 3 Apr 2000, Tom Handley wrote:

{Quote hidden}

Tom,

I know hardly anything about how spectrum analyzers are designed. I'm sure there
are others here to provide better suggestions. But, I can offer a few. First,
one important feature of spectrum analyzers is their huge dynamic range. It's
important in some cases to see frequencies 100dB down or more (relative to some
other signal). This is hard to achieve! I'd go as far as saying that for a pure
digitizing system it's impossible with today's technology. Why? Well a 16-bit
A/D converter can provide only 96dB of dynamic range and these are generally
limited to a few 100khz at best. I suspect that this is the approach taken by
SRS in their low frequency spectrum analyzers (which are limited to about
120khz).

The high frequency spectrum analyzers use a very low noise sweeping oscillator
to down-convert the signal to a lower frequency. You mention synthesis; is this
what you're talking about? Once the signal has been down converted, you can then
use a low pass filter and digital signal processing techniques to analyze it
further.

If you really need a few Hz to 40MHz, then I'd probably try to design a really
low noise programmable (analog) oscillator and a low noise down converter. The
output of this would drive a low pass filter whose attenuation is on the order
of the dynamic range you seek (e.g. if you want 80db then at least a 4th-order
analog filter would be required - a 5th-order filer is just an R and a C more
expensive than a 4th-order one). The cutoff frequency of the analog filter can
more or less be arbitrary, but I'd suggest something below 60Hz just to be
safe. The output of this filter can drive an amplifier and an A/D converter. If
you made the low pass filter with a higher cutoff (say 1khz) then you could use
digital filters to resolve finer frequencies. In other words, you could combine
the best of both worlds (big caveat below)

As an example on how the sweeping oscillator thing works, recall the trig
identity:

 sin(x)sin(y) = [cos(x-y) - cos(x+y)]/2


If your programmable clock can be represented as:

f(t) = A*sin(w1*t+phi1)

and the signal you wish to spectrally examine:

g(t) = B*sin(w2*t+phi2)

then the product

f(t)*g(t) = A*B*cos( (w1-w2)t + phi1-phi2) -
            A*B*cos( (w1+w2)t + phi1+phi2)

The first cosine term is the difference between your clock and the signal that
you're examining. If these two are about the same frequency, then the difference
is close to zero, but their sum is large. If you passed this product through a
low pass filter that had a cutoff greater than w1-w2, then you would be able to
examine g(t). Now if you abstract g(t) to an arbitrary wave, then you'll still
be able to extract the small frequency bin centered at your clock and with a
width of twice your low pass filter.

But you do have to be careful (as always). Suppose that the unknown signal
contains two frequencies equally spaced from your clock (e.g w1 + delta1 and w1
- delta1). These two would get aliased onto one another:

g(t) = B*sin( (w1+delta)t + phi2a) + C*sin( (w1-delta)t + phi2b)

f(t)*g(t) = A*B*cos( (-delta)t + phi1-phi2a) -
            A*B*cos( (2*w1-delta)t + phi1+phi2a) +
            A*C*cos( (delta)t + phi1-phi2b) -
            A*C*cos( (2*w1-delta)t + phi1-phi2b)

and after the carrier get's knocked out by the low pass filter you're left with:

f(t)*g(t) = A*B*cos( (-delta)t + phi1-phi2a) -
            A*C*cos( (delta)t + phi1-phi2b) -

The frequency -delta and delta will generate identical cosine waves. So how do
you differentiate these? Well, maybe this is where our HAM guys can jump in.
Certainly if you made the low pass filter very narrow then you could resolve
them. Now if you step back and look at this from a practical point of view, it's
going to be very difficult to design a high frequency clock with very low
jitter. Consequently, the spectrum analyzer's frequency resolution is a strong
function of the stability of the sweeping oscillator. But as far as devising a
modulation technique to single out only one of these frequencies, I don't have
an answer. There is one observation to be made, however. Suppose we return to
analyzing a single frequency again. As the oscillator sweeps pass this frequency
the demodulated version of the signal will be converted to a sine wave with a
low frequency. Let's say that the low pass filter has a cutoff of 1kHz. If we
looked at the demodulated signal after the low pass filter and let the
oscillator sweep from low to high frequencies then this is what we'd see: First
the sine wave would appear at 1kHz. Then as the oscillator increases in
frequency, the demodulated sine wave would decrease. When the oscillator and the
sine wave are the same frequency then we'd get DC. And finally, as the
oscillator passes the sine wave's frequency, then the demodulated version would
begin to increase again. Perhaps this observation can be put to use?


Scott

2000\04\06@080231 by Tom Handley

picon face
  Scott, thanks for the detailed info. My first instinct is to use a classic
swept-LO design. I have several of the pieces in the existing design but I'm
probably going to scale-back the frequency range. The spectrum analyzer was
an afterthought but I could easily implement a lower-frequency version. I
have 24-Bit A/Ds and a precision wide-band RMS to DC converter. This greatly
simplifies things. I've tested the ML2036 50KHz sinewave generator and I'm
looking at the MAX038 and Analog Devices' DDS chips. I've also looked at
programmable 5th and 8th-order filters for the low-frequency end.

  - Tom

At 07:57 AM 4/5/00 -0500, Scott Dattalo wrote:
{Quote hidden}

the
>> problem. In this case, the math would be implemented in a PIC. Thanks,
>>
>
>Tom,
>
>I know hardly anything about how spectrum analyzers are designed. I'm sure
there
>are others here to provide better suggestions. But, I can offer a few. First,
>one important feature of spectrum analyzers is their huge dynamic range. It's
>important in some cases to see frequencies 100dB down or more (relative to
some
>other signal). This is hard to achieve! I'd go as far as saying that for a
pure
>digitizing system it's impossible with today's technology. Why? Well a 16-bit
>A/D converter can provide only 96dB of dynamic range and these are generally
>limited to a few 100khz at best. I suspect that this is the approach taken by
>SRS in their low frequency spectrum analyzers (which are limited to about
>120khz).
>
>The high frequency spectrum analyzers use a very low noise sweeping
oscillator
>to down-convert the signal to a lower frequency. You mention synthesis; is
this
>what you're talking about? Once the signal has been down converted, you
can then
>use a low pass filter and digital signal processing techniques to analyze it
>further.
>
>If you really need a few Hz to 40MHz, then I'd probably try to design a
really
>low noise programmable (analog) oscillator and a low noise down converter.
The
>output of this would drive a low pass filter whose attenuation is on the
order
>of the dynamic range you seek (e.g. if you want 80db then at least a
4th-order
>analog filter would be required - a 5th-order filer is just an R and a C more
>expensive than a 4th-order one). The cutoff frequency of the analog filter
can
>more or less be arbitrary, but I'd suggest something below 60Hz just to be
>safe. The output of this filter can drive an amplifier and an A/D
converter. If
>you made the low pass filter with a higher cutoff (say 1khz) then you
could use
>digital filters to resolve finer frequencies. In other words, you could
combine
{Quote hidden}

that
>you're examining. If these two are about the same frequency, then the
difference
>is close to zero, but their sum is large. If you passed this product
through a
>low pass filter that had a cutoff greater than w1-w2, then you would be
able to
>examine g(t). Now if you abstract g(t) to an arbitrary wave, then you'll
still
>be able to extract the small frequency bin centered at your clock and with a
>width of twice your low pass filter.
>
>But you do have to be careful (as always). Suppose that the unknown signal
>contains two frequencies equally spaced from your clock (e.g w1 + delta1
and w1
>- delta1). These two would get aliased onto one another:
>
>g(t) = B*sin( (w1+delta)t + phi2a) + C*sin( (w1-delta)t + phi2b)
>
> f(t)*g(t) = A*B*cos( (-delta)t + phi1-phi2a) -
>             A*B*cos( (2*w1-delta)t + phi1+phi2a) +
>             A*C*cos( (delta)t + phi1-phi2b) -
>             A*C*cos( (2*w1-delta)t + phi1-phi2b)
>
>and after the carrier get's knocked out by the low pass filter you're left
with:
>
> f(t)*g(t) = A*B*cos( (-delta)t + phi1-phi2a) -
>             A*C*cos( (delta)t + phi1-phi2b) -
>
>The frequency -delta and delta will generate identical cosine waves. So
how do
>you differentiate these? Well, maybe this is where our HAM guys can jump in.
>Certainly if you made the low pass filter very narrow then you could resolve
>them. Now if you step back and look at this from a practical point of
view, it's
>going to be very difficult to design a high frequency clock with very low
>jitter. Consequently, the spectrum analyzer's frequency resolution is a
strong
>function of the stability of the sweeping oscillator. But as far as
devising a
>modulation technique to single out only one of these frequencies, I don't
have
>an answer. There is one observation to be made, however. Suppose we return to
>analyzing a single frequency again. As the oscillator sweeps pass this
frequency
>the demodulated version of the signal will be converted to a sine wave with a
>low frequency. Let's say that the low pass filter has a cutoff of 1kHz. If we
>looked at the demodulated signal after the low pass filter and let the
>oscillator sweep from low to high frequencies then this is what we'd see:
First
>the sine wave would appear at 1kHz. Then as the oscillator increases in
>frequency, the demodulated sine wave would decrease. When the oscillator
and the
>sine wave are the same frequency then we'd get DC. And finally, as the
>oscillator passes the sine wave's frequency, then the demodulated version
would
>begin to increase again. Perhaps this observation can be put to use?
>
>
>Scott


------------------------------------------------------------------------
Tom Handley
New Age Communications
Since '75 before "New Age" and no one around here is waiting for UFOs ;-)

More... (looser matching)
- Last day of these posts
- In 2000 , 2001 only
- Today
- New search...