Searching \ for '[PIC]: The tintinnabulation of the noise' in subject line. ()
Help us get a faster server
FAQ page: www.piclist.com/techref/microchip/devices.htm?key=pic
Search entire site for: 'The tintinnabulation of the noise'.

Exact match. Not showing close matches.
PICList Thread
'[PIC]: The tintinnabulation of the noise'
2000\11\07@102044 by

(above quote is from Poe)
(yes, tintinnabulation is a word)

I'm sampling a DC voltage with lots O noise.  I've set up a routine that takes a sample, waits a few milliseconds, then takes another sample, etc.  When 15 samples are taken, it takes the median of the samples as the result.  Still get false alarms due to noise pulses every once in a while.
Now, I am not real sure if I will have 60 hz noise (picked up from the air) or 120 hz noise (from a full wave rectified power supply) as the major noise component.  It may actually be both, at various times.
Does Old Nyquist rule in this situation, should I sample at twice 120 hz or more to cancel out the effects of noise?   I'm not interested in the frequency of the signal at all, just in an accurate DC level.

Let's see: 120 hz = 8.33 milliseconds per oscillation.  If I sample at twice this, that's 4.33 milliseconds.  sampling at exactly twice is cutting it fine, so say we sample at four times that's ~2 milliseconds.

I've got enough memory to increase the number of samples, maybe that will help as well?
-- Lawrence Lile
Sr. Project Engineer
Salton inc. Toastmaster Div.
573-446-5661 Voice
573-446-5676 Fax

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

>Let's see: 120 hz = 8.33 milliseconds per oscillation.  If I sample at
twice this, that's 4.33 milliseconds.  sampling at exactly twice is cutting
it fine, so say we sample at four times that's ~2 milliseconds.
>
>I've got enough memory to increase the number of samples, maybe that will
help as well?

This only helps you pick up the noise..
If you want it to go away, you need to narrow your bandwidth, by averaging.

If you take one sample per second, and the sample is an average over the 1S
period, then you won't see any 60 Hz noise.  If you just take a 1uS sample
once a second, then you will get the full value of the 60 hz noise,
wherever in it's waveform you happen to snag it.  An input RC with a long
time constant will work, but you'll have to adjust the time constant
depending on any other requirements in the system.

You could also take multiple samples at some rate that is an integer
multiple of 60 Hz, and mathematically average them. It's important that you
take an integer multiple of samples though, because otherwise you'll be
left with a "stub" of a cycle that won't average out.

It gets stickier if the noise can be amplitude modulated.  Then the
cleanest technique is to average in hardware.

--
Where's dave? http://www.findu.com/cgi-bin/find.cgi?kc6ete-9

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

Lawrence Lile wrote:

>I'm sampling a DC voltage with lots O noise.  I've set up a routine that
takes a sample, waits a few milliseconds, then takes another sample, etc.
When 15 samples are taken, it takes the median of the samples as the result.
Still get false alarms due to noise pulses every once in a while.
>
>Now, I am not real sure if I will have 60 hz noise (picked up from the air)
or 120 hz noise (from a full wave rectified power supply) as the major noise
component.  It may actually be both, at various times.
>

Sounds like you are taking way too few samples here. Also,
remember that you will only get the "correct" DC value of a
periodic waveform if the total sampling time is an "exact"
multiple of the waveform period. If you get something like the
following, your average will be off:

x                      x
x   x                  x   x
x      x               x
x----------x-----------x----------
x       x
x   x
x

The last four samples are going to skew the result here. It
would seem a good way to compensate for this problem would
be to take a lot of samples, and as many per period as possible,
then the endpoint discontinuities will have less effect.

Some sort of h.w. low-pass filtering might also help.

regards,
- dan michaels
===============

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

At 09:21 AM 11/7/00 -0600, Lawrence Lile wrote:

>I'm sampling a DC voltage with lots O noise.  I've set up a routine that
>takes a sample, waits a few milliseconds, then takes another sample,
>etc.  When 15 samples are taken, it takes the median of the samples as the
>result.  Still get false alarms due to noise pulses every once in a while.
>
>Now, I am not real sure if I will have 60 hz noise (picked up from the
>air) or 120 hz noise (from a full wave rectified power supply) as the
>major noise component.  It may actually be both, at various times.

I'd be tempted to hardware filter (RC network) any hash and sample often
enough so that the 120 Hz component averages out.  That means setting the
sample rate to some multiple of 120 Hz - usually fairly easy to do.

I have done jobs where I had to run unshielded a/d lines in the same 800
foot long conduit as the 208 Vac lines that were powering the system.  The
a/d lines went thru a RC network and into the PIC where I had trimmed the
timer tick for exactly 8 samples per AC half cycle, for a total of 32
samples per a/d channel.  No problems at all.

dwayne

Dwayne Reid   <dwaynerplanet.eon.net>
Trinity Electronics Systems Ltd    Edmonton, AB, CANADA
(780) 489-3199 voice          (780) 487-6397 fax

Celebrating 16 years of Engineering Innovation (1984 - 2000)

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Do NOT send unsolicited commercial email to this email address.
This message neither grants consent to receive unsolicited
commercial email nor is intended to solicit commercial email.

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

You mean a software filter?  Got any slick ways of performing this trick?

----- Original Message -----   From: Olin Lathrop   To: Lawrence Lile   Sent: Tuesday, November 07, 2000 9:43 AM
Subject: Re: [PIC]: The tintinnabulation of the noise

>>
Now, I am not real sure if I will have 60 hz noise (picked up from the air)
or 120 hz noise (from a full wave rectified power supply) as the major noise
component.  It may actually be both, at various times.

Does Old Nyquist rule in this situation, should I sample at twice 120 hz or
more to cancel out the effects of noise?   I'm not interested in the
frequency of the signal at all, just in an accurate DC level.

Let's see: 120 hz = 8.33 milliseconds per oscillation.  If I sample at twice
this, that's 4.33 milliseconds.  sampling at exactly twice is cutting it
fine, so say we sample at four times that's ~2 milliseconds.

I've got enough memory to increase the number of samples, maybe that will
help as well?
<<

I would recommend a different approach.  Sample as fast as you possibly can,
then run the result thru a one or two pole low pass filter.  You only need
to store one number per pole, not one number per sample in the average.
This type of filter (gaussian) is better at squashing high frequencies than
the box filter you described, requires less state, and is generally easier
to implement.  The main drawback is the slower response, which sounds like
can be much slower than 60Hz from your description.  Make the time constant
as long as you can tolerate to maximize the random noise reduction.  It
seems I end up doing at least a single pole filter in 90% of the projects
that measure an analog voltage.  The trick is to make the filter fraction a
power of 2 so that it can be done with bit shifting, and the rest is real
easy.

*****************************************************************
Olin Lathrop, embedded systems consultant in Devens Massachusetts
(978) 772-3129, olincognivis.com, http://www.cognivis.com

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

This could easily ignite the Average vs Median debate that took up so much bandwidth about two years ago.  I am convinced that median filter techniques are superior to averaging, for this reason:  Say I have some pulse noise, with a nasty 60 hz hum superimposed on it  (that's probably what I'm dealing with):

3.0, 3.1, 2.9, 3.0, 3.1, 2.9, 1.0, 3.0, 2.9, 3.1

What is the average?   2.8 by my calculation
What if you eliminate the last data point?  the average is 2.76   hmmmmm....
What is the best representation of the data?  3.0, the median, no matter how many data points we have.

Also accurate averaging requires at least 16 bit math - with 8 bit math rounding errors add up to gitcha.   I'm trying to avoid hogging memory with yet another 16 bit divide routine.  Median is very code efficient.

I think I am convincing myself that this would not work well anywhere close to 60 or 120 hz X2  Nyquist sample rate - you are better off with 4 or 5 or 8  samples during each cycle, and sampling over a few cycles.  Nyquist was an optimist, just like Murphy!

So far I have figured on having a low pass RC filter on the input tuned to attentuate 60 hz severely.
I am curious about a "software low pass filter" technique mentioned in another post....
{Original Message removed}
Lawrence Lile wrote:
>This could easily ignite the Average vs Median debate that took up so much
bandwidth about two years ago.  I am convinced that median filter techniques
are superior to averaging, for this reason:  Say I have some pulse noise,
with a nasty 60 hz hum superimposed on it  (that's probably what I'm dealing
with):
>
>3.0, 3.1, 2.9, 3.0, 3.1, 2.9, 1.0, 3.0, 2.9, 3.1
>
>What is the average?   2.8 by my calculation
>
>What if you eliminate the last data point?  the average is 2.76   hmmmmm....
>
>What is the best representation of the data?  3.0, the median, no matter
how many data points we have.
>

By selectively eliminating far outliers, you might get a better
values either way.

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

>I think I am convincing myself that this would not work well anywhere
close to
60 or 120 hz X2  Nyquist sample rate - you are better off with 4 or 5 or 8
samples during each cycle, and sampling over a few cycles.  Nyquist was an
optimist, just like Murphy!

No, but he's often misunderstood.
The Nyquist rate is :

The reciprocal of the Nyquist interval, i.e., the minimum theoretical sampling
rate that fully describes a given signal, i.e., enables its faithful
reconstruction from the samples. Note: The actual sampling rate required to
reconstruct the original signal will be somewhat higher than the Nyquist rate,
because of quantization errors introduced by the sampling process

If you want to accurately describe a non-square wave, you need more samples.

>So far I have figured on having a low pass RC filter on the input tuned to
attentuate 60 hz severely.

If you're measuring DC, that works. If not.. read on for a horror story.

>I am curious about a "software low pass filter" technique mentioned in
another
post....

>  You could also take multiple samples at some rate that is an integer
>  multiple of 60 Hz, and mathematically average them. It's important that you
>  take an integer multiple of samples though, because otherwise you'll be
>  left with a "stub" of a cycle that won't average out.

I got involved with this, as a cleanup in a project where the departed head
engineer stuck a stepper motor, with 24W drive, an inch away from a magnetic
pickup head, whose output on a good day is a few uA of the desired signal.
Famous last words: "If it causes a problem, then I'll fix it."

The only good part was that the motor stepping and the ADC sampling off the
head amp, were locked in sync, and an exact multiple of 60 Hz, in order to
reject power line noise.

In our case, the motor noise waveform was an entirely variable quantity, from
near zero to several volts, due to unrepeatable internal magnetic balancing in
the mag heads. (move the pickup coil 3 mils either way.) Shielding was very
effective. Without it, there would have been much more signal.

The amplifier output, on the desired signal was about 1V. (+/- 50% in
amplitude
due to the media)

Due to some un-named person's excellent layout skills (me) the amp got no
pickup directly from the chopped 1A drive circuits less than an inch away, or
anything else to speak of. The only noise source was that which was
magnetically coupled into the head.

To be fair to the head manufacturer (Vikron) the head design was excellent.
The
motor field was HUGE (As in Biblical, plague of locusts huge), and their
careful design nearly eliminated all the external fields.

This rather noisy output was fed into a fun little "ADC", consisting of three
Xor gates, arrainged to form a sigma-delta converter, with about 6 bits
output,
due to timing constraints in the Z8 controller. (the Z8 divides the xtal by
12,
and many instructions are multi-cycle)

How we got rid of the noise:

We sampled at 7200s/s for 5 seconds. During that time, data would happen,
and a
period where there was no data.

We averaged the data into a buffer 120 samples long, inverted it, and added
the
buffer to the original samples. This gave us an inverted copy of the
reinforced
noise signal (exactly like in a boxcar averager) at the same amplitude. This
gave us motor noise, 60 Hz junk, and 120 Hz junk, all at one fell swoop.

NOTE: Analog filters would not have been possible. The data being recovered is
also in these bands, and all analog filters have phase shifts around their
cutoffs. It would be possible to get rid of the noise, but the data would also
have been gone.

Unfortunately, this couldn't completely eliminate the noise, since the head
moves during the read process, and this amplitude modulates the motor signal,
and to a lesser degree, external 60 and 120 hz fields (inverse square law).
The
period of the modulation was too long for the same technique to be used to
remove it's residual.

We sold a european version, with the sample rate set to match up to 50 Hz
power.

FWIW, the mag head mounts directly on the two-layer PCB, which is routed in a
serpentine pattern to create a spring, loading the head against the document
and drive wheel, and eliminating the need for any cables or mounting brackets.
(patented :)

--
Where's dave? http://www.findu.com/cgi-bin/find.cgi?kc6ete-9

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

> > I would recommend a different approach.  Sample as fast as you
> > possibly can, then run the result thru a one or two pole low pass
> > filter.  You only need to store one number per pole, not one number
> > per sample in the average.  This type of filter (gaussian) is better
> > at squashing high frequencies than the box filter you described,
> > requires less state, and is generally easier to implement.  The main
> > drawback is the slower response, which sounds like can be much slower
> > than 60Hz from your description.  Make the time constant as long as
> > you can tolerate to maximize the random noise reduction.  It seems I
> > end up doing at least a single pole filter in 90% of the projects
> > that measure an analog voltage.  The trick is to make the filter
> > fraction a power of 2 so that it can be done with bit shifting, and
> > the rest is real easy.
>
> You mean a software filter?  Got any slick ways of performing this
> trick?

As I said, I would use either a single pole or double pole filter (which is
really just two single pole filters cascaded).  Note that for the single
pole Butterworth filter

FILT = (1 - F)*FILT + F*NEW

the multiply by F becomes just a right shift when F = 2**(-N) where N >= 0.
Rearranging the equation to

FILT = FILT - F*FILT + F*NEW

shows that all you need to do is two shifts and two adds.

Here is one of my math library routines that perform this operation.  You
won't be able to assemble it outside my environment, but the algorithm
should be obvious enough:

;   Subroutine FILT
;
;   Filter the value in REGB into REGA.  REGB will be left unchanged.  The
;   filter fraction in bits is in REG8.  REG8 must be zero or a positive
;   number which determines the weighting fraction of the new value in
;   REGB.  This is a single pole filter operation described by the
;   following equations:
;
;     W = 2 ** -N
;     RESULT = OLD * (1 - W) + NEW * W
;
;   W is the weighting fraction for the new value in REGB.  W is determined
;   from N, which is the REG8 value.  REGA contains OLD on entry, and
;   contains RESULT on return.
;
extern  shiftrlb    ;shift right logical of REGB by REG8 bits
extern  suba        ;subtract REGA - REGB into REGA
extern  adda        ;add REGA + REGB into REGA

.filt    code
glbsub  filt, regfb
;
;   Remove the W fraction of REGA from itself.  REGA is copied into
;   REGB, REGB is then shifted the selected amount right and subtracted
;   from REGA.
;
movf    rega+0, w   ;copy REGA into REGB
movwf   regb+0
movf    rega+1, w
movwf   regb+1
movf    rega+2, w
movwf   regb+2
movf    rega+3, w
movwf   regb+3

gcall   shiftrlb    ;shift REGB right to make fraction to sub from
old val
gcall   suba        ;subtract the fraction of the original value
from itself
;
;   Add the W fraction of the new value in REGB into REGA to make the final
;   filtered result.
;
popregs regfb
> >tore the original REGB value
movlw   4           ;diddle stack pointer so old REGB back on the
stack
subwf   stackp

gcall   shiftrlb    ;shift REGB right to make fraction of new value
gcall   adda        ;add fraction of new value into REGA to make
final value

leaverest

*****************************************************************
Olin Lathrop, embedded systems consultant in Devens Massachusetts
(978) 772-3129, olincognivis.com, http://www.cognivis.com

--
http://www.piclist.com hint: To leave the PICList
piclist-unsubscribe-requestmitvma.mit.edu

You could try low-pass filter with average+=(sample-average)/X (where
constant X defines the frequency response time of the filter), no
FIR-style sample array required so you can sample more quickly. But 8 bit
samples don't leave much headroom for X, so you can't sample TOO fast.
I've never seen an analysis of how best to derive X. Anyone have ideas on
this? == Rich

On Tue, 7 Nov 2000, Lawrence Lile wrote:

{Quote hidden}

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
"[PIC]:","[SX]:","[AVR]:" =uP ONLY! "[EE]:","[OT]:" =Other "[BUY]:","[AD]:" =Ads

On Wed, 8 Nov 2000 rich+piclistLCLOGIC.COM wrote:

> You could try low-pass filter with average+=(sample-average)/X (where
> constant X defines the frequency response time of the filter), no
> FIR-style sample array required so you can sample more quickly. But 8 bit
> samples don't leave much headroom for X, so you can't sample TOO fast.
> I've never seen an analysis of how best to derive X. Anyone have ideas on
> this? == Rich

This is a recursive digital filter AKA IIR filter. So the answer is yes, I'm
sure many people have been taught this and have ideas on how to design optimum
IIR filters. However, what they most probably haven't been taught are the
consequences of the errors introduced by using 8-bit arithmetic. This is
somewhat non-linear and as such falls into `emperical research' (try it out and
see what happens...). I've posted the z-transform for this filter back in Feb
'00, but that was the extent of the analysis.  This would be a good place to
begin searching for optimum solutions.

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
"[PIC]:","[SX]:","[AVR]:" =uP ONLY! "[EE]:","[OT]:" =Other "[BUY]:","[AD]:" =Ads

>> Let's see: 120 hz = 8.33 milliseconds per oscillation.  If I sample at
twice this, that's 4.33 milliseconds.  sampling at exactly twice is cutting
it fine, so say we sample at four times that's ~2 milliseconds.

IF you're exactly locked to the interference waveform, IF it's symmetrical,
and IF it's consistent from one wave to the next, then two samples
equidistant in time will cancel the waveform.  Any two points, 180 degrees
apart on a symmetrical waveform cancel to zero. (that's the inverse
definition of this sort of symmetry)

However. If the waveform is more complex, or varies much, or you're not
exactly synced to it, then more samples will be needed to clean it up.

Since you're trying to read DC, I'd low-pass it as much as you can stand,
and start there.

--
Where's dave? http://www.findu.com/cgi-bin/find.cgi?kc6ete-9

--
http://www.piclist.com hint: PICList Posts must start with ONE topic:
"[PIC]:","[SX]:","[AVR]:" =uP ONLY! "[EE]:","[OT]:" =Other "[BUY]:","[AD]:" =Ads

How does one calculate the cutoff frequency in the butterworth filter
algorithm shown below?

> the multiply by F becomes just a right shift when F = 2**(-N) where N >=
0.
> Rearranging the equation to
>
>   FILT = FILT - F*FILT + F*NEW

{Original Message removed}
> How does one calculate the cutoff frequency in the butterworth filter
> algorithm shown below?
>
> > the multiply by F becomes just a right shift when F = 2**(-N) where N >=
> 0.
> > Rearranging the equation to
> >
> >   FILT = FILT - F*FILT + F*NEW

Look at the impulse response.  If FILT is set to 1 and all future NEW values
are 0, then FILT will be a decaying exponential.  It will decay by (1 - F)
each iteration.  Therefore, after N iterations:

FILT = (1-F)**N

Now let's look at this in log space:

Log(FILT) = Log(1-F) * N

To find the number of iterations it takes to decay to one standard "time"
constant (1/e), assuming e as the log base:

Log(1/e) = -1 = Log(1-F) * N

-1 / Log(1-F) = N

If filter iterations are performed at regular intervals with a period t,
then the filter time constant is:

time constant = -t / Log(1-F)

and the roll off frequency is

roll off frequency = -Log(1-F) / t

*****************************************************************
Olin Lathrop, embedded systems consultant in Devens Massachusetts
(978) 772-3129, olincognivis.com, http://www.cognivis.com

--
http://www.piclist.com hint: The list server can filter out subtopics
(like ads or off topics) for you. See http://www.piclist.com/#topics

>noise pulses

Since you know nothing about the noise, first try to analyze it imho. That
will tell you what you need to do. You probably want a spectrum (over
longer time) and a peak envelope. The peak envelopes (upper and lower)
will tell you how to tune your median filter and the spectrum will help to
choose the sampling frequency. If the noise is constant frequency then you
can probably implement a noise cancelling algorythm by using the delta
between every two samples (you sample at twice the noise's frequency) to
average. I.e. S0 = Si + Sn, S1 = Si - Sn and you do Sii = (S0 + S1)/2
which gets rid of the synchronous noise.

You can modify your application to dump data to a PC over a serial link
(at 9600 you can pass about one byte per msec) and analyze with Excel or
whatever you use for numerology ;-).

In my experience it pays to improve the pickup/sensor system much more
than the algorythms because a poor S/N will always be a poor S/N. If there
is really a lot of noise maybe you can do something with your sensor to
make IT narrow band (like modulating the emitter of the sensor for
example). Then you detect syncronously (you do exactly as above but this
time you want +/- Sn which is your signal, so you substract. There are
also other options in this case).

hope this helps,

Peter

--
http://www.piclist.com hint: The PICList is archived three different
ways.  See http://www.piclist.com/#archives for details.

More... (looser matching)
- Last day of these posts
- In 2000 , 2001 only
- Today
- New search...