Post by thread:[PIC:] 10-bit A2D: How to soft filter w/o lose LSB

This is a repost, I forgot the colon ':' after the PIC tag. I have a P-I temperature controller using a PIC 16f877. There is apparent random noise on the least significant 2 or 3 bits of the thermistor sensor connected to the 10-bit A/D. At least random as it appeared on my spreadsheet. I am using a 200ms sample rate. How can I software filter this noise without losing the least significant bits of the A/D? I do not have a black-belt in math so I was thinking about something simple like: Avg=Avg + (Sample-Avg)/k This would be simple to implement if k is a power of 2, however, this appears to lose accuracy. Thanks in advance for any tips or suggestions, -Ed -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

Median Filters have been suggested in the past as an easy to implement filter type. Take an odd number of readings, sort into ascending order, then use the centre one. Richard P This is a repost, I forgot the colon ':' after the PIC tag. I have a P-I temperature controller using a PIC 16f877. There is apparent random noise on the least significant 2 or 3 bits of the thermistor sensor connected to the 10-bit A/D. At least random as it appeared on my spreadsheet. I am using a 200ms sample rate. How can I software filter this noise without losing the least significant bits of the A/D? I do not have a black-belt in math so I was thinking about something simple like: Avg=Avg + (Sample-Avg)/k This would be simple to implement if k is a power of 2, however, this appears to lose accuracy. Thanks in advance for any tips or suggestions, -Ed -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

> I have a P-I temperature controller using a PIC 16f877. There is > apparent random noise on the least significant 2 or 3 bits of the > thermistor sensor connected to the 10-bit A/D. At least random as it > appeared on my spreadsheet. I am using a 200ms sample rate. > > How can I software filter this noise without losing the least > significant bits of the A/D? > > I do not have a black-belt in math so I was thinking about something > simple like: > Avg=Avg + (Sample-Avg)/k > > This would be simple to implement if k is a power of 2, however, this > appears to lose accuracy. That equation is one way to describe a standard software low pass filter. When K = 2**N, then the divide by K is just a right shift by N bits. The way to not loose accuracy is to use fixed point with at least N fraction bits for SAMPLE and AVG. ***************************************************************** Embed Inc, embedded system specialists in Littleton Massachusetts (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

Thanks to everyone for their suggestions. I am starting to feel more comforatble with the PIC ( except for code pages and bank selections ). I implemented Neil's approach for an A/D averaging filter. Avg = (sample_1 + sample_2 + ... + sample_n) / n It was quite cheap RAM-wise and seems to work well enough. I had the time in my main loop, so I sample sequentially 16 times, use the fractional part to round-up the LSB of my 10-bit average, and stuff it back into ADRESH/L so the original code is none the wiser. I pasted my implemented solution below. -Ed Disclaimer: The code below was written by a PIC newbie ;************************ ;* ReadA2DChanW ;* ;* Read A2D channel passed in W. ;* Reads the specified A2D chan 16 times in succession, calculates the average, ;* and rounds up the LSB if the fractional random noise part is 50% or more, ;* and stores the result as a left-justified result in ADRESLH/L. ;* It is stored left-justified so as not to disturb the original code that ;* expects left-justified. ;* ;* Inputs: Pass A2D channel in W ;* Modifies: R0, ACCaHI/LO ;* ;* Disclaimer: Warning this code was written by a PIC newbie *** ;************************ ReadA2DChanW: banksel ADRESH movwf ADRESH ;Use ADRESH to temporarily hold the A/D channel passed in W BANKSEL ADCON1 movlw (0x01<<ADFM) ;8 analog inputs, right-justified to make averaging easier movwf ADCON1 ;Or-in the A/D channel passed in W with ADCON0 bits ;and start the conversion banksel ADRESH movf ADRESH,W iorlw ((0x01<<ADCS0)|(0x01<<ADON)) movwf ADCON0 ;Why was the 1ms delay originally here? ;Channel select settling time? ;If so, perhaps we could set the channel to the next channel to be acquired on exit banksel PORTA call wait_1ms ;A/D acquisition ;Use RO to count the n-number of A/D samples we will average ;The average will be stored in ACCaHI/LO movlw AVG_FILTER_N_READINGS movwf R0 clrf ACCaHI clrf ACCaLO loopWhileNAvgsBegin: #ifdef SIM_DEBUG ;For testing A/D filter math/logic in simulator movlw 0x02 ; 0x02a9 is 25C using 10K thermistor movwf ADRESH movlw 0xa9 banksel ADRESL movwf ADRESL banksel PORTA #else ;Loop until A/D conversion is done bsf ADCON0,GO btfsc ADCON0,NOT_DONE goto $-1 #endif ;Sum the result in ACCaHI/LO using a 16-bit add banksel ADRESL movf ADRESL,W ; Get low byte banksel PORTA addwf ACCaLO,F ; Add to destination movf ADRESH,W ; Get high byte btfsc STATUS,C ; Check for carry incf ADRESH,W ; Add one for carry addwf ACCaHI,F ; Add high byte into DST loopWhileNAvgsEnd: decfsz R0,F goto loopWhileNAvgsBegin samplesDone: ;The sum is now stored in ACCaHI/LO ;Calculate average and store result back in ADRESH/L ;We have done 16 averages of a left-justified A/D value ;Effectively we have done a shift-left<<4 or a multiply by 16 ;The 4 LSB's should be the accumulation of random noise roundUp: ;If the noise part is >= half(the first LSB is set), ;then round-up and add 1 to the LSB btfss ACCaLO,FRACTIONAL_BIT_POS goto noRoundUp ADDL16 ACCaHI,(0x01<<FRACTIONAL_BIT_POS) noRoundUp: ;Shift left two more times so that the code that is expecting left-alligned A2D ; will still work as expected ;b00000011 11111111 1 - We started with a right-justified 10-bit result ;b00111111 1111xxxx 2 - Then added 16 samples together, we effectively shifted left <<4 ;b11111111 11xxxx00 3 - Need to shift<<2 to left allign to make existing software happy LSL16 ACCaHI LSL16 ACCaHI ;Save result in the A2D result registers after masking off the unused bits ;Hmmm.. if I had kept the LSB noise bit, would this be quivalent to an 11-bit A2D?? movf ACCaHI,W movwf ADRESH ;Mask off the unused bits of the left-justified 10-bit A/D value before ;putting back into ADRESL movf ACCaLO,W andlw b'11000000' banksel ADRESL movwf ADRESL banksel PORTA return -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

> I implemented Neil's approach for an A/D averaging filter. > > Avg = (sample_1 + sample_2 + ... + sample_n) / n Yes, that will get you some random noise reduction. However, this is just a box filter, which is not a very good filter and is rather memory intensive. Think beyond just "averaging" and consider this filter in frequency space. For random noise reduction and other low pass filtering, I usually use one or two stages of a filter: FILT <-- FILT + (NEW - FILT) * FP Where FILT is the filter output, NEW is the new value being accumulated into the filter, and FP is the "filter pass fraction". FP is intended to be in the range of 0 to 1. FP = 0 means the filter output never changes, and FP = 1 causes the filter to take on each new input value without any filtering. Someone else posted basically the same equation earlier today. ***************************************************************** Embed Inc, embedded system specialists in Littleton Massachusetts (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

----- Original Message ----- From: "Olin Lathrop" <spam_OUTolin_piclistTakeThisOuTEMBEDINC.COM> <snip> > For random noise reduction and other low pass filtering, I usually use one > or two stages of a filter: > > FILT <-- FILT + (NEW - FILT) * FP > > Where FILT is the filter output, NEW is the new value being accumulated into > the filter, and FP is the "filter pass fraction". FP is intended to be in > the range of 0 to 1. FP = 0 means the filter output never changes, and FP = > 1 causes the filter to take on each new input value without any filtering. Beginner question - Is FP variable in realtime? This is beginning to sound an awful lot like a simplified Kalman filter. - Robert -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

On Thu, 21 Aug 2003, Olin Lathrop wrote: > > I implemented Neil's approach for an A/D averaging filter. > > > > Avg = (sample_1 + sample_2 + ... + sample_n) / n > > Yes, that will get you some random noise reduction. However, this is just a > box filter, which is not a very good filter and is rather memory intensive. > Think beyond just "averaging" and consider this filter in frequency space. Actually, it can be an excellent filter and it uses hardly any memory. Of course, "not very good" and "excellent" are hardly objective terms! But for filtering a thermistor signal I'm sure it's just fine. The reason it doesn't take any memory is that the sum is "in situ". And as the original poster noted, the division by n is a simple shift right when n is an even power of two. If you were concerned with frequency response, it's not hard to turn the box into a triangle - then your sinc's get turned into sinc^2's. {Quote hidden}> > For random noise reduction and other low pass filtering, I usually use one > or two stages of a filter: > > FILT <-- FILT + (NEW - FILT) * FP > > Where FILT is the filter output, NEW is the new value being accumulated into > the filter, and FP is the "filter pass fraction". FP is intended to be in > the range of 0 to 1. FP = 0 means the filter output never changes, and FP = > 1 causes the filter to take on each new input value without any filtering. > > Someone else posted basically the same equation earlier today. This is a good filter too. Depending on "FP" however it could be time consuming to perform the multiplication. If FP is of the form: A/(A+B) and A+B is an integer power of two and A and B are positive integers, then this filter can be implemented efficiently: http://www.dattalo.com/technical/software/pic/twist.asm Scott -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

Olin, Since this is implemented as a 2-byte counter (with just 2 registers for the sum and one to count the number of samples), I'm curious why you call it memory intensive. This is not a "moving window" average (or circular buffer), since the display is updated only after a full set of samples have been averaged, rather than updating with each new sample. See my previous post if you want the details. Anyway, the equation you posted is interesting -- I like the idea of filtering in the freq. domain, so I'd like to understand the theory and derivation of the equation. It seems like you're steering the output towards the new sampled value but capping or limiting the change at each signal. Can you add some detail or provide a link to some more info? Cheers, -Neil. On Thursday 21 August 2003 18:47, Olin Lathrop scribbled: {Quote hidden}> > I implemented Neil's approach for an A/D averaging filter. > > > > Avg = (sample_1 + sample_2 + ... + sample_n) / n > > Yes, that will get you some random noise reduction. However, this is just > a box filter, which is not a very good filter and is rather memory > intensive. Think beyond just "averaging" and consider this filter in > frequency space. > > For random noise reduction and other low pass filtering, I usually use one > or two stages of a filter: > > FILT <-- FILT + (NEW - FILT) * FP > > Where FILT is the filter output, NEW is the new value being accumulated > into the filter, and FP is the "filter pass fraction". FP is intended to > be in the range of 0 to 1. FP = 0 means the filter output never changes, > and FP = 1 causes the filter to take on each new input value without any > filtering. > > Someone else posted basically the same equation earlier today. > > > ***************************************************************** > Embed Inc, embedded system specialists in Littleton Massachusetts > (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics

>> FILT <-- FILT + (NEW - FILT) * FP >> >> Where FILT is the filter output, NEW is the new value being >> accumulated into the filter, and FP is the "filter pass fraction". FP >> is intended to be in the range of 0 to 1. FP = 0 means the filter >> output never changes, and FP = 1 causes the filter to take on each new >> input value without any filtering. > > Beginner question - Is FP variable in realtime? This is beginning to > sound an awful lot like a simplified Kalman filter. FP can change if you want it to, but that makes the code a lot more complicated. In the usual noise filtering application, the sample rate and desired filter frequency is know, in which case FP can be a constant. You can do a shift instead of a multiply when FP is 2**-N. ***************************************************************** Embed Inc, embedded system specialists in Littleton Massachusetts (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com hint: The PICList is archived three different ways. See http://www.piclist.com/#archives for details.

> Since this is implemented as a 2-byte counter (with just 2 registers > for the sum and one to count the number of samples), I'm curious why > you call it memory intensive. This is not a "moving window" average > (or circular buffer), since the display is updated only after a full > set of samples have been averaged, rather than updating with each new > sample. OK, I didn't catch that. Yes, that means it doesn't require much memory, but is an even worse than a box filter in terms of random noise reduction. This kind of filter (and a box filter) can be useful to eliminate specific frequencies because the response goes to zero at the reciprocal of the sample window period and all its harmonics. This can be handy, for example, for eliminating 60Hz line hum. > Anyway, the equation you posted is interesting -- I like the idea of > filtering in the freq. domain, Actually, I was encouraging you to think of the filter in the frequency domain. After all, your motivation for wanting a filter is based on frequencies, so it makes sense to evaluate filters on their frequency response. The equation I posted is more of a time domain realization of the filter. Nothing is being converted to frequecies, mucked around with, then converted back to the time domain. > so I'd like to understand the theory and > derivation of the equation. It seems like you're steering the output > towards the new sampled value but capping or limiting the change at > each signal. Can you add some detail or provide a link to some more > info? This is straight out of first semester digital signal processing. There must be many references to this, but I haven't looked. This particular filter is a very simple case of an "IIR" (infinite impulse response) filter. I don't want to reiterate digital signal processing 101, but briefly this filter models a single R/C analog low pass filter. If the filter starts at 0 and you suddenly change the input values from 0 to 1, the output will follow an exponential approaching 1. Each sample it will get the same fraction closer to the input value. Flip this around and it can be thought of as having a "half life". That is the period, in samples, where the filter value will get half way to the input value. There is also therefore a period in which the output will get to (1 - 1/e) (about 63%) toward the input. This period is refered to as the "time constant". ***************************************************************** Embed Inc, embedded system specialists in Littleton Massachusetts (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com hint: The PICList is archived three different ways. See http://www.piclist.com/#archives for details.

On Friday 22 August 2003 07:19, Olin Lathrop scribbled: > > Anyway, the equation you posted is interesting -- I like the idea of > > filtering in the freq. domain, > > Actually, I was encouraging you to think of the filter in the frequency > domain. After all, your motivation for wanting a filter is based on > frequencies, so it makes sense to evaluate filters on their frequency > response. Okay, I'm relieved. I was trying to get the connection between that equation and anything related to the freq domain. Agreed, it makes sense to filter based on freq. response, but I'm sure there are tons of (common?) shortcuts for doing this processing in high-speed assembly/microcontrollers. > The equation I posted is more of a time domain realization of the filter. I chalked it up to emulating a large filter cap, by "tracking" the newest input signal value. > Nothing is being converted to frequecies, mucked around with, then > converted back to the time domain. Gotcha. {Quote hidden}> > so I'd like to understand the theory and > > derivation of the equation. It seems like you're steering the output > > towards the new sampled value but capping or limiting the change at > > each signal. Can you add some detail or provide a link to some more > > info? > > This is straight out of first semester digital signal processing. There > must be many references to this, but I haven't looked. This particular > filter is a very simple case of an "IIR" (infinite impulse response) > filter. > > I don't want to reiterate digital signal processing 101, but briefly this > filter models a single R/C analog low pass filter. If the filter starts > at 0 and you suddenly change the input values from 0 to 1, the output will > follow an exponential approaching 1. Each sample it will get the same > fraction closer to the input value. Flip this around and it can be > thought of as having a "half life". That is the period, in samples, where > the filter value will get half way to the input value. There is also > therefore a period in which the output will get to (1 - 1/e) (about 63%) > toward the input. This period is refered to as the "time constant". Thanks for the explanation. Makes sense. Never did any class on DSP's, but I'll find a book or google for it. I'm also aware of Fourier transforms for doing stuff like this in the freq domain, but the only bits I've seen on this are long equations that I'm sure would be a pain to replicate in a microcontroller. I've not seen any simplified versions of these for use in microcontrollers? ... are there? Thanks, -Neil. -- http://www.piclist.com#nomail Going offline? Don't AutoReply us! email .....listservKILLspam@spam@mitvma.mit.edu with SET PICList DIGEST in the body

> Thanks for the explanation. Makes sense. Never did any class on > DSP's, but I'll find a book or google for it. I'm also aware of > Fourier transforms for doing stuff like this in the freq domain, but > the only bits I've seen on this are long equations that I'm sure would > be a pain to replicate in a microcontroller. I've not seen any > simplified versions of these for use in microcontrollers? ... are there? I didn't mean to do the actual computations by using Fourier transforms, only to *think* about what the filter is doing in the frequency domain. Most digital filters actually work in the time domain, partly because that's how the data comes in. For example, a specific desired frequency response would be just multiplying by a function of frequency in the frequency domain. While that part is easy, converting the incoming time samples to frequency domain so that the multiplies can be performed, then converting back to time domain requires a lot of computation. Instead, you can take the fourier transform of the desired frequency response to convert it to the time domain, then perform a convolution on the time domain data instead of a multiply on the frequency domain data. A convolution is only a multiply and add per element of the convolution function per input sample. This is more easily streamlined in hardware, and is the main reason why DSPs all have multiply-accumulate instructions, along with the addressing modes so that a convolution can be performed between two arrays while keeping the ALU busy with a multiply-accumuate every cycle. In fact, this is the defining feature that makes a DSP a DSP and not just a microcontroller. Again, this is right out of digital signal processing 101. I'm sure there are lots of references on this available on the net. ***************************************************************** Embed Inc, embedded system specialists in Littleton Massachusetts (978) 742-9014, http://www.embedinc.com -- http://www.piclist.com#nomail Going offline? Don't AutoReply us! email listservKILLspammitvma.mit.edu with SET PICList DIGEST in the body