Searching \ for '[EE] Writing two CF cards at once.. .' in subject line. ()
Make payments with PayPal - it's fast, free and secure! Help us get a faster server
FAQ page: www.piclist.com/techref/mems.htm?key=cf
Search entire site for: 'Writing two CF cards at once.. .'.

Exact match. Not showing close matches.
PICList Thread
'[EE] Writing two CF cards at once.. .'
2006\05\11@130105 by Russell McMahon

face
flavicon
face
I'd be interested in taking the signals meant to write data to a CF
card and using them to write to two cards in parallel so that I have a
redundant copy. Making one card faster than the other or delaying the
clocking slightly would be acceptable. Without having looked into it
I'd imagine that two structurally identical cards should allow one
card to be driven "blind" without too much risk of data loss.

Comments ?

The object is to provide data backup for a system which presently
writes to a single CF card. Occasionally cards fail and if you want to
avoid this a duplicate copy is essential. The writing system is not
amenable to modification - the dual media system must be transparent.


       RM

2006\05\11@131803 by Peter Todd

picon face
On Fri, May 12, 2006 at 05:00:49AM +1200, Russell McMahon wrote:
> I'd be interested in taking the signals meant to write data to a CF
> card and using them to write to two cards in parallel so that I have a
> redundant copy. Making one card faster than the other or delaying the
> clocking slightly would be acceptable. Without having looked into it
> I'd imagine that two structurally identical cards should allow one
> card to be driven "blind" without too much risk of data loss.
>
> Comments ?
>
> The object is to provide data backup for a system which presently
> writes to a single CF card. Occasionally cards fail and if you want to
> avoid this a duplicate copy is essential. The writing system is not
> amenable to modification - the dual media system must be transparent.

Sounds like you need two controllers for sure. If you try to write them
in parallel without the full two controller setup, how will you know
when one fails?

I'd make something that can masqurade as a CF card first. Once that's
done, tested, works essentially make a RAID backend for it. Otherwise I
think it'll be hard to put in the robust error checking you sound like
you need.

--
spam_OUTpeteTakeThisOuTspampetertodd.ca http://www.petertodd.ca

2006\05\11@171815 by Russell McMahon

face
flavicon
face
> Sounds like you need two controllers for sure. If you try to write
> them
> in parallel without the full two controller setup, how will you know
> when one fails?

The application is for photos. I'm a keen amateur photographer and in
some cases I'm extremely keen not to lose photos. And, the system
would be saleable if it worked well. [Build it before I do (and I
probably wont) and I'll buy one :-) ].

If the system is good enough to write at least one good photo of the
pair with a suitably low error rate then the loss of an occasional A
or B copy is acceptable as long as both A & B are lost with only very
low probability. Say 1:1000 for A or B and 1:100,000 for A & B copies
of a single pghoto would be very acceptable. Better would be better.

What I am trying to avoid is the loss of a whole card full of data, as
has happened to me enough times so far as to make me nervous when I'm
doing critical photo sessions. Sometimes the card is recoverable by
formatting and somtimes its dead. Losing eg the middle  half of a
wedding service or similar would be 'unfortunate'. What I do now is
carry a number of CF cards and alternate amongst them whenever there
is a 'gap in the traffic'. That way, if one card of say 4 fails, you
lose a number of time stripes and not a whole block. I have a laptop
setup to suck data when time allows. Next stage is to write to an
external drive, check the external and internal copies against the CF
card and only then delete it.



       Russell McMahon

2006\05\11@175101 by Marcel duchamp

picon face
Russell McMahon wrote:
> What I am trying to avoid is the loss of a whole card full of data, as
> has happened to me enough times so far as to make me nervous when I'm
> doing critical photo sessions.
>
>         Russell McMahon
>

Do you have any idea of how or why the cards failed?  I have never heard
of this so am interested.  Does your camera report that the save to disk
was ok but then later you find no photos? Or...?

I have a digital camera that uses CF cards also. One of my concerns is
the fairly large force required to seat the card in the cameras
connector.  It makes me wonder if it might wear out sooner rather than
later.  But as for failed operations, I have yet to see any although I
have only taken a few thousand shots so far. And this was distributed
over four cards.

2006\05\11@183255 by Robert Rolf

picon face
Russell McMahon wrote:

>>Sounds like you need two controllers for sure. If you try to write
>>them
>>in parallel without the full two controller setup, how will you know
>>when one fails?
>
> The application is for photos. I'm a keen amateur photographer and in
> some cases I'm extremely keen not to lose photos. And, the system
> would be saleable if it worked well. [Build it before I do (and I
> probably wont) and I'll buy one :-) ].

So you'd happily beta test a solution?

> If the system is good enough to write at least one good photo of the
> pair with a suitably low error rate then the loss of an occasional A
> or B copy is acceptable as long as both A & B are lost with only very
> low probability. Say 1:1000 for A or B and 1:100,000 for A & B copies
> of a single pghoto would be very acceptable. Better would be better.

Given the MTBF of CF cards, I'd expect better than 1:1E6 failure rate.

> What I am trying to avoid is the loss of a whole card full of data, as
> has happened to me enough times so far as to make me nervous when I'm
> doing critical photo sessions. Sometimes the card is recoverable by
> formatting and somtimes its dead.

This suggests that the problem is a SOFTWARE error, and so
having a redundant card would just mean that you have the SAME
error TWICE (e.g. on both cards). Perhaps your camera firmware
has a bug (and hopefully an update to fix it).

I am also surprised that you have had problems with cards
being 'dead'. Of course you are using high quality, brand name card,
like the SanDisk Extreme II and III series? Do you still have a dead
card around? Have you tried 'Image Rescue' software to recover?

Have you tried using a micro hard drive instead?

The only time I have had a bad card was when I was impatient and
pulled it out before the write completed. Even then, only the
last photo was garbage. But if the FAT is corrupted (file allocation
table, as used by the DCIM standard) you could have bit of a mess.

> Losing eg the middle  half of a
> wedding service or similar would be 'unfortunate'. What I do now is
> carry a number of CF cards and alternate amongst them whenever there
> is a 'gap in the traffic'. That way, if one card of say 4 fails, you
> lose a number of time stripes and not a whole block.

A good plan, since it appears that the problem MAY be your camera
firmware.

> I have a laptop setup to suck data when time allows.

> Next stage is to write to an
> external drive, check the external and internal copies against the CF
> card and only then delete it.

Presumably AFTER you have redundantly written the data to optical
media. Hard drives fail too.


In theory you could write to two absolutely identical cards in parallel,
but only READ one (since the software needs to update the FAT by
reading it).
A small bit of logic would isolate the cards from the bus
so that a silicon failure in one wouldn't cause data corruption.

But would you be happy having your cards stick out an extra inch
or two to accommodate the needed connectors and logic?
(assuming the camera supports the extra thickness of two cards
at once.)

Given the form factor issues, would a CF to dual microSD card
be a solution? The penalty is much higher cost and
much smaller capacity, but at least it would completely
fit into a CF slot.

Robert

2006\05\11@190651 by Mike Harrison

flavicon
face
On Fri, 12 May 2006 09:17:49 +1200, you wrote:

>> Sounds like you need two controllers for sure. If you try to write
>> them
>> in parallel without the full two controller setup, how will you know
>> when one fails?
>
>The application is for photos. I'm a keen amateur photographer and in
>some cases I'm extremely keen not to lose photos. And, the system
>would be saleable if it worked well. [Build it before I do (and I
>probably wont) and I'll buy one :-) ].

You can get little pocket devices that copy from one USB memory device to another.
I wonder if one of these would work with a USB CF reader, so you could copy CF cards to USB keys...?
ISTR reading that some CF cards (or was it SD...?) have a 'native USB' mode, so these certainly
ought to work this way.

I'm sure I've also seen a standalone device somewhere that will copy cards to a small hard drive.
 

2006\05\11@191213 by Alex Harford

face picon face
On 5/11/06, Mike Harrison <.....mikeKILLspamspam@spam@whitewing.co.uk> wrote:
>
> I'm sure I've also seen a standalone device somewhere that will copy cards to a small hard drive.

Something like this:

http://www.dvshop.ca/digcamera/photoviewer.html

2006\05\11@211612 by David VanHorn

picon face
Why would you not just do forward error correction, and store it on multiple
cards conventionally?  Then you're protected from small bit/byte/block
errors, and from "dead card" errors.

--
Feel the power of the dark side!  Atmel AVR

2006\05\13@091839 by Howard Winter
face
flavicon
picon face
Dave,

On Thu, 11 May 2006 21:16:11 -0400, David VanHorn wrote:

> Why would you not just do forward error correction, and store it on multiple
> cards conventionally?  Then you're protected from small bit/byte/block
> errors, and from "dead card" errors.

Reading between Russell's lines (!) I think for data-security he wants a device to plug into the camera
instead of the card, which then records the photo twice immediately, as part of taking the photo, rather than
doing something manually with the single card afterwards.  Hence his not being able to change the recording
side, only the card-emulator.

I wonder if using the camera's USB interface (I guess it has one) to record to a belt-worn USB hard drive
would be feasible?

Cheers,


Howard Winter
St.Albans, England


2006\05\13@144058 by Robert Rolf

picon face
Howard Winter wrote:
> Dave,
>
> On Thu, 11 May 2006 21:16:11 -0400, David VanHorn wrote:
>
>
>>Why would you not just do forward error correction, and store it on multiple
>>cards conventionally?  Then you're protected from small bit/byte/block
>>errors, and from "dead card" errors.
>
>
> Reading between Russell's lines (!) I think for data-security he wants a device to plug into the camera
> instead of the card, which then records the photo twice immediately, as part of taking the photo, rather than
> doing something manually with the single card afterwards.  Hence his not being able to change the recording
> side, only the card-emulator.
>
> I wonder if using the camera's USB interface (I guess it has one) to record to a belt-worn USB hard drive
> would be feasible?

Nope. I already asked a camera expert at a professional camera store.
There are some cameras that can fire their images over USB to a program
running on a laptop, and some that can store on a remote hard drive (high end
pro cameras typically used by newspaper photographers) but none of the consumer grade
cameras will talk to a stand alone remoted drive. Most consumer cameras stop
being cameras when you plug in the USB connection. A few will od 'webcam' mode,
but I doubt that Russell is using such a camera.

The expert was also surprised to hear of Russell's problem with the memory cards.
He said that in 5 years of card sales, amounting to well over 7 thousand cards,
he has had only 5 returns, 2 of which were due to physical damage to the card
(drowned and bent). None of the returns failed during use as far as he knew.
It might be useful for Russell to phone around to the bigger photo shops and
see if they're have a high failure rate with a particular brand or batch of cards.

I think my original comment about there being something wrong with
the camera's firmware still holds.
Has Russell checked the makers web site to see if there is a firmware update
for the camera, and what the update fixed? What make/model camera is it Russell
so others can be 'forewarned'?

Robert

2006\05\13@163257 by Russell McMahon

face
flavicon
face
Many thanks. There has been lots of interesting comment re my original
question and I'll try and sum up some time soon. Here's a very quick
comment to date.

What I wanted was a plug in that fitted a CF slot and wrote to an as
small as possible external cage that took 2 x CF cards in parallel OR
to an equivalent "redundant at source" store to overcome media failure
once a frame was stored. CF is now very cheap and card cost is not the
limiting factor. I'd run a 4 GB card IF I knew it was essentially
bulletproof. As it is, I use N x 512 MB and interleave them manually.

Camera now is not what I had when failures occurred. So far I've taken
20,000 odd shots since January 6th this year with no failures.

Old camera was a Minolta 7Hi "prosumer" 5 MP. Very nice camera. Now
dead after 200,000+ photos. Brands that failed were mixed and quality
in most cases. Included a Kingston 256 MB (London, very hot weather -
totally unreadable by any means I've tried). Another 512 MB in Taiwan
in hot but not so hot as London was evening weather. And a second 512
MB failed within minutes adding weight to the camera related
suspicions. Camera gets very hot under extended shooting which cannot
help. Of these one was able to be reformatted and another appeared
dead-dead. On trying it in several cameras, USB download, USB-card
reader, CF adaptor ... it was dead. BUT on trying it in my new 7D DSLR
recentlyish it would display in the camera sometimes (nice to see
photos I thought were lost) but won't download (:-( ). - When viewed
as a drive via USB the file names are scrambled and there are many
rubbish directories and no tool I have will touch it BUT camera can
read it in camera. ie card is probably logically scrambled but OK. I
have had very occasional other failure which was soft formattable
recoverable. I think I've had one other 512 MB failure with all frames
lost but reformat recoverable.

My new camera has USB on-the-fly download to PC capability. (Minolta
maxxum 7D DSLR, 7 MP).

Note that the very top cameras have two card slots (usually different
media types) that can be run either sequentially OR written
redundantly in parallel. The latter facility suggests that the pros
also expect some failures (or that the design engineers hold CF
company shares :-) ).

I will ask further about CF loss in action. However, having been burnt
enough so far I can't assume all will be well in crucial shoots in
future. Download to PC on the fly may turn out to be the way to go for
critical stuff, with auto background file copy to an external drive
incrementally. Wedding or other photographer wearing a laptop PC in a
backpack (plus extra battery capacity) would be a sight to see :-). As
I sometimes wear a 3 or 4 lens quick access bag at my waist and a belt
bag at the back with a ?5? AH 6V SLA therein and quick change cable
the step to wearable laptop may also be feasible :-). Almost. My 2 x
Librettos are not quite up to the task. A Linux box with dual drives
would do but camera software is not compatible.

This is a 'hobby' btw - not my 'day job'.

Re comment on being bad compared to film. Not so for me. I have lost a
far higher percentage of film shots over time. One whole roll lost in
Japan by processing lab. Occasional open back with film in
[Doh!](preventable) (destroys surprisingly few frames if fast).
Occasional failure to wind on at all (preventable). Occasional bad lab
work leads to negatives that deteriorate. Some of these are not
preventable without taking all work in house which is OK if that's
what you want to do but ... . Apart from the bad cards i have never so
far lost a digital picture in around ???300,000??? photos. I still, so
far, have every frame that i have not purposefully deleted. All online
on this LAN too fwiw.


       Russell McMahon

2006\05\13@210427 by Mike Harrison

flavicon
face
On Sat, 13 May 2006 14:18:36 +0100 (BST), you wrote:

>Dave,
>
>On Thu, 11 May 2006 21:16:11 -0400, David VanHorn wrote:
>
>> Why would you not just do forward error correction, and store it on multiple
>> cards conventionally?  Then you're protected from small bit/byte/block
>> errors, and from "dead card" errors.
>
>Reading between Russell's lines (!) I think for data-security he wants a device to plug into the camera
>instead of the card, which then records the photo twice immediately, as part of taking the photo, rather than
>doing something manually with the single card afterwards.  Hence his not being able to change the recording
>side, only the card-emulator.
>
>I wonder if using the camera's USB interface (I guess it has one) to record to a belt-worn USB hard drive
>would be feasible?

With some cameras, when plugged into USB they appear as another drive. I don't know enough about USB
to kow how similar this looks to a USB memory key, i.e. whether one of those USB-USB copier devices
might work to make a copy of the card without removing it from the camera. A possible issue though
is many cameras go into a 'pc connect' mode which suspends normal operation while connected.


2006\05\14@024945 by Russell McMahon

face
flavicon
face
>>I wonder if using the camera's USB interface (I guess it has one) to
>>record to a belt-worn USB hard drive
>>would be feasible?

> With some cameras, when plugged into USB they appear as another
> drive. I don't know enough about USB
> to kow how similar this looks to a USB memory key, i.e. whether one
> of those USB-USB copier devices
> might work to make a copy of the card without removing it from the
> camera. A possible issue though
> is many cameras go into a 'pc connect' mode which suspends normal
> operation while connected.

The 7D has several modes.
I usually use its USB interface to download photos and in this mode it
appears, as you say, as a disc drive. But it also suspends camera
action. Its not apparent that this is necessary and it would be
extremely useful if it could act as a camera while having it's memory
appear as a disc volume. But it cant. This would certainly present
some challenges both in file sharing, memory access and file name
control but would be worthwhile.

It also has a mode where it does direct download to a PC while in
camera mode. This uses the USB but (as I haven't yet tried it) I
imagine that the memory store is effectively the PC memory. This
effectively implements what I said would be ideal in the paragraph
above BUT requires full capability on the part of the target PC to
handle transfers in a manner that does not impact camera operation. I
will certainly be able to use this in some contexts. If the protocol
was published it would allow the construction of a non-PC target box.
I'm not aware that it is published and Minolta have, as of 31 March
2006, ceased making cameras :-( so it's less likely to become
officially available in future. Hopefully some hackers have/will
decode it.

My camera has focusing issues and I will be returning it for
correction. (Warranty and support still good). After that I will look
at the firmware change needed for direct download. (I have the
software which is and was a free download for 7D owners.)

The dual CF solution was and is attractive as it is transparent to the
camera, needs no great processing power and is applicable to a wide
range of top cameras. CF is still pretty much king for serious cameras
despite the large number of newer contenders.



       Russell McMahon


2006\05\15@081341 by Russell McMahon

face
flavicon
face
I discovered this site

       http://www.datarescue.com/photorescue/index.htm

(link from someone here or Googled ?) and it looks like it's going to
recover the 357 images from my crawling-wounded 512 MB which died 'in
the hat of the night' in Taiwan. Cost is $US29. They have a demo
version which recovers data but displays only thumbnails. If it can do
this they guarantee full photo recovery or your money back.

Reading their self testimonials and descriptions I get a very good
feeling of the product's capabilities. It also just happens to be the
cheapest I could find that claimed to do what I wanted. it also claims
to do far far more. If it's half as good as they say it will be an
excellent addition to my toolbox.

They claim to not touch the drive data and to often be able to still
recover data after other programs have further trashed a munged disk
and failed to recover anything. Belgian sourced.

Download code has just dropped into my mailbox so I should know
shortly how good it really is. It's only midnight now so ... :-)


       Russell McMahon

2006\05\16@084230 by Russell McMahon

face
flavicon
face
>> I discovered this site
>>
>> http://www.datarescue.com/photorescue/index.htm

I think the following was offlist but I've removed pesonal identifiers
and content seems generic techo, so

> There's also a program called Data Rescue (now Data Rescue II) from
> Prosoft Engineering that does a very good job of recovering files.
> It takes the approach that the suspect disk is not to be touched.
> All recovered files are written to another drive.

At http://www.prosofteng.net/

MAC version $US99
PC version $US 129.

May provide more functionality than the $US29 Photo Rescue, but PR
worked very very well for me. I'd definitely not have considered a
$US129 program at this stage (about the cost of 8 x 512 MB el junko
CFs or 70% of that in good ones). But $US29 felt good for what it
did - and what it is liable to do in future. The money back if it
doesn't work feature is good.

> Both the better grade Sandisk Extreme CF cards and the Lexar 80X CF
> cards come bundled with data recovery software.  I've tested both to
> make sure I have them available if needed.  But then, since I'm
> using
> pro grade CF cards, I've never needed them for either a Sandisk or
> Lexar card.

I'll have to look at those to see if the extra price is justified by
the software. Card speed for me has often seemed camera and card
combination dependant. There are certainly occasions where I'd like a
faster card write (many shots in sequence at 4 fps) but mostly the 7D
and anything I have got seems OK. My old 7Hi would with some cards
work far faster the first time used after formatting. I started with
premium cards and found they didn't make enough (if any) real world
difference to be worth the extra.

7D max speed is same at all resolutions.
7Hi had a 1 MP x 7 fps mode.
Useful for eg yumping rally cars or most sports action but low res a
shame and it then took forever to write the buffer once you stopped
shooting - better to keep shooting till the camera's ~~ 100 MB
internal buffer filled 'just in case".

The 7Hi was/is an utterly amazing camera - a far better photo taking
"system" than the best DSLRs in the world. Photo quality was only
superb compared to the "ridiculous" of the newer cameras.

> As icing on the cake, each Sandisk Extreme CF comes with a neat
> zippered wallet.

I use a small zippered lady's coin purse discarded by my wife :-)
Holds more CFs than I ever carry but fits in any pocket.
When shooting a function I run a multi-pocket system :-)
Bare / part written / full for download / downloaded but still full.

"Part written" because I interleave cards for security.
"Downloaded but full" because, why delete until you must as extra
security.

I'm going to change to a system whereby I rename files after
downloading so there's no doubts - provided speed impact is not too
high.
Also a PC plus external drive system where files are downloaded to
laptop HDD, written to external drive, both compared with CF and then
deleted.
As FC on CF can be slow it MAY be acceptable to FC only between
internal and external HDD.

Download is to an IBM thinkpad with a Cardbus CF reader,  carried in a
backpack when required. No extra battery yet (about 2 hours running)
but may add that too. External drive is USB2 when used.
Camera/electro geeks anonymous :-).
Ma - what's that strange man over there doing ... ?

As you can see, I'm somewhat paranoid about data loss. Several times
bitten, suitably shy.
If after the next 100,000 photos I have no losses I may get more blase
again.

On our 2003, 25 country 45,000 photo 9 week sprint I had failures of
just about every sort but due to duplicates or better at all possible
stages lost no photos apart from those on the failed 256 MB CF which I
have now largely recovered.


       RM







'[EE] Writing two CF cards at once.. .'
2006\08\27@181459 by William Chops Westfield
face picon face

On May 11, 2006, at 10:00 AM, Russell McMahon wrote:

> I'd be interested in taking the signals meant to write data
> to a CF card and using them to write to two cards in parallel
> so that I have a redundant copy.

This is a really old message, but it occurs to me that with some
of the ultra-mini flash formats like microSD/TransFlash, it might
be possible to have a CF card sized device that has sockets for
two (or more?) smaller flash format cards, plus additional logic
to emulate CF and write using assorted redundancy options...

BillW

More... (looser matching)
- Last day of these posts
- In 2006 , 2007 only
- Today
- New search...