Post by thread:[PIC] MBR isn't where I think it should be ?

part 1 1302 bytes content-type:text/plain; charset="iso-8859-1" (decoded quoted-printable) Hi all, I've returned to a project of a few weeks ago, and run straight into a problem After finding some SDHC cards I bought had partitions, I was advised to find the MBR by using the LBA information in the partition table, like so read_fat: clr sec_adr_lo ;sector 0 clr sec_adr_hi mov #ram2,w8 call getsector ;read sector into 512 bytes RAM ;find Boot Sector / Root Directory mov ram2,w0 ;test first byte of sector 0 bra z,partition ;if 0, then MBR + partition, read 0x1c6 - 0x1c9 ...... partition: mov ram2+#0x1c6,w0 ;fetch 0x1c6 and 0x1c7, LSW mov w0,sec_adr_lo mov ram2+#0x1c8,w0 ;fetch 0x1c8 and 0x1c9, MSW mov w0,sec_adr_hi mov #ram2,w8 call getsector ;read sector In the attached picture, of a 4GB, Frhed shows that the MBR is at sector 0x2000 (or 0x2000 * 0x200 bytes/sector = byte 0x400000), as 0x1c6/0x1c8 say, yet requesting that sector returns nothing What am I missing ? I've tried reseting/re-initialising the card and reading sector 0x2000 but with the same result TIA Joe part 2 5601 bytes content-type:image/gif; name="partition_2000.gif" (decode) part 3 181 bytes content-type:text/plain; name="ATT00001.txt" (decoded base64) -- http://www.piclist.com PIC/SX FAQ & list archive View/change your membership options at mailman.mit.edu/mailman/listinfo/piclist

Hi all, Supplementary issue - some cards do not appear to provide the 0xfe token to show that that data is available. I suspect this might be because of the value of the pull-up on DI. Presently I have 47k There is a final 0xff before data appears. The delay between request and that final 0xff looks about right (500us) but because the PIC is waiting for 0xfe, the data following isn't read into RAM. Does anyone have an opinion on whether the pull-up would be the cause and if so what would be a more reliable value ? Could experiment but time for only one problem at a time today TIA Joe

Hi Joe, On 19/12/2011 3.04, IVP wrote: > Hi all, > > Supplementary issue - some cards do not appear to provide the > 0xfe token to show that that data is available. I suspect this might > be because of the value of the pull-up on DI. Presently I have 47k > > There is a final 0xff before data appears. The delay between request > and that final 0xff looks about right (500us) but because the PIC is > waiting for 0xfe, the data following isn't read into RAM. Does anyone > have an opinion on whether the pull-up would be the cause and if so > what would be a more reliable value ? Could experiment but time for > only one problem at a time today I think that SD is outside the standard. You time is, a lot, more valuable so... throw it away and search for another! Bye Nic

> I think that SD is outside the standard. You time is, a lot, more > valuable so... > throw it away and search for another! The thing is, you see, I bought two of this particular card. Card #1 went into my camera, Card #2 I use for dsPIC file storage Card #2 I formatted in a card reader, its MBR ended up in sector 0. It works perfectly and I can read any file saved to it with a PC Card #1 was formatted by the camera and its MBR is (apparently) at sector 0x2000 It really puzzles me. Frhed, if dd has made a good file, shows clearly that the MBR is at sector 0x2000 and yet that sector, as can be seen in the analyser trace, appears to be full of zeroes. The transition from high to low right under the first 0 of '0000 2000' is the 0xfe token, so there's no mistake that what follows is data, and that should be the 'ex.MSDOS5.0' etc string, not zeroes Either I'm doing something wrong or I don't know something One of those days Jo

> I think that SD is outside the standard. You time is, a lot, more > valuable so... throw it away and search for another! My apology. I replied as if you'd replied to the MBR question The cards with what looks to me like an 0xff token instead of an 0xfe token are a Strontium 4GB and a Strontium 8GB micro Strontium (or at least legitimate ones) are fairly average and typical cards AFAICT. Maybe these aren't genuine, although initialisation seems OK, so they aren't complete duffers. However, initialisation is done at a much lower speed than later data transfers, so it could be that the low-side driver on DI is just weak/slow at higher speeds Maybe, as I'm not progressing with the MBR issue, I should just try a variety of pull-ups and see what the effect is. A suggestion of 50k - 100k was on a page somewhere, maybe these cards need something more like 100k than the current 47k Jo

Hi Joe, On Mon, 19 Dec 2011 23:16:20 +1300 IVP wrote: > >The thing is, you see, I bought two of this particular card. Card #1 >went into my camera, Card #2 I use for dsPIC file storage > >Card #2 I formatted in a card reader, its MBR ended up in sector 0. >It works perfectly and I can read any file saved to it with a PC >Card #1 was formatted by the camera and its MBR is (apparently) >at sector 0x2000 I think you are mixing up terms here, which would make unsterstanding and answering the actual question a bit difficult ... The sector you are looking for is the Partition-Boot-Sector (PBR), which holds the FAT specific boot and layout information. The MBR is the Master-Boot-Sector, which holds boot-code plus a table with FOUR possible partitions on the medium. Your card #2 is a card WITHOUT partitioning (no MBR) with the PBR in the very first sector. Often called 'large floppy format' since that is what the classical floppy disks used to have. Your card #1 is card with a partition table in the MBR, which is the first sector, and the first partition table entry points you to sector 0x2000 where the partition starts, and the PBR for the FAT filesystem should be ... >It really puzzles me. Frhed, if dd has made a good file, shows clearly >that the MBR is at sector 0x2000 and yet that sector, as can be seen >in the analyser trace, appears to be full of zeroes. The transition from >high to low right under the first 0 of '0000 2000' is the 0xfe token, so >there's no mistake that what follows is data, and that should be the >'ex.MSDOS5.0' etc string, not zeroes This string signature (which can have many other contents not just MSDOS) represents the 'creator' of the filesystem, it is probably wiser to check for the string at offset 0x36, which should start with uppercase 'FAT'. See the start of a FAT boot sector below, created by something else than MSDOS: ++++++++++++ 00000- eb 44 90 44 46 53 65 65 38 2e 78 00 02 40 01 00 [.D.DFSee8.x..@..] 00010- 02 00 02 00 00 f8 00 01 3f 00 f0 00 3f 00 00 00 [........?...?...] 00020- 11 e8 3f 00 80 00 29 08 ea d9 48 44 65 76 65 6c [..?...)...HDevel] 00030- 6f 70 20 20 20 20 46 41 54 20 20 20 20 20 00 00 [op FAT ..] ++++++++++++ It has the creator signature 'DFSee8.x' instead of 'MSDOS5.0' Of course this does not help you with finding out why reading that sector 0x2000 does not seem to work, but it may help in the discussion ... One other thing that may go wrong in some cases, you seem to be checking the very first byte of the first sector being ZERO to determine wether it is an MBR sector with partition tables or a FAT-PBR. That is NOT very reliable, although it may work on many SD cards. But many MBR sectors DO have (PC) executable code there, often starting with a JUMP instruction. You need somewhat more sophisticated checks to see if it is an MBR. Regards, JvW Jan van Wijk, author of DFSee; http://www.dfsee.com ------------------------------------------------------------------------------

At 07.27 2011.12.20, you wrote: {Quote hidden}>Hi Joe, > >On Mon, 19 Dec 2011 23:16:20 +1300 IVP wrote: >> >>The thing is, you see, I bought two of this particular card. Card #1 >>went into my camera, Card #2 I use for dsPIC file storage >> >>Card #2 I formatted in a card reader, its MBR ended up in sector 0. >>It works perfectly and I can read any file saved to it with a PC > >>Card #1 was formatted by the camera and its MBR is (apparently) >>at sector 0x2000 > >I think you are mixing up terms here, which would make unsterstanding >and answering the actual question a bit difficult ... > >The sector you are looking for is the Partition-Boot-Sector (PBR), >which holds the FAT specific boot and layout information. > >The MBR is the Master-Boot-Sector, which holds boot-code plus >a table with FOUR possible partitions on the medium. > > >Your card #2 is a card WITHOUT partitioning (no MBR) with the PBR >in the very first sector. Often called 'large floppy format' since that >is what the classical floppy disks used to have. > > >Your card #1 is card with a partition table in the MBR, which is the >first sector, >and the first partition table entry points you to sector 0x2000 where the >partition starts, and the PBR for the FAT filesystem should be ... > > >>It really puzzles me. Frhed, if dd has made a good file, shows clearly >>that the MBR is at sector 0x2000 and yet that sector, as can be seen >>in the analyser trace, appears to be full of zeroes. The transition from >>high to low right under the first 0 of '0000 2000' is the 0xfe token, so >>there's no mistake that what follows is data, and that should be the >>'ex.MSDOS5.0' etc string, not zeroes > > >This string signature (which can have many other contents not just MSDOS) >represents the 'creator' of the filesystem, it is probably wiser to check for >the string at offset 0x36, which should start with uppercase 'FAT'. >See the start of a FAT boot sector below, created by something else than MSDOS: > >++++++++++++ >00000- eb 44 90 44 46 53 65 65 38 2e 78 00 02 40 01 00 [.D.DFSee8.x..@..] >00010- 02 00 02 00 00 f8 00 01 3f 00 f0 00 3f 00 00 00 [........?...?...] >00020- 11 e8 3f 00 80 00 29 08 ea d9 48 44 65 76 65 6c [..?...)...HDevel] >00030- 6f 70 20 20 20 20 46 41 54 20 20 20 20 20 00 00 [op FAT ..] >++++++++++++ > >It has the creator signature 'DFSee8.x' instead of 'MSDOS5.0' > > >Of course this does not help you with finding out why reading that >sector 0x2000 >does not seem to work, but it may help in the discussion ... > > >One other thing that may go wrong in some cases, you seem to be checking >the very first byte of the first sector being ZERO to determine wether it is >an MBR sector with partition tables or a FAT-PBR. > >That is NOT very reliable, although it may work on many SD cards. >But many MBR sectors DO have (PC) executable code there, often >starting with a JUMP instruction. >You need somewhat more sophisticated checks to see if it is an MBR. The best way would be to use the same algorithm that the OS uses itself to make this distinction. But I don't have the details anymore.. lost them in a HD crash. > >Regards, JvW > > > >Jan van Wijk, author of DFSee; http://www.dfsee.com >------------------------------------------------------------------------------- >

Hi Jan > The sector you are looking for is the Partition-Boot-Sector (PBR), > which holds the FAT specific boot and layout information > The MBR is the Master-Boot-Sector, which holds boot-code plus > a table with FOUR possible partitions on the medium. As I understood it, sector 0 would contain either the MBR, containing the creator string and sector address of the directory or, if partitioned, at 0x1c6-0x1c9 would hold the sector address of where the MBR is. You then read that sector to find the directory etc > Of course this does not help you with finding out why reading that > sector 0x2000 doesn't seem to work, but it may help in the discussion ... No, I'm back on it tonight. It's frustrating after success with cards that don't have partitions. The rest of the s/w is written, if only I could get past this and reliably find the directory I take your point about a more sophisticated check for the MBR in sector 0. With the cards I have to hand obviously I can see what the contents are, but something better will be needed when it's not my card in the product Thanks > The best way would be to use the same algorithm that the OS uses > itself to make this distinction Hi Electron, I'm looking for that

Hi Joe, On 20/12/2011 8.54, IVP wrote: > As I understood it, sector 0 would contain either the MBR, containing the > creator string and sector address of the directory or, if partitioned, at > 0x1c6-0x1c9 would hold the sector address of where the MBR is. You then read > that sector to find the directory etc If the first byte is 0xEB or 0xE9 (long or short jump) usually you have a Boot Record. If the first byte is 0x00 or 0x33 (first byte of XOR AX,AX) usually is a MBR. But if you have an SD prepared with Grub will find 0xEB at the start of MBR.... this is unusual! >> Of course this does not help you with finding out why reading that >> sector 0x2000 doesn't seem to work, but it may help in the discussion .... Try to read sequentially from first sector and stop when you find the boot sector: you can identify it, in first instance, because of his signature: 0x55 and 0xAA the last two byte. Maybe this can help you to find something...

MBR = Master Boot Record. This should be at the physical sector 0. That contains a small boot program that loads the first active paritition's Boot Sector plus the Primary Partition Table. At the 0x1BE you can see that partition information which lead you to the Logical Block Address (LBA) 0x2000 within the SD card. According to the picture you have provided this is the Boot Sector of the partition, so your calculation was correct. If there is an extended partition, you would see another Boot Record with a partition table information in it. Tamas On 19 December 2011 02:53, IVP <spam_OUTjoecolquittTakeThisOuTclear.net.nz> wrote: {Quote hidden}> Hi all, > > I've returned to a project of a few weeks ago, and run straight into > a problem > > After finding some SDHC cards I bought had partitions, I was advised > to find the MBR by using the LBA information in the partition table, like > so > > read_fat: clr sec_adr_lo ;sector 0 > clr sec_adr_hi > mov #ram2,w8 > call getsector ;read sector into 512 bytes RAM > > ;find Boot Sector / Root Directory > > mov ram2,w0 ;test first byte of sector 0 > bra z,partition ;if 0, then MBR + partition, read > 0x1c6 - 0x1c9 > ...... > > partition: mov ram2+#0x1c6,w0 ;fetch 0x1c6 and 0x1c7, LSW > mov w0,sec_adr_lo > mov ram2+#0x1c8,w0 ;fetch 0x1c8 and 0x1c9, MSW > mov w0,sec_adr_hi > mov #ram2,w8 > call getsector ;read sector > > In the attached picture, of a 4GB, Frhed shows that the MBR is at > sector 0x2000 (or 0x2000 * 0x200 bytes/sector = byte 0x400000), > as 0x1c6/0x1c8 say, yet requesting that sector returns nothing > > What am I missing ? I've tried reseting/re-initialising the card and > reading sector 0x2000 but with the same result > > TIA > > Joe >

> MBR = Master Boot Record. This should be at the physical sector 0 Thanks for confirming that Tamas. I've spent some of today on it but no joy I'm still trying to figure where the problem is (a) the PIC - on the same voltage (3.3V) as the card and seems to drive CLK and DO alright. Fairly sure I can rule the PIC out (b) the card - well, sector 0 can be read, and no error is returned when requesting sector 0x2000. But maybe the internal controller isn't requesting 0x2000. Perhaps for some reason it's ignoring the address I send it and somehow auto-incrementing, fetching sector 1, which will be empty A project I completed recently with a similar card has no problems at all. The only difference being that it isn't partitioned (unfortunately I don't have access to that card until the Christmas break so I can compare). I can read its directory and retrieve any sector/file at a good speed and with no errors Think I'll have to go back to the SD Association's pdfs and see if there's some nuance I've missed The ultimate aim is to investigate 4-bit transfers, so not being able to get it working with simple SPI is a bit of a choker Jo

Hi Joe, On Tue, 20 Dec 2011 21:54:34 +1300 IVP wrote: > >> The MBR is the Master-Boot-Sector, which holds boot-code plus >> a table with FOUR possible partitions on the medium. > >As I understood it, sector 0 would contain either the MBR, containing >the creator string and sector address of the directory or, Yes, but you are still mixing up the terminology, this is the PBR. The Partition-Boot-Sector, which holds bootcode and basic information about the filesystem (FAT) that is there, including the info you need to find the root-directory etc. >if partitioned, at 0x1c6-0x1c9 would hold the sector address of where the MBR is. Eh, the PBR :-) In this case sector 0 IS an MBR, and 0x1c6 is the field inside the partition-table that holds the LBA (linear) sector address of the PBR, which is the first sector of the partition with the actual filesystem (FAT) in there ... >You then read that sector to find the directory etc Exactly. Note that interpreting that PBR to find the root-directory and everything else, is slightly different for a FAT32 filesystem, which you likely will encounter on any card larger than 2Gb (FAT16 is limited to 2Gb). For FAT16 the root-directory is always in cluster 2, and you can calculate the first sector easily by using the size of the FAT areas and an initial offset which are both available in the PBR sector. For FAT32 there is additional information about the clusternumber where you can find the root directory (which is usually 2 anyway, but can be different!). You may also need to interpret the 'long filenames' (longer than the 8.3 convention) that can be used on some FAT16 and all FAT32 directories which is non trivial. If you need additional info about this (after Googling for it perhaps :-) contact me offline, and I will dig some stuff up from my own code. (You can use 'jvw' at 'dfsee' dot 'com' to contact me directly) >I take your point about a more sophisticated check for the MBR in >sector 0. With the cards I have to hand obviously I can see what the >contents are, but something better will be needed when it's not my >card in the product Indeed, I have collected some logic below, used in my own disk-analysis tool, feel free to use that anyway you like (some of it may be overkill for you) First of all, any bootsector (MBR or PBR) must have the unique 0x55 + 0xAA signature byte at the end of the sector (at 0xFE, or decimal offset 510). Assuming your main interest is with FAT filesystems (FAT16 and FAT32) you could check for that first, to see if sector 0 is a PBR for a FAT filesystem. FAT16 filesystems have the uppercase string 'FAT' at offset 0x36 FAT32 filesystems have the uppercase string 'FAT32' at offeset 0x52 If that is not the case, it will be either an MBR, or a bootsector for an entirely different filesystem, or jst empty/garbage. In theory, there could be FAT filesystems that do NOT use these identification strings, but that is extremely rare, and will confuse most disk tools and drivers. To check if it is a valid MBR sector, with a partition table: 1) First byte, offset 0x00 is almost always (99.9%): 0x33, 0xfa or 0x00 2) There should be some non-zero bytes between 0x1be and 0x1fd (which is the partition table, 4 entries of 16 bytes each) 3) The first byte of each entry can only have a few bits set: mask 0x85 So you could do a test like ((contents of 0x1be) & 0x7a) should be 0 Same check for 0x1ce, 0x1de and 0x1ee Checking this filters out posible garbage sectors that vaguely resemble an MBR :-) 4) One of the four entries must have a valid partition type, which will most likely have value 0x04, 0x06 (for FAT16) or 0x0c or 0x0b for FAT32. (but for simplicity, you could just check for any non-zero value). 0x1c2 is the offset for the partition type in the first entry, 0x1d2, 0x1e2 and 0x1f2 are the other three. On most (single partition) SD cards, only the first entry will be used.. 5) The entry (or entries) with a valid partition type should also have an non-zero LBA sector address, that points you to the PBR sector. 0x1c6 is the offset for the LBS sector address in the first entry, 0x1d6, 0x1e6 and 0x1f6 are the other three. The LBA address is a 32 bit number (unsigned long) with the lsb first (standard intel-reversed format) 6) The four bytes after the LBA are the partition size in sectors, and should be non-zero as well, but you probably do not require this value, so checking it is sort of optional (but can't hurt). So if you want to do it proper, check all four entries,if you are cramped for code space, you may resort to just checking the first entry, but that may cause some obscure cards to appear 'empty' then ... There is some info on partitioning (probably more than you need to know) at: http://www.dfsee.com/present/pcpartit/pcpartit.pdf and on FAT (and several other) filesystems at: http://www.dfsee.com/present/fsystems/fsystems.pdf When playing arround with SD cards and disks in general, feel free to use my disk analysis tool, that was specifically made for low level analysis and display of partitioning and filesystem structures. The latest download always allows an evaluation period of several months, so no need to buy a registration any time soon (or at all). (unless you are going to use it large scale in a commercial setting :-) It can display contents of MBR and PBR sectors (formatted with all fields) and also FAT area, directories and file contents if you wish. But it also includes a binary sector editor for low level analysis, including a disassembler for x86 (PC) code in bootsectors etc. You can download the latest official release from: http://www.dfsee.com/dfsee/dfsee109.zip or use the Windows installer: http://www.dfsee.com/dfsee/dfsee109.msi Or get the release-candidate for version 11.0 (to be released next week or so): http://www.dfsee.com/download/dfsee110.zip The ZIPs contain executable versions for DOS (32bit), Windows, Linux and OS/2. Note thet the tool was made for developers and technicians, not novices and is TEXT mode, but does include a menu/dialog user interface to make using it easier. Regards, JvW Jan van Wijk, author of DFSee; http://www.dfsee.com ------------------------------------------------------------------------------

Hi Joe, On Wed, 21 Dec 2011 22:50:14 +1300 IVP wrote: > >(b) the card - well, sector 0 can be read, and no error is returned >when requesting sector 0x2000. But maybe the internal controller >isn't requesting 0x2000. Perhaps for some reason it's ignoring the >address I send it and somehow auto-incrementing, fetching sector 1, >which will be empty You can use a binary sector editor to check the actual contents of these sectors (0x2000, and perhaps 0x0001) and even put some unique string contents in there just to be able to recognize them in testing ... Regards, JvW Jan van Wijk, author of DFSee; http://www.dfsee.com ------------------------------------------------------------------------------

> Jan van Wijk, author of DFSee; http://www.dfsee.com Thank you very much for your detailed reply, all good stuf

> You can use a binary sector editor to check the actual contents > of these sectors (0x2000, and perhaps 0x0001) and even put > some unique string contents in there just to be able to > recognize them in testing ... I was thinking along the same lines. With a pretty much blank card it's impossible to tell what sector is actually being fetched when you just get zeroes But you know how it goes. "I'll just try this first" ..... and 4 hours later you're still reading zeroe

> You can use a binary sector editor to check the actual contents > of these sectors (0x2000, and perhaps 0x0001) and even put > some unique string contents in there just to be able to > recognize them in testing ... > You can use a binary sector editor to check the actual contents > of these sectors (0x2000, and perhaps 0x0001) and even put > some unique string contents in there just to be able to > recognize them in testing ... The card comes up as I: in a card reader And I do this with dd to get 50M off it dd if=\\?\Device\Harddisk5\Partition0 of=d:\dd\card.img bs=1M count=50 --progress After filling selected sectors with known values with Frhed, would I use this to write it back to the card ? dd if=c:\dd\card.img of=\\?\Device\Harddisk5\Partition0 bs=1M count=50 --progress Just checking. The last thing I need is a wayward write to one of the HDD Joe

Hi Joe, On Thu, 22 Dec 2011 14:01:12 +1300 IVP wrote: > >> You can use a binary sector editor to check the actual contents >> of these sectors (0x2000, and perhaps 0x0001) and even put >> some unique string contents in there just to be able to >> recognize them in testing ... > >The card comes up as I: in a card reader > >And I do this with dd to get 50M off it > >dd if=\\?\Device\Harddisk5\Partition0 of=d:\dd\card.img bs=1M >count=50 --progress OK >After filling selected sectors with known values with Frhed, would >I use this to write it back to the card ? > >dd if=c:\dd\card.img of=\\?\Device\Harddisk5\Partition0 bs=1M >count=50 --progress I would think so, but I have never used dd on Windows since I have my own tool for that :-) >Just checking. The last thing I need is a wayward write to one of the HDD Hmm, I understand it is just as scary to use an unfamiliar tool then it is to use dd, but with DFSee, you just 'open' the disk, the start the binary editor. File -> Open object to work with -> Disk -> ... select disk from list ... (identify on size usually) Edit -> Sector, HEX/ASCII This will start the binary edit on sector 0 of the selected disk. It will write back to the same disk (and sectors) as it was reading from .... You can edit the sector contents either in HEX or in ASCII (tab to right pane) and it will prompt to write back as soon as you navigate to the next sector.. Regards, JvW Jan van Wijk, author of DFSee; http://www.dfsee.com ------------------------------------------------------------------------------

> Hmm, I understand it is just as scary to use an unfamiliar tool > then it is to use dd, but with DFSee, you just 'open' the disk, > the start the binary editor. I've downloaded your DFSee suite and will look into it over the next few days. Many thanks Of the 4 cards I have here, all of them work OK in a card reader/ writer and a digital DSLR, both of which use the native 4-bit Using the PIC however - Physon 4GB, not partitioned - fine (first one I bought, was lucky I started development with it and had immediate and encouraging success) Physon 4GB, partitioned - troubles (0x2000) Strontium 4GB partitioned - troubles (0x2000 and possibly pull-up value) Strontium 8GB partitioned - tried it this morning, seems fine. The 0x2000 issue with the other two is not apparent (same DI pull-up) and MBR and directory were found first go with existing s/w I don't know if it's just coincidence that the two 4GB with partitions are problematic. Perhaps SPI is not that good for general purpose and they have a common or similar design SPI controller Jo