Searching \ for '[OT] Additional info regarding database requiremen' in subject line. ()
Make payments with PayPal - it's fast, free and secure! Help us get a faster server
FAQ page: www.piclist.com/techref/mems.htm?key=data
Search entire site for: 'Additional info regarding database requiremen'.

Exact match. Not showing close matches.
PICList Thread
'[OT] Additional info regarding database requiremen'
2000\06\05@032351 by David Huisman

flavicon
face
Maybe my last post was not specific enough. Here is a more precise example
of what I need the database to operate like.

Say a region of various libraries are all part of a single database.

Top level down would be

LibraryA
 - Reference Books
    - Electronic
       - Details of book1
       - Details of book2
    - Construction
       - Details of book1
 - Periodicals
    - Monthly
       - Details of periodical1
       - Details of Periodical2
Library2 ....
 - Videos
    - Training
       - Details of video1
 - Books

etc etc.

The data saved in the eprom must reflect the above structure but the PC
database must be fully searchable. ie. Address of library1, date opened,
contact etc.

I spose it all sounds fairly straight forward but it is the COMPILING that I
have trouble with. I need an algorithm of how to search the database and
place each data block in the appropriate sub directory.

Regards
David Huisman

2000\06\05@082301 by M. Adam Davis

flavicon
face
Well, you asked for an algorith for searching the DB with a PIC?  You really
need to determine the structure of the DB first, and what info you'll have in
it.

If it is as simple as you outlined below, I would end up with two tables,
Category data and Detail data.

Each Category record would have its own unique number, and another field for the
unique number of the Category it falls under.  (backwards references)

Likewise each product(book, video, other) would reference the unique ID of the
category it falls under.

Depending on the application, this would be a good general solution.  If you
need faster searching, you'll need to create one or two more tables which will
act as forward references (category to category and product to category).

Once you decide how the data will be arranged you can think about searching.

What do you need to search for?  How fast does a search need to be?  Are there
some shortcuts to searching (ISBN for the unique number)?

It's difficult for us to come up with a search algorithm for your DB without
knowing a lot more about it.  We can come up with general searches, though
(given a list, or method of obtaining a list, return the item # that matches
this input)...

-Adam

David Huisman wrote:
{Quote hidden}

2000\06\05@120123 by David Huisman

flavicon
face
Maybe my last post was not specific enough. Here is a more precise example
of what I need the database to operate like.

Say a region of various libraries are all part of a single database.

Top level down would be

LibraryA
 - Reference Books
    - Electronic
       - Details of book1
       - Details of book2
    - Construction
       - Details of book1
 - Periodicals
    - Monthly
       - Details of periodical1
       - Details of Periodical2
Library2 ....
 - Videos
    - Training
       - Details of video1
 - Books

etc etc.

The data saved in the eprom must reflect the above structure but the PC
database must be fully searchable. ie. Address of library1, date opened,
contact etc.

I spose it all sounds fairly straight forward but it is the COMPILING that I
have trouble with. I need an algorithm of how to search the database and
place each data block in the appropriate sub directory.

Regards
David Huisman

2000\06\05@121618 by M. Adam Davis

flavicon
face
Well, you asked for an algorith for searching the DB with a PIC?  You really
need to determine the structure of the DB first, and what info you'll have in
it.

If it is as simple as you outlined below, I would end up with two tables,
Category data and Detail data.

Each Category record would have its own unique number, and another field for the
unique number of the Category it falls under.  (backwards references)

Likewise each product(book, video, other) would reference the unique ID of the
category it falls under.

Depending on the application, this would be a good general solution.  If you
need faster searching, you'll need to create one or two more tables which will
act as forward references (category to category and product to category).

Once you decide how the data will be arranged you can think about searching.

What do you need to search for?  How fast does a search need to be?  Are there
some shortcuts to searching (ISBN for the unique number)?

It's difficult for us to come up with a search algorithm for your DB without
knowing a lot more about it.  We can come up with general searches, though
(given a list, or method of obtaining a list, return the item # that matches
this input)...

-Adam

David Huisman wrote:
{Quote hidden}

2000\06\05@185557 by David Huisman

flavicon
face
The items do not actually have unique ID numbers. The data will be in a
directory that has the structure.
Section
  - Sub Section
     - Sub Sub Section
        - Data block

This data will be searched only and will always be in sequential order.
ie.
First screen shows list of sections available.
Then user selects one of them.
Now the user is presented with list of sub sections. The user chooses sub
section.
Now user is shown list of sub sub section. User chooses which sub sub
section.
Now is simply a block of data. The micro will read through the blocks and
display the first line of each section (ie. title of book). and select one.
Now the display will show all other lines of data for that item.

I was anticipating using pointers at each level.
ie. each section heading has pointer to sub section, then each sub section
has pointer to sub sub section etc.

The user can then escape back up through the levels by popping the last ptr
of a stack.

The data is static and will be read only. It will only be modified if a new
compiled version is programmed into the eprom via a PC. This new data will
have its own set of pointers.

The problem is at the higher level of creating a compiler to create this
eprom file with pointers. How to process data table. I am thinking of using
a Paradox 7 table and writing the application with Delphi 4.

I have also posted a request for help on the Delphi database newsgroup but
have no response yet.

Regards
David Huisman

2000\06\05@194217 by Brandon, Tom

flavicon
picon face
Please note, I deal only with the indexing side of things, you will still
want to divide the data into seperate tables as Adam suggested, if you
efficiently divide data so each table will primarily be serached on 1 (or 2)
fields you can use a simple Primary key for each table.

The best structure depends on how much data, the data and how you'll be
searching. If you have a natural Primary Key (a good value to serach on)
such as ISBN then you may want to order it simply by  that in a flat table
structure. A simple Primary key will also suffice if you will have small
ammounts of data and search time isn't neccesarily critical.

However, if you'll need to search lots of data quickly on different fields
(e.g. Libraries a books held at) then you may want to think about extra
indexes. If you'll often be searching on given sets of fields (e.g. Book
title, Library, Borrowed) you could hash those fields for indexinbg a
Hashtable.

If you'll be searching inmdependently on many fields you could use a B-Tree
or the like to index various fields seperately. This would speed up
searching on the PC but may slow things down on the PIC. You could have
indexes on the PC that weren't transferred to the PIC to reduce EEPROM usage
on the PIC. You will still be able to search on other fields it'll just be a
search of unordered records.

If you looking for info on B-Trees and the like look for a book on Data
Structures and Algorithms. Most Comp. Sci. courses include a course on
different storage structures (B-Trees, AVL Trees, Hashtables etc.) and
algorithms (Binary search, Insertion search etc). The textbooks for these
courses provide quite a lot of info on efficient data structures\algorithms
for different uses. Generally not aimed at PIC-level devices (e.g. assume
caches and the like) but will help with the PC and give some ideas for the
PIC. If you're not sure about concepts such as Primary\Foreign keys then I
suggest looking for some introductory database material. Articles about
writing database programs using existing database products should give you
an adequate description of basic indexing\relationships.

So, the best structure depends, among other things, on:
How much data?
How will it be searched?
Whether it will be searched in the PIC?

Tom.

{Original Message removed}

2000\06\06@084221 by M. Adam Davis

flavicon
face
David Huisman wrote:
>
> The items do not actually have unique ID numbers. The data will be in a
> directory that has the structure.
> Section
>    - Sub Section
>       - Sub Sub Section
>          - Data block

So what you are looking for are ideas on how to convert the directory structure
and info within it to a database which can be searched and used by the PIC?

{Quote hidden}

Here is the difficulty, and the reason I suggested you use backwards references
instead of forwards references:

Microchip
 12Cxxx
   508
   509
   672
 16c5x
   2
   4
   5
   6
Atmel
 ...
   ...

If the record 'Microchip' is to refer to all its children, you have to have the
ability to store at least 2 child references in its record.  If 12Cxxx is to
refer to all its children, you'll need at least 3 child references in its
record.  Where will you stop?  You may need only one record with 20 children,
but all the records in that database have that much space, so much of it is
wasted.  You can use linked lists to alleviate this issue, but then you are
trying to do normal DB activity and linked list activity.  In my estimation, it
would be better to use just one method to hold data unless there are other
reasons to do it this way.

Of course, this only works if each child has only *one* parent.  If your parents
are sharing children (which is just plain wrong... ;-) then you will need a
seperate table (which will act like a linked list, but be accessed like a DB).

Have you done much with DB design?  There are tons of good books out 'there' on
the subject, and would probably help you immensely.  While you aren't exactly
designing a huge database, the theory behind database design is quite advanced,
and you will save lots of brain cells if you read up on it before you go too
much further.

{Quote hidden}

Well, you can do this in nearly any language you like.  Pick the one you are
most familiar with.  I am not familiar with Delphi, so you are on your own
there.  There will, however, be directory functions which should return lists of
files and directories in any given directory.  You should be able to use a
recursive function to go through all the directories, gather the data, and write
one file with all the info without too much difficulty.  Unless the data is
already in Paradox, I would skip using a database program at all.  If the data
is in a database (paradox, access, other) then you can write a program/macro to
take the info and write the file directly from the database.  I understand
Paradox has a decent language, and I know access is usable in this respect.

I hope this helps.

-Adam

2000\06\06@085855 by Alan B Pearce

face picon face
It seems to me that you are making/sorting the database on a PC, and then
transferring it to a PIC to access. is this correct? If so I would use a table
to access the records.

E.G.

parent1    table
child1    pointer to text for parent1/child1
child2    pointer to text for parent1/child2
...
...
...
parent2    table
child1    pointer to text for parent2/child1
child2    pointer to text for parent2/child2
...
...

I am not a database designer, but it would seem that this should not be too hard
to organise if you are already using Delphi for the PC database. It should be
possible to write a module that would generate the necessary PIC source for you
from the database contents.

It would get more complex if the data then requires you to have grandchild
fields (i.e. a third level of selection, but this could probably still be
handled using a suitable table structure.

It may well be that with a bit of thought the text could be put in an external
large eeprom with the tables in internal program memory. you gave no specific
info about how large each record is, so this may be necessary anyway.

2000\06\06@191348 by David Huisman

flavicon
face
The structure is fixed.
It is

Parent(1)
  - Child(1)
     - Grandchild(1)
        - Data(1)
        - Data(2)
        - Data(n)
     - Grandchild(n)
  - Child(n)
Parent(2)
  - ......

The Parent/Child/Grandchild are levels only, each one only has a title and a
pointer to level below until the Grandchild level. This level then points to
the start of the first data block and each data block has a pointer to the
next data block within this grandchild.

The data blocks can vary in size at creation on the PC. The max size per
block is apprx 400 characters.

The data is being stored in a 4MB storage device.

The PIC only needs to track which item you select in each level and store
its ptr on a stack (3 bytes).
So... The levels are basically just a menu, the only data resides in the
lowest blocks.
The structure will never grow beyond the number of levels shown.

Regards
David Huisman

More... (looser matching)
- Last day of these posts
- In 2000 , 2001 only
- Today
- New search...