Please read inline.
On 9/18/06, David Moss <[EMAIL PROTECTED]> wrote:
Yes. Excuse the slip please.
This is unexpected - I understood that the footprint could actually be made to grow/shrink on each open call by passing an appropriate mimimumSize. Does that hold for (only) regular files?
That explains. I was expecting the size of only valid values/key-values, since that is actually the logical written data length, whereas what's being returned is actually the physical written data length. I understand that the physical written data included exactly keys and values - no meta information is written to that reserved space, and is therefore not included in the return value of getDataLength. Please correct me if I'm wrong.
Not necessarily. The reserved locations not yet written to contain alien data (or maybe FF's), and once the file is out of Blackbook, there's no way to determine the end of "real" data and start of alien data. This may be a trivial issue for ASCII files being manually interpreted back at the computer, but I'm using the file to form network packets.
This seems an accurate value. For now, I can work with the limitation that it's accurate only at the time of open call. I have another question here - is totalSize the ceil-to-page-multiple value of reservedLength or it's the same as minimumSize at creation (and correspondingly for remainingBytes)? The difference will be same either way, so I can still use it.
Repeating my question, is this the behavior for non-dictionary files also?
If the open-time (totalSize - remainingBytes) doesn't work good enough, I'll implement this. A more efficient scheme that occurs to me is to keep a RAM counter throughout the open-time of the file that keeps updating, and is initialized at open either through the traversal or through it's last value stored in flash metadata. The latter would be messy at application level, but easier (my guess) at Blackbook level.
Alot! And Blackbook is working, yeah. It's taken off a lot of development time, thanks! And thanks for your detailed reply.
Manu.
Hi Manu,
> In Blackbook5, what does BDictionary.getDataLength return?
> .. what should minimumSize include?
First, are you referring to the BFileDir.getDataLength(char *filename)
command, being run on a Dictionary file?
Yes. Excuse the slip please.
Dictionary files have special properties to them that allow them to be
updated indefinitely. The way it works is when you first open a Dictionary
file, you must specify the minimum size. On the flash, Blackbook will
reserve *at least* that minimum size to store data - but if you want the
background information on that, the actual size on flash will extend to the
next page boundary. This can make your actual file size up to 255 bytes
larger than you requested. Better a file be made a few bytes too big than
too small.
After you have opened a Dictionary file for the first time, its footprint on
flash will remain constant throughout its lifetime. The dictionary file
itself will physically move around to various locations on flash as it fills
up and removes invalid keys throughout its lifetime, but the valid
(undeletable) footprint on flash will always stay the same.
This is unexpected - I understood that the footprint could actually be made to grow/shrink on each open call by passing an appropriate mimimumSize. Does that hold for (only) regular files?
Because flash can't actually "delete" data very easily, is better to leave
old data written to flash and just ignore it. That means that as you delete
or update existing keys in your Dictionary file, the amount of data in that
Dictionary file is growing, not shrinking or changing. There comes a point
where the amount of data (invalid + valid data) in the Dictionary file fills
up to the size you requested when you first created the file. At that
point, the Dictionary file will physically move to another location on flash
and copy all the previous valid key-values over, leaving all the old
key-value pairs behind (to be deleted by the garbage collector when the time
is right).
So... if you call getDataLength(..) on your dictionary file, you'll see the
size is changing all over the place as you add or update keys - or maybe it
changes only after you've closed and opened the file again. However it does
That explains. I was expecting the size of only valid values/key-values, since that is actually the logical written data length, whereas what's being returned is actually the physical written data length. I understand that the physical written data included exactly keys and values - no meta information is written to that reserved space, and is therefore not included in the return value of getDataLength. Please correct me if I'm wrong.
it, a better indicator of Dictionary file size is getReservedLength(..),
which tells you the dictionary file's footprint on flash. If you were to
copy a complete dictionary file from flash to your computer, you'd want to
copy its entire footprint, which is getReservedLength(..). The reserved
Not necessarily. The reserved locations not yet written to contain alien data (or maybe FF's), and once the file is out of Blackbook, there's no way to determine the end of "real" data and start of alien data. This may be a trivial issue for ASCII files being manually interpreted back at the computer, but I'm using the file to form network packets.
length should remain constant throughout the file's lifetime, as I said.
But it could be much bigger than the amount of data actually written to the
dictionary file.
Also, when you call Bdictionary.open, you get an opened() event with
uint32_t totalSize, uint32_t remainingBytes. These two variables give you a
snapshot of the state of the Dictionary file, but it's only valid at the
time you open the Dictionary file. The amount of actual data in the
Dictionary file should be (totalSize - remainingBytes).
This seems an accurate value. For now, I can work with the limitation that it's accurate only at the time of open call. I have another question here - is totalSize the ceil-to-page-multiple value of reservedLength or it's the same as minimumSize at creation (and correspondingly for remainingBytes)? The difference will be same either way, so I can still use it.
Back to the minimumSize parameter on open... Because the Dictionary file's
footprint remains constant throughout its lifetime, that means that the
minimumSize parameter in Bdictionary.open(..) gets completely ignored if
it's not the first time you've opened that specific Dictionary file. The
minimumSize parameter is only accessed the first time you open a Dictionary
file.
Repeating my question, is this the behavior for non-dictionary files also?
The larger you make the minimumSize value when you call Bdictionary.open for
the first time, then A) you'll be able to store more and larger key-value
pairs in your file, and B) it's going to take more time and energy to insert
and retrieve key-value pairs as your Dictionary file fills up.
And yes, the only real way to count the number of keys is to actually
traverse the key list like you mentioned. If Blackbook provided that
functionality itself, that's exactly what it would do. And since that's not
needed in all applications, it's better to save ROM space for the apps that
don't need it and only implement it at a higher level above Blackbook for
the apps that do need it.
If the open-time (totalSize - remainingBytes) doesn't work good enough, I'll implement this. A more efficient scheme that occurs to me is to keep a RAM counter throughout the open-time of the file that keeps updating, and is initialized at open either through the traversal or through it's last value stored in flash metadata. The latter would be messy at application level, but easier (my guess) at Blackbook level.
I hope that clears things up and I also hope Blackbook is working out for
you. Let me know if you have any more questions
Alot! And Blackbook is working, yeah. It's taken off a lot of development time, thanks! And thanks for your detailed reply.
Manu.
-David
-----Original Message-----
From: Manu Bansal [mailto: [EMAIL PROTECTED]]
Sent: Monday, September 18, 2006 11:50 AM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Blackbook5: dataLength, minimumSize for dictionary files
Dear Mr. Moss,
I am an undergraduate student at Indian Institute of Technology Kanpur (IIT
Kanpur), working on a sensor network project and using Blackbook5. There's a
question to which I couldn't find an answer in the included pdf or the
interface inline comments. I couldn't get any help from the tinyos-help list
either, hence a personal mail.
In Blackbook5, what does BDictionary.getDataLength return? Is it the length
of
data inserted as values, or keys and values both, or other? And similarly,
what should minimumSize include? For regular files, the semantics is clear,
but I'm not sure about dictionary files, and getDataLength doesn't return
the
expected value. For 5 key-value pairs each having value 12B long, the
returned dataLength is 1D/1E/14 on different occasions. Being able to obtain
the no. of entries in a dictionary file will also solve my problem. The only
way I can think to do this is to traverse the key-link-list.
With warm regards and thanks,
Manu Bansal.
--
Manu Kumar Bansal
Fourth Year Undergraduate
B.Tech-M.Tech Dual Degree
Computer Science and Engineering
IIT Kanpur, Kanpur, India - 208016
http://home.iitk.ac.in/~manub
_______________________________________________
Tinyos-help mailing list
[email protected]
https://mail.millennium.berkeley.edu/cgi-bin/mailman/listinfo/tinyos-help
_______________________________________________ Tinyos-help mailing list [email protected] https://mail.millennium.berkeley.edu/cgi-bin/mailman/listinfo/tinyos-help
