Joe,
Thanks for the advice on using specialized-arrays. I started to use them, but
then realized that what I wanted to do was more dynamic in nature. I have this
file of log entries where each entry consists of a fixed size bunch of
bookkeeping data followed by a blob of application dependent bytes. Different
log files may have different sized blobs of application data for their entries.
Thus I could not create one struct that captured all of the possibilities, and
thus could not think of a clean way to use a specialized array. So, instead
what I do now is divide my 128 K blocks into "groups" (using the "group" word)
at runtime. Then I can create a TUPLE from each group. The TUPLE is defined
as my known fixed-size bookkeeping struct and a byte-array. Here is some of
the code for what I did (ignore any Factor newbie ugliness please :)):
STRUCT: event-entry
{ uDateTime alien.c-types:uint }
{ uSystemTick alien.c-types:uint }
{ uTimeStampLSW alien.c-types:uint }
{ uTimeStampMSW alien.c-types:ushort }
{ uWindowID alien.c-types:char }
{ cEventString alien.c-types:char[17] } ;
TUPLE: event-record
{ header event-entry }
{ data byte-array } ;
: set-data-size ( size -- )
data-size set ;
: block-to-event-record ( a-byte-block -- an-event-record )
dup event-entry memory>struct
swap
data-size get tail
event-record boa ;
! The the-byte-array object on stack is a 128 K array of bytes.
! The global symbol data-size contains the size of the user "blob" in bytes
!
:: block-to-records ( the-byte-array -- event-record-array )
event-entry alien.c-types:heap-size data-size get + :> entry-size
BLOCKSIZE entry-size / :> entries-in-block
! Check whether there is any remaining extra data in the block that
! should be ignored
entries-in-block dup truncate =
[ the-byte-array entry-size group ]
[ the-byte-array entry-size group entries-in-block truncate head ] if
! break
[ block-to-event-record ] map ;
This seems to be working for me. To my surprise, it also seems to perform
rather well. I expected all of the "group" calls to make this code really
slow. But it seems to run faster than I ever expected.
From my miniscule knowledge of Factor, it does seem like there ought to be a
way to define a new struct type completely at runtime, but I cannot figure out
how to do that. That would be a cool thing to learn how to do someday. Then
I COULD use specialized-arrays and also have a completely dynamic log file
reader which could read any log file with different sizes of user "blob" data
as long as user code tells me what the user data-size is before the struct is
defined at runtime (by calling set-data-size).
Thanks again,
John Whittaker
---- Joe Groff <[email protected]> wrote:
> On Jan 4, 2011, at 11:18 AM, John Whittaker wrote:
>
> > I have been able to figure out how to read a large block of these into a
> > big byte array (128 K). What I would next like to do is somehow iterate
> > over this byte-array and create a sequence of event-entry structures.
> > Unfortunately I cannot figure out how to do this. I have been able to
> > create a single instance of an event-entry using memory>struct with the
> > 128 K byte-array.
>
> You want to create a specialized array of event-entry structs over your
> byte-array. Create a specialized-array instance for your struct type as
> follows:
>
> --
> USE: specialized-arrays
> SPECIALIZED-ARRAY: event-entry
> --
>
> Then use the generated word «event-entry-array-cast» to wrap your byte-array
> in an event-entry-array. You can then iterate over the event-entry-array as a
> Factor sequence. For more information on specialized arrays, use
> «"specialized-array" help» to bring up the documentation.
>
> -Joe
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment, and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Factor-talk mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/factor-talk
------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Factor-talk mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/factor-talk
