You can pretty easily just read the pieces out of the file stream one at a
time. E.g. if the file starts with the four-byte magic sequence "FuZz" and
then has two big-endian two-byte words, and a bunch of bytes, you can do
this:
# create a "file" in memory:
julia> io = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true,
append=false, size=0, maxsize=Inf, ptr=1, mark=-1)
julia> write(io, "FuZz")
4
julia> write(io, hton(UInt16(12)), hton(UInt16(34)))
4
julia> data = rand(UInt8, 12, 34)
12x34 Array{UInt8,2}:
0x32 0x12 0xc5 0x8d 0x30 0x32 … 0x99 0x57 0x41 0x14 0xb1 0x28
0x8f 0x0b 0x8a 0x81 0x1c 0x53 0xc5 0x9b 0x2b 0x88 0x87 0x6f
0x1e 0xff 0xb1 0xac 0x74 0x08 0x1a 0x61 0x6a 0x54 0x8c 0x25
0xca 0x70 0x87 0x9d 0x44 0xc7 0x48 0x62 0x10 0xf2 0x3e 0x40
0xce 0x39 0x23 0xc9 0x54 0x15 0x8d 0xfd 0x32 0xfe 0xab 0x00
0x0c 0xd1 0x86 0x66 0x06 0xa9 … 0x58 0x4f 0x45 0x4c 0x7e 0xe3
0x1e 0x98 0xde 0x87 0x71 0x14 0x65 0x5b 0x0f 0xdb 0x5b 0xc5
0x42 0xc1 0x75 0xc5 0x8d 0xd8 0x91 0x5d 0xce 0xa5 0x84 0x58
0xf5 0xd7 0xdf 0x71 0x65 0x6e 0xd2 0xc8 0xec 0xcf 0x46 0xc7
0x64 0x88 0x57 0x58 0x3f 0x5b 0x41 0xad 0x14 0xf8 0x03 0xf4
0xa0 0xb5 0x42 0xed 0xed 0x80 … 0xb4 0xe3 0x5e 0xa7 0xde 0xa3
0x6a 0x30 0x15 0xd9 0xe5 0xbd 0x17 0x3b 0xfd 0x6f 0x4e 0x98
julia> write(io, data)
408
julia> seek(io, 0)
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true,
append=false, size=416, maxsize=Inf, ptr=1, mark=-1)
# now read data back from that file:
julia> readbytes(io, 4) == b"FuZz" || error("invalid magic bytes")
true
julia> m = ntoh(read(io, UInt16))
0x000c
julia> n = ntoh(read(io, UInt16))
0x0022
julia> data = Array(UInt8, m, n);
julia> read!(io, data)
12x34 Array{UInt8,2}:
0x32 0x12 0xc5 0x8d 0x30 0x32 … 0x99 0x57 0x41 0x14 0xb1 0x28
0x8f 0x0b 0x8a 0x81 0x1c 0x53 0xc5 0x9b 0x2b 0x88 0x87 0x6f
0x1e 0xff 0xb1 0xac 0x74 0x08 0x1a 0x61 0x6a 0x54 0x8c 0x25
0xca 0x70 0x87 0x9d 0x44 0xc7 0x48 0x62 0x10 0xf2 0x3e 0x40
0xce 0x39 0x23 0xc9 0x54 0x15 0x8d 0xfd 0x32 0xfe 0xab 0x00
0x0c 0xd1 0x86 0x66 0x06 0xa9 … 0x58 0x4f 0x45 0x4c 0x7e 0xe3
0x1e 0x98 0xde 0x87 0x71 0x14 0x65 0x5b 0x0f 0xdb 0x5b 0xc5
0x42 0xc1 0x75 0xc5 0x8d 0xd8 0x91 0x5d 0xce 0xa5 0x84 0x58
0xf5 0xd7 0xdf 0x71 0x65 0x6e 0xd2 0xc8 0xec 0xcf 0x46 0xc7
0x64 0x88 0x57 0x58 0x3f 0x5b 0x41 0xad 0x14 0xf8 0x03 0xf4
0xa0 0xb5 0x42 0xed 0xed 0x80 … 0xb4 0xe3 0x5e 0xa7 0xde 0xa3
0x6a 0x30 0x15 0xd9 0xe5 0xbd 0x17 0x3b 0xfd 0x6f 0x4e 0x98
On Thu, Sep 17, 2015 at 9:22 PM, Tom Breloff <[email protected]> wrote:
> I have an alternative to StrPack.jl here:
> https://github.com/tbreloff/CTechCommon.jl/blob/master/src/macros.jl. If
> you have a type that mimics a c-struct, you can create like:
>
> @packedStruct immutable MyStruct
> field1::UInt8
> field2::Byte
> end
>
> and it creates some methods: read, reinterpret, etc which can convert raw
> bytes into the immutable type.
>
> I've only used it on very specific types of data, but it may work for you.
>
> On Thu, Sep 17, 2015 at 6:04 PM, David McInnis <[email protected]> wrote:
>
>> I'm in the process of switching from python to julia and have gotten
>> stuck for a couple of days trying to read a, for me, typical data file.
>>
>> In python I'd create a C-style format, open the file and read the data.
>> I don't see an equivalent method in Julia.
>>
>> Ex:
>> Using a data structure of something like: "<4sii28s4i"
>> I'd figure out the size of the structure, point to the beginning byte,
>> and then unpack it.
>>
>> In Julia it looks like *maybe* I could make a data type to do this, but I
>> can't figure out how.
>> There's also StrPack.jl, but it too is a little beyond what I understand.
>>
>> I work with a lot of different instruments, each with its own file
>> format. Usually I only need to read these files. After processing I'll
>> save everything into an hdf5 file.
>>
>> Thanks, David.
>>
>>
>
