This slightly different from your topic, but related. Numpy has a very nice
interface for reading structured data that I've liked using. I felt like I
would learn something if I tried to do similar things in Julia.
For example
import numpy as np
# generate file to read
dt = [("i",np.int64,1),("time", "<f8",1), ("data", np.float64, 100)]
data0 = np.zeros(10, dtype=dt)
for i in range(len(data_in)):
data0[i] = (i, time.time(), np.random.rand(100))
data0.tofile("test.dat")
# read file
data = np.fromfile("test.dat",dtype=dt)
data["i"] # gives [0,1,2,3,4,5,6,7,8,9]
data[0] # (0, 1442593345.768639, [0.27580074453117376, 0.9292904639313211,,
...])
It's easy to do most of the same in julia.
# generate file like object to read
io = IOBuffer()
for i = 1:10
write(io, i)
write(io, time())
write(io, rand(100))
end
# define read type (like dtype in numpy)
immutable ReadType
i::Int
time::Float64
data::NTuple{100,Float64} # not sure how else to do fixed size array, how
would I do say a 10x10 Matrix?
end
len = position(seekend(io))
data = read(seekstart(io), ReadType, div(len,sizeof(ReadType)))
Int[d.i for d in data] # equivalent to data["i"] numpy example, not sure
why I need to specify Int here
data[1] # equivalent to data[0] in numpy example
A few things appear to be missing in my Julia version. Specification of
byte order and the data["i"] syntax. The second is easy to add.
function Base.getindex(v::Vector, s::Symbol) # not sure it's possible to
make it type stable while accepting a symbol
T=fieldtype(eltype(v), s)
T[getfield(entry,s) for entry in v]
end
data[:i] # equivalent to data["i"] in numpy
