The main problem you have is `DateTime`. Because the `DateTime` contains
functions its very hard to serialize. What are you going to do serialize the
assembly machine code of the functions? Huge security risk!
If you instead use a `float64` timestamp or my `chrono` library you would not
have this problem.
type Bar = object
d: Timestamp
o: float
h: float
l: float
c: float
v: float
Run
Next what do you want to serialize to? Json is a common format because its
human readable. I have one of the fastest json serialization library: `jsony`
let data = bars.toJson()
echo data
echo data.fromJson(seq[Bar])
Run
But really json is kind of slow. Even the fastest json is slow compared to
binary! I have a library for that too: `flatty` its faster then `jsony` but
uses its own binary format:
let data = bars.toFlatty()
echo hexPrint(data)
echo data.fromFlatty(seq[Bar])
Run
But you can go even faster with compression. Because reading and writing even
to an SSD is kind of slow, compressing your data (with fast compressor) and
writing is usually both faster and takes up less space. I recommend excellent
`supersnappy` library for this.
let data = bars.toFlatty().compress()
echo hexPrint(data)
echo data.uncompress().fromFlatty(seq[Bar])
Run
Try and see which method if fastest/easiest for you... don't forget to compile
with -d:release.
Full code:
import times, flatty, jsony, supersnappy, chrono, print, flatty/hexprint
type Bar = object
d: Timestamp
o: float
h: float
l: float
c: float
v: float
var bars: seq[Bar]
bars.add(Bar(
d: parseTs("{year}-{month}-{day}", "2000-01-01"),
o: 123.123,
h: 12.23,
l: 3.23,
c: 3.344,
v: 34.4,
))
block:
echo "using json"
let data = bars.toJson()
echo data
echo data.fromJson(seq[Bar])
block:
echo "using flatty"
let data = bars.toFlatty()
echo hexPrint(data)
echo data.fromFlatty(seq[Bar])
block:
echo "using flatty+snappy"
let data = bars.toFlatty().compress()
echo hexPrint(data)
echo data.uncompress().fromFlatty(seq[Bar])
Run
