Hello,
I have some code that I am trying to understand why it is slower than
Python/Numpy. I am not a C programmer and Nim is my first real attempt to learn
a systems level, statically compiled language. But to my understanding Nim
should approach C speeds to a certain extent. If what I present can be improved
please let me know. I want to learn and I want to use Nim correctly.
I know Python/Numpy has been well optimized for certain tasks. I understand my
Nim code may be naive. I would love to see the Nim code that competes with the
Numpy code. I want to understand how to write it well.
Ubuntu 18.04 x86-64
Python/Numpy perfoms this at 0:50.0. Nim performs this at: 1:43.151
When I remove the sum() code Nim is 5x faster.
I need fast summing and averaging of slices of arrays. Numpy does this
exceptionally.
I know about Arraymancer but am lost at looking at it. I do not do high level
maths. I only need simple arrays and accessing, summing, averaging, slicing and
doing similar on the slices. I do not need any matrix type operations.
I am okay with doing what I need with Arraymancer should it be the solution to
speed up my app. But I am currently clueless on how to do so with the below
code.
Any help in accomplishing these goals and competing reasonably with
Python/Numpy would be greatly appreciated. I can easily complete my app in
Python. And I may do so. But I would like either initially or app v2.x to be
able to do it in something like Nim.
I understand very well I may have done something stupid and/or naive. I am
willing to learn.
Thanks.
#latest nim dev
proc pricesFromGainCsv*(fullpath: string): array[337588, float] =
let csvfile = open(fullpath, fmRead)
let contents = csvfile.readAll()
let lines = contents.splitLines()
for i in 1..<lines.len:
let line = lines[i]
if line.len > 0:
let parts = line.split(",")
result[i-1] = parseFloat(parts[5])
csvfile.close()
var ctime = getTime()
echo("time: ", ctime)
var cpustart = cpuTime()
#http://ratedata.gaincapital.com/2018/12%20December/EUR_USD_Week1.zip
let gaincsvpath = "/home/jimmie/data/EUR_USD_Week1.txt"
let prices = pricesFromGainCsv(gaincsvpath)
var psum = 0.0
var pcount = -1
var pips = 0.0
var psumsum = 0.0
var farray: array[prices.len-1, float]
for price in prices:
pcount += 1
farray[pcount] = ((price * price) * ((pcount+1) / prices.len))
pips += price
psum = farray.sum()
psumsum += psum
echo("pcount: ", $pcount, " pips: ", $pips, " farray[^1]: ", farray[^1],
" psum: ", $psum, " psumsum: ", $psumsum)
echo("clock: " & $(getTime()-ctime) & " cpuTime: " &
$(cpuTime()-cpustart))
#pcount: 337587 pips: 383662.5627699992 farray[^1]: 1.295654754912296
psum: 218237.9662717213 psumsum: 24517634293.9183
#clock: 1 minute, 43 seconds, 151 milliseconds, 497 microseconds, and 564
nanoseconds cpuTime: 102.814928774
Run
#python 3.7 latest numpy
#http://ratedata.gaincapital.com/2018/12%20December/EUR_USD_Week1.zip
stime = time()
fpcsv = "/home/jimmie/data/EUR_USD_Week1.txt"
count = -1
pips = 0
psum = 0
psumsum = 0
f = open(fpcsv,"r")
csvlines = f.readlines()[1:]
f.close()
csvreader = csv.reader(csvlines)
pasize = len(csvlines)
parray = np.ndarray(shape=(pasize,),dtype="f8")
for tid,d,pair,dt,b,a in csvreader:
count += 1
price = float(a)
parray[count] = ((price * price) * ((count+1) / pasize))
pips += price
psum = parray.sum()
psumsum += psum
print(count, pips, psum, psumsum, time()-stime0)
#337587 383662.5627699992 218239.26158885768 73674955841.14886
50.33330512046814
Run
