Johannes Wilke created SYSTEMML-678:
---------------------------------------
Summary: MLContext parallelization
Key: SYSTEMML-678
URL: https://issues.apache.org/jira/browse/SYSTEMML-678
Project: SystemML
Issue Type: Question
Components: Algorithms, Parser, Runtime
Affects Versions: SystemML 0.10
Reporter: Johannes Wilke
I try to execute script in the MLContext. It is executing, but it dont
parallel. For smaller scripts, it works fine. But this script doesnt and it is
not clear why. I think it is because of the 4 loop levels, but I am not sure.
Is there a documentation what is parallizable and what isnt?
If I change the main while-loop, i wish to parallize, to a parfor loop it works.
Here is the script:
X = read($Xin)
P = read($Pin)
#errorMatrix = matrix(0.0,rows=1,cols=1)
j = 1
sum = 0
while (j <=nrow(X) & sum >= 0){ # this should be parallelized
#parfor(j in 1: nrow(X),check=0){
first = TRUE
windows = matrix(0,rows=1,cols=1)
offsetPreWindowDefinitions = 0
sumWindowLength = 0
mastercount = 0
totalwindowLength = 0
s = 0
for(i in 1: nrow(P)){
if((as.scalar(P[i,1])*as.scalar(P[i,2]))>totalwindowLength){
totalwindowLength =
(as.scalar(P[i,1])*as.scalar(P[i,2]))
}
s = s+1
}
lastWindow = matrix(0,rows=sum(P[,1]),cols=1)
for(i in 1:nrow(P)){# for every Window-Definition
for(k in 1: as.integer(as.scalar(P[i,1]))){# for every pnum
column =
matrix(0,rows=as.integer(as.scalar(P[1,4])),cols=1)
for(l in 1: nrow(column)+1){
offsetPreWindowDefinitions = totalwindowLength
- (as.scalar(P[i,1])*as.scalar(P[i,2]))
tsindex = ((k-1) * as.scalar(P[i,2])) + l-1 +
offsetPreWindowDefinitions
if(l==nrow(column)+1){
lastWindow[sumWindowLength+k,1] =
X[j,tsindex+1]
} else {
column[l,1] = X[j,tsindex+1]
}
mastercount = mastercount +1
#print(mastercount)
}
if(first){
first = FALSE;
windows = column
} else {
windows = cbind(windows,column)
}
}
sumWindowLength = sumWindowLength + as.scalar(P[i,1])
}
result = matrix(14.3,rows=as.integer(as.scalar(P[1,4])),cols=1)
for(i in
totalwindowLength:as.integer(as.scalar(P[1,4]))+totalwindowLength-1){
result[i-totalwindowLength+1,1] = X[j,i+1]
s = s+1
}
params = solve(windows,result)
print(j)
predict = matrix(0,rows=1, cols=1)
for(i in 1:nrow(lastWindow)){
predict[1,1] = predict[1,1] + (params[i,1] * lastWindow[i,1])
s = s+1
}
predictscalar = as.scalar(predict[1,1])
targetscalar = as.scalar(X[j,ncol(X)])
sum = sum + ((targetscalar - predictscalar) * (targetscalar -
predictscalar))
j = j+1
#write(lastWindow,
"/media/johannes/Data/Seafile/UNI/Beleg/sysml_output/lWOut.csv", format="csv",
header=TRUE, sep=",", sparse=TRUE);
#write(windows,
"/media/johannes/Data/Seafile/UNI/Beleg/sysml_output/windowsOut.csv",
format="csv", header=TRUE, sep=",", sparse=TRUE);
#write(result,
"/media/johannes/Data/Seafile/UNI/Beleg/sysml_output/resultOut.csv",
format="csv", header=TRUE, sep=",", sparse=TRUE);
}
print(sum/nrow(X))
I hope that you can help me!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
