Hi, I have the same question. Upon running code attempting to use GraphViews,
I get an error during pickling:
error: 'i' format requires -2147483648 <= number <= 2147483647
More specifically, it looks like a line inside joblib is unhappy:
CustomizablePickler(buffer, self._reducers).dump(obj)
And this takes us to a struct packing line: header = struct.pack("!i", n)
So, if I had to guess, I'd suspect joblib is trying to pickle the whole
graph rather than the GraphView reference, or something like this. Was
either of you able to get code to successfully parallelize using GraphViews
to avoid copying?
This is on python 3.6.5 with graph_tool version '2.26 (commit , )' , joblib
version '0.11'
Below is a minimal breaking example, if it helps. I am also happy to provide
other information such as tracebacks.
def toy_func(g):
return g.vertex_properties['skim'][0][1]
vmr = [0, 1]
g = load_graph(path) # 22,000 vertex directed graph (a road network)
skim_table = shortest_distance(g, weights=g.edge_properties["weight"])
g.properties['skim'] = skim_table
p(joblib.delayed(toy_func)(GraphView(g)) for i in range(10))
--
Sent from:
http://main-discussion-list-for-the-graph-tool-project.982480.n3.nabble.com/
_______________________________________________
graph-tool mailing list
[email protected]
https://lists.skewed.de/mailman/listinfo/graph-tool
