Am 16.07.2018 um 03:34 schrieb cmos:
> Hi, I have the same question. Upon running code attempting to use GraphViews,
> I get an error during pickling:
>
> error: 'i' format requires -2147483648 <= number <= 2147483647
>
> More specifically, it looks like a line inside joblib is unhappy:
> CustomizablePickler(buffer, self._reducers).dump(obj)
>
> And this takes us to a struct packing line: header = struct.pack("!i", n)
>
>
> So, if I had to guess, I'd suspect joblib is trying to pickle the whole
> graph rather than the GraphView reference, or something like this. Was
> either of you able to get code to successfully parallelize using GraphViews
> to avoid copying?
It is impossible to say anything, without a minimal and self-contained
example that shows the problem.
> Below is a minimal breaking example, if it helps. I am also happy to provide
> other information such as tracebacks.
>
> def toy_func(g):
> return g.vertex_properties['skim'][0][1]
>
> vmr = [0, 1]
> g = load_graph(path) # 22,000 vertex directed graph (a road network)
> skim_table = shortest_distance(g, weights=g.edge_properties["weight"])
> g.properties['skim'] = skim_table
> p(joblib.delayed(toy_func)(GraphView(g)) for i in range(10))
That is not a complete minimal example; the function 'p' is undefined and
there are other errors. Please provide one that actually runs, and does not
depend on external data.
Best,
Tiago
--
Tiago de Paula Peixoto <[email protected]>
_______________________________________________
graph-tool mailing list
[email protected]
https://lists.skewed.de/mailman/listinfo/graph-tool
