Hello all,
I'm trying to pickle a very large graph (23 million vertices, 152 million
edges, two vertex attributes), but keep getting an `OSError: [Errno 22]
Invalid argument` error. However, I think that's erroneous, as if I
subsample the graph and save with exact same code I have no problems.
Here's the traceback:
g.summary()
Out[8]: 'IGRAPH UN-- 23331862 152099394 -- \n+ attr: name (v), pt (v)'
g.write_pickle(fname='graphs/with_inferred/vz_inferred3_sms{}_voz{}.pkl'.format(sms,
voz))
Traceback (most recent call last):
File "<ipython-input-9-6b5409a79251>", line 1, in <module>
g.write_pickle(fname='graphs/with_inferred/vz_inferred3_sms{}_voz{}.pkl'.format(sms,
voz))
File
"/Users/Nick/anaconda/lib/python3.5/site-packages/igraph/__init__.py", line
1778, in write_pickle
result=pickle.dump(self, fname, version)
OSError: [Errno 22] Invalid argument
g=g.vs[range(3331862)].subgraph()
g.write_pickle(fname='graphs/with_inferred/vz_inferred3_sms{}_voz{}.pkl'.format(sms,
voz))
[success]
The graph takes up about 10gb in memory, and the pickle command expands
Python's memory footprint to about 35gb before the exception gets thrown,
but I'm on a machine with 80gb ram, so that's not the constraint.
Any suggestions as to what might be going on / is there a work around for
saving?
Thanks!
Nick
_______________________________________________
igraph-help mailing list
[email protected]
https://lists.nongnu.org/mailman/listinfo/igraph-help
