[Networkit] graph.indexEdges() dies on large graph

Maximilian Vogel maximilian.vogel at student.kit.edu
Fri May 20 11:26:05 CEST 2016


Hi Stephen,

I tried to estimate the size of the graph object. It takes about 2GB 
(including edge ids), probably more.

On 19.05.2016 22:21, Stephen North wrote:
> I get various errors.
Which errors besides a stopping kernel?

>   That’s about 60 bytes of memory per edge.
An edge is stored in the adjacency array of both nodes to enable fast 
iterations over the neighbors of a node. Hence, an edge (including its 
id) takes up 32 byte (not counting in any overhead of C++'s vector).
Also, do you have the full 4GB available? What about the OS's and other 
processes usage of memory?

> Any thoughts about that?  Would one expect to see an error when we hit the
> wall with memory, or only random errors? (ipython3 just says the kernel stopped
> and it’s going to restart it automatically.)
You're raising an interesting point. When I'm out of memory, the kernel 
stops or the ipython shell just crashes. I guess it comes down to what 
happens internally. If the C++ runtime just "crashes", I don't know 
(yet) what can be done. If a bad allocation exception is thrown, this 
could and should be passed to Python already. (At least I vaguely recall 
receiving such an error in an IPython terminal.)

>   I can try a bigger VM, so how big?
At least 8GB. It also depends on how many and which experiments you want 
to do as you need some memory for them as well.

Please keep us updated if the errors persist with more memory.

Best,
Max



More information about the NetworKit mailing list