[Networkit] parallel Graph iterators for Python

Michael Hamann michael.hamann at kit.edu
Thu Nov 19 12:51:00 CET 2015


Hi,

On Wednesday 18 November 2015 19:25:29, Christian Staudt wrote:
> is there a reason why only the sequential Graph iterators are pythonized?
> Can the parallel iterators be accessed via Python and/or Cython?

There are several reasons. First of all the parallel iterators use new 
threads. Each of these threads needs to be initialized for Python (see also 
[0]). The second reason is that all code that is pure Python code needs to 
hold the global interpreter lock (GIL). We would need to acquire that lock for 
every single node or edge which is a huge overhead and means that that the 
only way that the code for several nodes or edges can run in parallel is if it 
releases the GIL again. For example if the Python callback calls other 
NetworKit methods this would be the case for run()-methods, but not for simple 
getter or setter methods. We could add that there of course, but I think that 
in many cases releasing and acquiring the lock is more work than the actual 
method call.

Therefore I think that it doesn't make sense to find out how to make the 
parallel iterators accessible in Python as there is probably no benefit of 
having them available.

Best regards,
Michael

[0] https://gist.github.com/liuyu81/3473376 for a C-API example, 
http://stackoverflow.com/questions/29243881/communication-between-python-cython-c-in-multithreaded-environment describes the setting we would need - 
but does not have a real answer.



More information about the NetworKit mailing list