[Networkit] Distributed graph analysis in NetworKit

Christian Staudt christian.staudt at kit.edu
Tue Jul 25 11:40:35 CEST 2017


> But as I already wrote on in my response in Henning, I will
> try running NetworKit on a large-memory cluster node and see if that
> might work. Thanks,

Cool, hope that works out for you.


> GraphLab seems to have been bought
> out and unless I am mistaken, the only available open-source
> components are the basic graph datastructures, but not the analysis
> routines.

Off-topic: That’s interesting, because I always thought something like NetworKit would never work as a commercial product. Makes you wonder what the important differences are. One is certainly a difference in perspective: e.g. NetworKit says it "provides link prediction algorithms” while GraphLab shows off that you can "train a recommender model" in one line of code. Not saying that NetworKit should become a startup (:O), but that the connection network analysis / machine learning is underdeveloped and worth exploring.

Best
Chris

> 
> On Sat, Jul 22, 2017 at 2:45 AM, Christian Staudt
> <christian.staudt at kit.edu> wrote:
>> Hi Ivan,
>> get more RAM. If that doesn’t help: Have you considered the many existing solutions for distributed graph processing? We had a paper on this a while ago [1], and surely there have been developments since then.
>> 
>> @all: Should there ever be plans to write a new distributed graph computing backend, my hope is that there will be a careful discussion about whether adding that to NetworKit is a case of scope creep. [2][3]
>> 
>> Best regards
>> Chris
>> 
>> 
>> 
>> [1]: https://www.researchgate.net/publication/289117288_An_Empirical_Comparison_of_Big_Graph_Frameworks_in_the_Context_of_Network_Analysis
>> [2]: https://www.youtube.com/watch?v=kmJ59yyYza4
>> [3]: https://www.pmi.org/learning/library/top-five-causes-scope-creep-6675
>> 
>>> On 22. Jul 2017, at 07:32, Henning Meyerhenke <henning.meyerhenke at kit.edu> wrote:
>>> 
>>> Dear Ivan,
>>> 
>>> Thank you for raising this issue. This aspect has been, of course, in my
>>> mind for some time. So far, there is no funding for this (nor did we
>>> apply). Thus, do not expect a distributed backend any time soon.
>>> 
>>> If there are, however, numerous serious requests in this regard, we
>>> would have very good arguments towards funding agencies...
>>> 
>>> On the other hand, for us there is rarely the need to go beyond shared
>>> memory. Machines with 1TB RAM are fairly cheap these days and you can
>>> fit a lot into 1TB (approx. 40G edges). This is sufficient for many
>>> applications -- but certainly not for all.
>>> 
>>> If you want to discuss your application in more detail, this could be
>>> helpful for assessment. If you don't want to do that on the public
>>> mailing list, feel free to send to my personal email address.
>>> 
>>> Best,
>>> Henning
>>> 
>>> 
>>> 
>>> Am 21.07.17 um 20:51 schrieb Ivan Raikov:
>>>> Hello,
>>>> 
>>>>  I have recently come across NetworKit and rather like its interface
>>>> and features. However, some of the graphs I would like to analyze are
>>>> too big to fit in the memory of a single workstation, and so I wonder
>>>> if there are any plans or interest in providing functionality for
>>>> graph analysis in distributed-memory compute environments, such as
>>>> MPI-based clusters. Thanks and best regards,
>>>> 
>>>> -Ivan Raikov
>>>> 
>>>> _______________________________________________
>>>> NetworKit mailing list
>>>> NetworKit at ira.uni-karlsruhe.de
>>>> https://lists.ira.uni-karlsruhe.de/mailman/listinfo/networkit
>>>> 
>>> 
>>> --
>>> 
>>> ==========================================================
>>> Karlsruhe Institute of Technology (KIT)
>>> Institute of Theoretical Informatics (ITI)
>>> 
>>> Prof. Dr. Henning Meyerhenke
>>> Theoret. Informatics / Parallel Computing
>>> 
>>> Phone: +49-721-608-41876
>>> Web: http://parco.iti.kit.edu/henningm/
>>> 
>>> KIT - The Research University in the Helmholtz Association
>>> ==========================================================
>>> 
>>> _______________________________________________
>>> NetworKit mailing list
>>> NetworKit at ira.uni-karlsruhe.de
>>> https://lists.ira.uni-karlsruhe.de/mailman/listinfo/networkit
>> 
>> 
>> _______________________________________________
>> NetworKit mailing list
>> NetworKit at ira.uni-karlsruhe.de
>> https://lists.ira.uni-karlsruhe.de/mailman/listinfo/networkit
> 
> _______________________________________________
> NetworKit mailing list
> NetworKit at ira.uni-karlsruhe.de
> https://lists.ira.uni-karlsruhe.de/mailman/listinfo/networkit




More information about the NetworKit mailing list