[graph-tool] IMPORTANT: migration of this list to discourse
Dear graph-tool users, I have decided to migrate this list to the discourse forum at: https://forum.skewed.de/c/graph-tool Discourse allows for web forums to work similarly to mailing lists, and many people nowadays consider them more friendly. All subscribed users to this list are now “staged users” of the discourse forum. This means that their identity can be claimed simply by creating an account with the same mail. Staged users do not have to create an account. They can post directly to the category by email via graph-tool@skewed.de. Replies will also be sent via email, which users can also reply via email — thus mimicking how a mailing list works. This is the same for all users, regardless if they are staged or not. Users can disable emails at any time if they so wish. IMPORTANT: Staged users will not receive email notifications for new topics posted by other users. If you wish this, then you need to open an account. Likewise, normal users need to “watch” the category to receive such notifications. In summary, if you do not create an account in the website you will cease to receive further notifications, until someone replies to one of your posts. (Emailing to the old mailing list has been disabled; all messages posted to graph-tool@skewed.de will henceforth come to discourse automatically. The mailing list archives are still available, but have been mirrored there in full.) Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: New discourse forum for graph-tool
Dear all, There was a database problem with the forum, and it needed to be installed from scratch. If you created an account there, you will need to create it again — sorry for the inconvenience. The good news is that the whole mailing list archive has been imported there, so the whole history is available. I'm in the process of configuring post via email, in which case the forum will behave just like a mailing list. If this works well, I can deprecate this list. However, in the mean-time it will remain active. Best, Tiago Am 06.11.22 um 21:49 schrieb Tiago de Paula Peixoto: Dear graph-tool users, There is now an official discourse forum for graph-tool available at: https://forum.skewed.de/c/graph-tool This mailing list is going to remain active — nothing will change, but many people nowadays prefer such web discussions over mailing lists. The link to both the mailing list and forum is now available in the project's webpage. Feel free to use what you prefer. Best, Tiago ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Choosing number of blocks in each side of the bi-partite SBM
Dear Bernardo, Am 05.10.22 um 18:49 schrieb Bernardo Modenesi: Hi, I am aware of a post asking for the same topic, but it was dated 2 years ago, but I wonder if there was any development on this front? In the function "model.minimize_nested_blockmodel_dl" it is possible to specify B_min=G and B_max=G in order to get exactly G blocks for the whole clustering. I wonder if it is possible to specify separately the number of blocks on each side of the bi-partite network, say G1 for one side of the network and G2 for the other side? Even maybe by running 2 different estimations? Unfortunately, this functionality has not been implemented yet. I also noticed that Tzu-Chi Yen has an alternative code for this, but I wonder how comparable it is to graph-tool in terms of efficiency? Code: https://github.com/junipertcy/bipartiteSBM <https://github.com/junipertcy/bipartiteSBM>Related doc:https://docs.netscied.tw/bipartiteSBM/usage/explore-consistency.html <https://docs.netscied.tw/bipartiteSBM/usage/explore-consistency.html> As you can see for yourself, that implementation is in pure Python, whereas graph-tool is implemented in C++. Therefore it should be much slower than graph-tool. Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] New discourse forum for graph-tool
Dear graph-tool users, There is now an official discourse forum for graph-tool available at: https://forum.skewed.de/c/graph-tool This mailing list is going to remain active — nothing will change, but many people nowadays prefer such web discussions over mailing lists. The link to both the mailing list and forum is now available in the project's webpage. Feel free to use what you prefer. Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Installing igraph tutorial from website: AttributeError: 'AxesSubplot' object has no attribute 'arc'
Hi,
I can't believe I have to explain this, but this is the mailing list for
graph-tool, not for igraph.
Please refer to the igraph mailing list for help.
Best,
Tiago
Am 21.09.22 um 12:21 schrieb Jos J:
I have python 3.8.13, igraph 0.10.1, with matplotlib 3.6.0, and when
running the sample scripts from the install page on the website, I am
receiving an error. The scripts can be found here:
https://igraph.org/python/tutorial/latest/install.html
<https://igraph.org/python/tutorial/latest/install.html>
I am referring to the following
>>> import matplotlib.pyplot as plt
>>> import igraph as ig
>>> fig, ax = plt.subplots()
>>> g = ig.Graph.Famous("petersen")
>>> ig.plot(g, target=ax)
>>> plt.show()
I receive the following errors at the ig.plot(g, target=ax) line:
---
AttributeError Traceback (most recent call last)
Cell In [120], line 6
4 fig, ax = plt.subplots()
5 g = ig.Graph.Famous("petersen")
> 6 ig.plot(g, target=ax)
8 plt.show()
File
~/anaconda3/envs/rapids-2208/lib/python3.8/site-packages/igraph/drawing/__init__.py:245, in plot(obj, target, bbox, *args, **kwds)
243 return
244 else:
--> 245 plotter(
246 backend,
247 target,
248 palette=palette,
249 *args,
250 **kwds,
251 )
252 return target
254 # Cairo backend
File
~/anaconda3/envs/rapids-2208/lib/python3.8/site-packages/igraph/drawing/graph.py:558, in __plot__(self, backend, context, *args, **kwds)
552 from igraph.drawing import DrawerDirectory
554 drawer = kwds.pop(
555 "drawer_factory",
556 DrawerDirectory.resolve(self, backend)(context),
557 )
--> 558 drawer.draw(self, *args, **kwds)
File
~/anaconda3/envs/rapids-2208/lib/python3.8/site-packages/igraph/drawing/matplotlib/graph.py:233, in MatplotlibGraphDrawer.draw(self, graph, *args, **kwds)
231 drawer_method = vertex_drawer.draw
232 for vertex, visual_vertex, coords in vertex_coord_iter:
--> 233 drawer_method(visual_vertex, vertex, coords)
235 # Construct the iterator that we will use to draw the vertex labels
236 vs = graph.vs
File
~/anaconda3/envs/rapids-2208/lib/python3.8/site-packages/igraph/drawing/matplotlib/vertex.py:60, in MatplotlibVertexDrawer.draw(self, visual_vertex, vertex, coords)
49 width = (
50 visual_vertex.width
51 if visual_vertex.width is not None
52 else visual_vertex.size
53 )
54 height = (
55 visual_vertex.height
56 if visual_vertex.height is not None
57 else visual_vertex.size
58 )
---> 60 stroke = visual_vertex.shape.draw_path(
61 ax,
62 coords[0],
63 coords[1],
64 width,
65 height,
66 facecolor=visual_vertex.color,
67 edgecolor=visual_vertex.frame_color,
68 linewidth=visual_vertex.frame_width,
69 zorder=visual_vertex.zorder,
70 )
71 ax.add_patch(stroke)
File
~/anaconda3/envs/rapids-2208/lib/python3.8/site-packages/igraph/drawing/shapes.py:168, in CircleDrawer.draw_path(ctx, center_x, center_y, width, height, **kwargs)
166 return mpl.patches.Circle((center_x, center_y), width / 2,
**kwargs)
167 else:
--> 168 ctx.arc(center_x, center_y, width / 2, 0, 2 * pi)
AttributeError: 'AxesSubplot' object has no attribute 'arc'
-
Not sure if its my python or package versions, or if its a configuration
issue, or possibly an issue with igraph or matplotlib. Any advice would
be most welcome
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
--
Tiago de Paula Peixoto
OpenPGP_signature
Description: OpenPGP digital signature
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Extra levels created by minimize_nested_blockmodel_dl
Am 16.08.22 um 15:42 schrieb Kevin Reuning: Hello, I am estimating a nested_block_model with bipartite data and am getting a strange result. There are about 4,000 nodes in the network and I am estimating a model by: ``` state_dc = minimize_nested_blockmodel_dl(G, state_args=dict(deg_corr=True, clabel=bipart.a, pclabel=bipart.a, recs=[G.ep.weight], rec_types=["discrete-poisson"])) for i in range(1000): # this should be sufficiently large state_dc.multiflip_mcmc_sweep(beta=np.inf, niter=10) ``` The results find 13 nested levels, but after level 5 there are only 2 nonempty blocks found in each additional level. I'm not sure if this is really a problem, or I can just ignore the levels where nothing really changes. Any advice/help would be greatly appreciated. Thank you If you pass a bipartite pclabel, then this bi-partition gets reflected in the upper levels of the hierarchy as well. This is normal. -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Calling node predicates while calculating subgraph_isomorphism?
Am 03.08.22 um 12:50 schrieb Sergey Mironov: Hi. I'm interested in finding subgraph isomorphisms on graphs representing tensor networks. Nodes of such graphs contain expressions (but here one could think of arithmetic expressions over +-/* and variables). Let's say I want to find an isomorphism between my graph and a pattern graph, which contains more generic expressions. Comparing expressions directly doesn't make much sense because of variables, so some form of unification[1] should be used instead. What strategy could I use to fit the existing API of subgraph_isomorphism which currently only allows me to calculate some fixed labels for nodes? The simplest strategy would be for you to define a hash function that maps your arithmetic expressions to integers. You need only to make sure that no collisions exist, but that should be very easy to enforce. The simplest way to do this is to list all the different expressions in all nodes in arbitrary order, and use the order index as a hash value. Is it hard(feasible) to modify the API by adding some form of predicate comparison of nodes? It is feasible, but such an API would be extremely slow, since this predicate would need to be computed in the inner loop of the algorithm. The hash approach outlined above is far faster, and easy to implement in general. Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: bipartite minimize_nested_blockmodel_dl() error: cannot move vertex across clabel barriers
Am 20.07.22 um 08:02 schrieb Siwei Zhang:
Hi Dr. Peixoto,
I am using graph-tool version 2.45 and I have two questions.
1. I am trying to reproduce the script in the document
2. g = gt.collection.data["celegansneural"]
state = gt.minimize_nested_blockmodel_dl(g,
state_args=dict(overlap=True))
and have the error:
/usr/lib/python3/dist-packages/graph_tool/inference/blockmodel.py:390:
UserWarning: unrecognized keyword arguments: ['overlap']
warnings.warn("unrecognized keyword arguments: " +
It seems the argument of "overlap" is removed.
The proper way to use an overlapping model is to pass the option:
state_args=dict(base_type=OverlapBlockState)
2. Regardless of the question1, I am trying to do a bipartite
version stochastic block model and I define "clabel" to constraint
labels on the vertices so that vertices with different label values will
not be clustered in the same group. But I always have the error of
"ValueError: cannot move vertex across clabel barriers". The below is
the code:
node_types = g.vp['kind']
node_types.get_array()
Output: PropertyArray([1, 1, 1, ..., 2, 2, 2], dtype=int32)
state = gt.minimize_nested_blockmodel_dl(
g,
state_args=dict(clabel=node_types,pclabel=node_types,deg_corr=True),
multilevel_mcmc_args = dict(niter=niter,beta=beta))
Could you please help me with these questions? Thanks!
In order to understand what is happening you would need to send us a
minimal but complete working example that shows the problem.
Best,
Tiago
--
Tiago de Paula Peixoto
OpenPGP_signature
Description: OpenPGP digital signature
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: minimize_nested_blockmodel_dl got an unexpected keyword argument 'multilevel_mcmc_args'
Am 13.07.22 um 16:43 schrieb Siwei Zhang: And may I ask another question? In the paper, it mentioned "by making beta goes to infinity and repeated many times, which yields a reliable estimate of the maximum". May I double-check what "repeated many times" refers to? Does it refer to the number of sweeps or refer to the whole algorithm? The whole algorithm. I also noticed there is a warning of "multilevel_mcmc_sweep" in NestedBlockState: "This function performs niter sweeps at each hierarchical level once. This means that in order for the chain to equilibrate, we need to call this function several times, i.e. it is not enough to call it once with a large value of niter." I found that the high-level function "minimize_nested_blockmodel_dl" seems already done that. But I am a little bit confused, if possible, could it be more specific? Thank you so much for your help! This warning is not applicable if minimize_nested_blockmodel_dl() is being used, only if you are attempting to sample from the posterior distribution (and not finding its maximum). Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: minimize_nested_blockmodel_dl got an unexpected keyword argument 'multilevel_mcmc_args'
Am 13.07.22 um 01:28 schrieb Siwei Zhang:
Hello,
I have a question about minimize_nested_blockmodel_dl(). The below is my
code:
state_args = {'clabel': node_types, 'pclabel': node_types}
multilevel_mcmc_args = {'niter': niter, 'beta': beta}
state = gt.minimize_nested_blockmodel_dl(
g,
deg_corr = True,
overlap = False,
state_args = state_args,
multilevel_mcmc_args = multilevel_mcmc_args
)
And I encountered an error that "minimize_nested_blockmodel_dl got an
unexpected keyword argument 'multilevel_mcmc_args'". I removed
"multilevel_mcmc_args" and it works. And I also tried example
"|minimize_blockmodel_dl(G, multilevel_mcmc_args=dict(B_min=4,
B_max=4))|" in https://git.skewed.de/count0/graph-tool/-/issues/725
<https://git.skewed.de/count0/graph-tool/-/issues/725> and I encountered
the same error. I took a look of the source code of graph-tool and I did
not find where is the problem. Could you please help me about it? Thanks!
I cannot reproduce this. What version of graph-tool are you using?
--
Tiago de Paula Peixoto
OpenPGP_signature
Description: OpenPGP digital signature
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Differential blocks between graphs
Am 30.06.22 um 22:21 schrieb Davide Cittaro: I was thinking this https://arxiv.org/pdf/2006.05176.pdf <https://arxiv.org/pdf/2006.05176.pdf> but with a principled formulation based on SBM. Is that even possible? Just curious… There is nothing yet in graph-tool about graph classification, unfortunately! Hopefully in the future... Best, Tiago -- Tiago de Paula Peixoto OpenPGP_signature Description: OpenPGP digital signature ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Differential blocks between graphs
Am 30.06.22 um 13:35 schrieb Davide Cittaro: Hello all, I was wondering if (and possibly how) graph-tool could be used to identify differential blocks across given graphs. More specifically, if two graph have the same nodes, is it possible to identify the sets of partitions that are blocks in one graph but not the other? Would it suffice to identify partitions in both graphs and then check for the differences? I'm not sure I completely understand. What do you mean by one block being in one graph, and not in the other? Could you give an example? -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: openmp_enabled() returns False.
Dear Naoki, This is best addressed by opening an issue with the homebrew project, which packages graph-tool. The packaging should be changed to enable openmp by default. Best, Tiago Am 14.05.22 um 15:08 schrieb Naoki Akazawa: Hi, I use graph-tool on MacBookPro 13-inch, 2018(intel). I installed graph-tool by running `brew install graph-tool` I expected using openMP, but it seems disable. スクリーンショット 2022-05-14 14.02.25.png To use openMP on MacOS, should I do something? I already installed gcc and libomp. The following is my environment: * MacBookPro 13-inch, 2018 * 2.7 GHz quad core Intel Core i7 * MacOS Monterey (12.3.1) * Libraries o jypyterlab: 2.1.0 o python: 3.9.12 o graph-tool: 2.44 ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: A question about generating simple graphs in graph-tool
Am 30.04.22 um 06:49 schrieb Snehal Shekatkar: However, if any of the degree value is greater than "n-1", the size of the graph, clearly a simple graph cannot be generated from such sequence, and it should be immediately changed by re-generating degrees for such vertices. This happens a lot when the degrees are sampled from a right-skewed distribution. But I don't see anything like this in the code. My question is, isn't it inefficient to keep randomly choosing vertices one at a time and change their degrees until the vertices with degree greater than n-1 are reassigned degrees? Is it being done for correctly sampling from a given degree distribution? Or am I missing something? This can be implemented in a trivial way simply by rejecting these large numbers already in the `deg_sampler` function that you supply to random_graph(). -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Inference for weighted, hierarchical stochastic block models
Am 11.04.22 um 16:09 schrieb Alexander Barron: Am I on the right track? Yes, the command seems right to me, assuming your edge weights are non-negative integers. And would it be better to specify the edge weights via the `*BlockState*` `*recs*` parameter instead of using `*eweight*`? No, the “eweight” parameter refers to edge multiplicities, not the covariates. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Clabels cause segmentation fault
Am 23.03.22 um 10:34 schrieb RESTOCCHI Valerio:
Hi all,
I am having some problems working with clabels.
Whenever I use mcmc algorithms with clabels the kernel dies (upon
inspection, I found the error is 'segmentation fault (core dumped)').
I am not sure what I am doing wrong, so here’s a minimum working example:
g = gt.collection.data["lesmis"]
clabel = g.vp["clabel"] = g.new_vp("int")
for v in g.vertices():
clabel[v]=np.random.randint(3)
state = gt.NestedBlockState(g,
state_args={'clabel':clabel,'pclabel':clabel})
for i in range(100):
state.multiflip_mcmc_sweep(niter=10)
If it helps, I’m getting the same problem with the following changes:
- use equilibrate instead of multiflip.
- use two labels instead of three.
- use minimize_nested_blockmodel_dl instead of NestedBlockState
- pass clabels dictionary as **args
Extra information:
- If I run the multifip MCMC fewer times (let’s say 10) sometimes it
works, sometimes it doesn’t, so it seems that the error is somewhat
stochastic.
- This happens on different machines, both with Linux and Mac
Is this a bug or am I doing something wrong?
This is indeed a bug, unfortunately.
Can you please open an issue in the website with the minimal example
above so I can keep track of this and fix it?
Thanks!
Best,
Tiago
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Fetch all the edges between given two vertices
Am 09.03.22 um 12:50 schrieb Snehal Shekatkar: Both these operations, especially the first one, is expensive. Also, `g.edge(u,v)` seems to return only one edge without any information about which one it is. How can I solve this problem? Just call: g.edge(u, v, all_edges=True) -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: overlap/mixed-membership sbm help
Am 04.03.22 um 06:37 schrieb Eli Draizen: I think I was able to figure it out using the code from the "Characterizing the posterior distribution" section. I start by equilibrating a degree-corrected NestedBlockState with edge weights and then inferring partition modes with ModeClusterState. Is this the same as running minimize_nested_blockmodel_dl? No, this is not the same. ModeClusterStates characterizes the whole distribution of answers, while minimize_blockmodel_dl() finds the single most likely/compressive partition. Are the minimize* functions not used anymore? Yes they are. They have different goals. Also, is it normal to have inferred only one mode? Yes, it can happen. This means the solutions are clustered around a single typical solution, rather than several different ones. This is discussed in this paper: https://dx.doi.org/10.1103/PhysRevX.11.021003 Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Accessing all ancestors
Am 22.02.22 um 15:02 schrieb RAFAEL SERAPILHA DURELLI: Ok Tiago thanks. My question is, given a source node how can I get all ancestors? I tried to use all_predecessors but I dunno what is dist_map and prep_map. The documentation says: dist_map : :class:`~graph_tool.VertexPropertyMap` Vertex property map with the distances from ``source`` to all other vertices. pred_map : :class:`~graph_tool.VertexPropertyMap` Vertex property map with the predecessors in the search tree. You can determine these values using the function shortest_distance(): dist_map, pred_map = shortest_distance(g, source=u, pred_map=True) -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Accessing all ancestors
Am 22.02.22 um 14:53 schrieb RAFAEL SERAPILHA DURELLI: I found the func all_predecessors, but I could not figured out how to use it.. since it is mandatory to pass diet_map and prep_map. Can you please help me inhere :D We cannot help with generic questions about how to use a function. There is a documentation that explains how to use it, and I cannot read your mind to determine what you did or did not understand from it. We can only help with specific questions. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Fwd: Adding label to vertices
Dear Rafael, >> In graphml, ids are not graph data. If your source uses node ids for >> data you should fix it and use / instead (see 'Additional >> Data' in <http://graphml.graphdrawing.org/specification.html> >> <http://graphml.graphdrawing.org/specification.html>). Node ids are >> for internal parsing use and are not guaranteed to be preserved. This is not quite right. If the node/edge ids are not in the canonical format, they are imported as property maps when the graph is loaded. The property maps can be accessed as: g.vp["_graphml_vertex_id"] and g.ep["_graphml_edge_id"] Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Generate graph from nested sbm
Am 18.02.22 um 17:40 schrieb Davide Cittaro: Hi all, given a SBM fit it should be possible to generate a random graph using the fitted model. As far as I understand this is done using gt.generate.generate_sbm() function. What about nested SBM instead? I guess generating a graph from blocks at level 0 doesn't work, is there a workaround for this? The difference between the nested and non-nested SBMs is only their priors, not the actual model. So indeed, when sampling from the nested model, all you need to do is to sample from the model at the lowest level. There is a convenience function for this, which is: state.levels[0].sample_graph() For non-nested model this is only: state.sample_graph() These are just convenience wrappers for generate_sbm(). Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Ubuntu Impish support
Dear Tanguy, Indeed there was a problem with the impish repository, but it is fixed now. Thank you for letting me know! Best, Tiago Am 11.02.22 um 10:16 schrieb Tanguy Fardet: Dear Tiago, the installation wiki [1] lists impish as a supported distro but the corresponding apt folder on the server [2] is empty. Is it indeed supposed to contain a deb or is it an issue with the documentation? [1]: https://git.skewed.de/count0/graph-tool/-/wikis/installation-instructions#debian-ubuntu [2]: https://downloads.skewed.de/apt/dists/impish/ Thanks in advance -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Main path algorithm
Dear Rafael, This is the mailing-list for graph-tool, not snap. Best, Tiago Am 11.02.22 um 21:00 schrieb RAFAEL SERAPILHA DURELLI: I would like to know if snap has implemented the following algorithms: (1) Search path link count (SPLC) -- which is a Traversal counts algorithm (2) Path Search Thanks in advance. -- At.te Prof. Dr. Rafael S. Durelli. O conteúdo deste e-mail e anexos são restritos aos seus destinatários e de responsabilidade do remetente. O uso do e-mail deve estar de acordo com os regulamentos institucionais vigentes. ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Passing different rec_types for different layers
Dear Arttu. This is currently not supported, but will soon be possible with a new StackedBlockState that will appear in the next release. Please open a feature-request issue in the website so you will be notified when this becomes available. Best, Tiago Am 02.02.22 um 18:46 schrieb arttu.malkam...@helsinki.fi: Hello, Given a situation where I'm trying to fit a LayeredBlockState with somewhat different edge weight distributions across the layers, is it possible to pass different rec_types for different layers when using LayeredBlockState? Or should I pick one that fits the aggregate distribution of my ten layers? Or is it perhaps best to convert the weights in each layer to discrete values? This is how I'm currently doing it, but I know that "real-normal" works better for the first two layers if I fit them separately. state = gt.minimize_blockmodel_dl(g, state = gt.LayeredBlockState, state_args = dict(deg_corr = True, overlap = True, layers = True, ec = g.ep.layer, recs = [g.ep.weight], rec_types = ["real-exponential"])) Best, Arttu ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: graph_tool.topology.shortest_distance() - Confusing documentation or possible bug in return_reached; wrong documentation
Dear João, Am 02.02.22 um 14:31 schrieb João Aveiro: Hello everyone, In the "shortest_distance" routine there is the option to return an array with the reached vertices by setting the flag/attribute "return_reached" to True. While using this routine, having set the max_dist, I have found that in the returned array not only are included the vertices in fact reached within the max_dist limit, but also some neighbouring vertices that probably were visited during the Dijkstra steps but exceeded the limit. I believe this behaviour is confusing and it may lead to erratic code/bugs for those who are not expecting it. I propose either: 1. Change the routine as to return only the vertices that are in fact reached within max_dist; 2. Change this flag to something like "return_visited", as to hint that the returned vertices may not in fact be reached within the expected distance limit; 3. Add a warning in the documentation regarding this behaviour. The usual procedure when reporting bugs is to provide a minimal working example that shows the problem. Otherwise we have to hunt blindly for an instance of the behavior you are reporting. Can you please provide a minimal example? Regarding the same method but on another subject, I believe there is an error in the documentation, namely on the return variables. There we can see two optional return values named "pred_map", while in fact one of them is the reached-vertices array mentioned above. Having the same name one may think that it might actually be a exclusive OR return where only either one is returned and it is just an matter of weird naming. This is indeed a bug in the documentation, and will be fixed. (Could you please open an issue in gitlab so that it is not forgotten?) I may try to implement one of the changes above, if it is decided so, but the access to the GitLab repo through GitHub account linking has not been accepted by the admin yet. I have approved your account. If you could open an issue for each bug with a minimal working example that would be appreciated. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Interactive graph visualization with gravis
Am 18.01.22 um 17:01 schrieb roberth...@protonmail.com: Hi everyone! I've written an open-source package for interactive graph visualization with Python and HTML/CSS/JS called gravis. It supports graph-tool by accepting its Graph object as input for the provided plotting functions. The output is a Figure object that can be displayed in a web browser (separate window or inline in Jupyter notebook) and exported to a standalone HTML file or a static image (JPG, PNG, SVG). In the browser you can drag around nodes, play with layout algorithms and their parameters, see additional information when hovering over or clicking on an element, and more. The package and its documenation can be found here: https://pypi.org/project/gravis https://robert-haas.github.io/gravis-docs The examples in the documentation contain a (preliminary) notebook that demonstrates the combined use of graph-tool and gravis: https://robert-haas.github.io/gravis-docs/code/examples/external_tools/graph-tool.html I hope you find it useful for inspecting your graphs and networks! Feedback of any sorts is always welcome. Please keep in mind that the package is in its beta version, so things may still change slightly in future releases, but you'll always be able to download any version from PyPI. This is very interesting! Thanks for sharing. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Stop current search path in a DFS
Am 24.12.21 um 02:29 schrieb Gerion Entrup: Hi, I'd like to perform a DFS to find some nodes. However, I only need the first occurrences of these nodes. To visualize the problem, I have drawn a little example graph: https://i.imgur.com/BgdUzOz.png The graph is directed but has no further constraints. The search should start at the orange node. It should reveal the two nearby red nodes but not the red node labeled with "This one not". Precisely said: It should not reveal a red node that is dominated by another red node. If I would implement the DFS by myself I just wouldn't follow the outgoing edges of a red node (and stop the search for this path). However, within graph-tool I have only found a way to stop the _whole_ search with an StopSearch exception. Is there a way to use graph-tool to perform a search like I have described it? Sorry for the delay in replying, but I don't see an obvious way to implement what you want with the current code. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: data structure (and graph edition)
Hi, The data structure is an adjacency list. The definition is here: https://git.skewed.de/count0/graph-tool/-/blob/master/src/graph/graph_adjacency.hh Adding nodes or edges is O(1). Removing a node is O(N + E) in the worst case if the node index ordering is preserved, otherwise it is O(k + k_last), where k is the degree of the vertex being removed, and k_last is the degree of the nodes with the highest index. Removing an edge is O(k_s + k_t), where k_s and k_t are the degrees of the source and target nodes, if Graph.set_fast_edge_removal() has not been set, otherwise it is O(1) (at the expense of more memory bookkeeping). This is all explained in the documentation... Best, Tiago Am 09.12.21 um 15:06 schrieb Matthieu Latapy: Well, not far: this is the file format. But it seems very close to the central memory representation, right? Does this mean that the representation is an array indexed by nodes, and then for each node an array of its neighbors? How does this deal with graph updates? (It was so long since I read hexdump for the last time, thanks for this too! *__*) Best, ML On Thu, Dec 09, 2021 at 01:48:51PM +, Lietz, Haiko wrote: Hi Matthieu, Is this what you're looking for? https://graph-tool.skewed.de/static/doc/gt_format.html Best Haiko ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Modelling with community structure data
Dear James, Am 23.11.21 um 16:07 schrieb James Ruffle: Hi Tiago, Apologies for not being clearer. Let me try and make my example more specific: I have a network defined from brain imaging and am passing an edge weight of a given clinical variable. In this example the nodes are voxels of brain tissue, the edges are the presence of the voxels being structurally connected in imaging space, and the edge weight is the relationship of this to a clinical variable, a weight which incorporates ageing. I want to firstly derive the community structure, passing the edge weight, which ultimately gives me clusters of voxels. But, in addition I want to derive some formulation of a weight for the community blocks for their relation to the passed edge weight. For instance, in this example I would want a block which contains voxels within the hippocampus to be negatively associated to an age weight given atrophy associated with age, but a block containing voxels of the ventricular system to be positively associated as they will enlarge with age. How would you go about doing this? The seemingly obvious answer is to look at the distribution of edge covariates on edges incident on the groups. But it is still not very clear exactly what you want to find. In any case, this is a question about a particular research problem, so I don't believe it is appropriate for this list, which is about using graph-tool. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Modelling with community structure data
Am 23.11.21 um 15:46 schrieb James Ruffle: This yields a hierarchical community structure, but how would you most suitably determine what communities were ‘most’ or ‘least’ important/influential/correlated with respect to the edge weight? I have considered whether this might be done with centrality metrics on the blocks (or perhaps vcount and ecount data from a condensation graph on the hierarchical blocks), but was keen to see if you had a more innovative idea... It is impossible to answer this kind of question absent of a very specific context and objective in mind. One of the biggest sins in network science is the proliferation of centrality metrics that attempt to define which node is "best" or "most important" as if there was a general answer to this question. So, I can't tell you which community is most "important"; you have to tell me what you mean by this. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Vertex pointer not updated on graph filtering
Am 15.11.21 um 17:55 schrieb Monecke, Stephan: Hi together! I just noticed that vertices that had been saved within an external list are not being updated when filtering the graph: >>> G.set_edge_filter( G.edge_properties[ 'e_isbus'], inverted=True) >>> G.set_vertex_filter(G.vertex_properties['v_isbus'], inverted=True) >>> print( [ int(n) for n in node.all_neighbors() ] ) [101, 22, 265, 496, 518, 22, 265, 496, 101, 518] >>> print( [ int(n) for n in G.vertex( int(node) ).all_neighbors() ] ) [101, 22, 265, 22, 265, 101] Did I miss something or is this a bug? I don't view this as a bug. Vertex descriptors are supposed to be ephemeral objects that you create in an ad hoc way. For performance reasons they keep a direct pointer to the graph view that created them. If you want to store vertices and look them up, it's better just to store their index, and get a descriptor at the time you need it. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: All Pairs Shortest Path with limitations
Am 17.11.21 um 19:22 schrieb Monecke, Stephan:
The main reason is that shortest_distance() has a "single source, all
targets" mode but no "single target, all sources" mode (the reverse) -
as soon as "source" is empty, it defaults to APSP, neglecting "target".
Of course it does. Just reverse the graph with:
GraphView(g, reversed=True)
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Core dump
Am 11.11.21 um 10:47 schrieb Monecke, Stephan: Any ideas on how I can debug this? Yes: try to isolate the problem by constructing a minimal, self-contained program that reproduces the crash. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: overlap/mixed-membership sbm help
Am 11.11.21 um 03:03 schrieb Eli Draizen:
Hi everyone,
I was wondering if it would be possible to provide some more examples of
how to run a nested mixed membership SBM with edge weights. The new
version seems to have removed the "overlap=True" option for state_args
in the minimize_* functions.
Indeed, I will add more examples about this. Could you please open an
issue in the website so I don't forget?
Is this the correct way to do it now?
import graph_tool as gta
import numpy as np
g = # build graph
e_score = #Set edge weights
state_args = dict(
deg_corr=deg_corr,
base_type=gta.inference.overlap_blockmodel.OverlapBlockState,
B=2*g.num_edges(), #B_max
deg_corr=True,
recs=[e_score],
rec_types=["real-normal"])
state = gta.inference.minimize_nested_blockmodel_dl(
g,
state_args=state_args,
multilevel_mcmc_args=dict(verbose=True))
# improve solution with merge-split
state = state.copy(bs=state.get_bs() + [np.zeros(1)] * 4, sampling=True)
for i in range(100):
if i%10==0: print(".", end="")
ret = state.multiflip_mcmc_sweep(niter=10, beta=np.inf,
verbose=True)
This is correct. But note that the "sampling=True" option is no longer
needed.
I am currently running this for a fully connected bipartite graph with
3454 nodes and 55008 edges. I understand it would take longer than the
non-overlapping version, but do you have any suggestions on how to speed
it up? The non-overlapping version takes about 15 minutes, while the
overlapping version is still running after 1 day.
The new version will contain a much faster code for the overlapping case!
But in the mean-time, what you can do is to fit the non-overlapping
model first, and use that as a starting point to the MCMC with overlap.
You do that simply by doing:
state = state.copy(state_args=dict(overlap=True))
Best,
Tiago
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Help - Unable to make install
D SYSTEM ADMINISTRATOR - RHCSA 8 Certification ID 210-168-829 <https://rhtapps.redhat.com/certifications/badge/verify/D5UDWS5NEBPLMO5ZTQQZHQIERQAEQU3CUPSQX2KSDXT6RW46LQ3T7ULZ55KZZ56SKO7EQ3ETTLYZQ4U5NQYTCNA62RUWOCM34WWBUYQ=> <https://redhat.com/trusted> ___________ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Speed up graph_tool.topology.shortest_distance
Am 27.10.21 um 13:20 schrieb Monecke, Stephan: On my test graph with 4201 nodes and 9683 edges I already tried a quick test with both: I need ~ 70 random (source, target) node pairs to approximate the real mean of all pairs shortest path with an error of about 1 %. This is about 40 % of the run-time of all pairs shortest path. Using single source all targets shortest path I need to sample at least 6 random sources to approximate the real mean of all pairs shortest path with an error of about 1 %. This is about 45-50 % of the run-time of all pairs shortest path. So, although single source all targets shortest path is waay more efficient in it's computation than shortest_path manually it apparently is not a good candidate for subsampling and effectively doing worse. I would not try to generalize much from this, since it is likely to depend substantially on the underlying graph. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Speed up graph_tool.topology.shortest_distance
Am 26.10.21 um 16:23 schrieb Tiago de Paula Peixoto: It is possible to pass a single source but multiple targets. It is possible to pass also a single source and no target, in which case the function computes the distances to all other vertices. By sub-sampling the source vertices you can avoid the "overhead" you were referring to. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Speed up graph_tool.topology.shortest_distance
Am 26.10.21 um 16:16 schrieb Monecke, Stephan: Hi! I use graph_tool.topology.shortest_distance for an all pairs shortest path calculation, what is the main run-time footprint of my algorithm and way to large. How would you speed it up / tackle this? If I knew of a general faster way to solve the all-pairs shortest path problem, I would have implemented it. I tried to sub-sample with manual source, target pairs but that's terribly inefficient since it does not use the internal bookkeeping. Nice would be graph_tool.topology.shortest_distance(G, U, V), where U and V are lists of same length with sources / targets but that's not implemented. It is possible to pass a single source but multiple targets. Subsampling is usually a good technique to reduce the computation time, but it is hard to know what is applicable to you. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: memory handling when using graph-tool's MCMCs and multiprocessing on large networks
Am 22.10.21 um 21:45 schrieb Sam G: thanks for your reply ``It is clear to you that if you run 10 processes in parallel you will need 10 times more memory, right?`` this is clear to me. however 2GB*10 ~= 20GB, and the machine has 260GB memory. so unless algorithm is creating 10 copies of the graph within each iteration, it should be within bounds The inference algorithm has internal data structures that take more memory than the original graph. For instance, it needs to keep track of the number of edges between groups, the degree distribution inside each group, several partitions during the bisection search, etc. This is all done with a trade-off in favor of overall speed, rather than memory usage. ideally the parallelization would not make a copy of the entire graph. since the edges are fixed, all it needs to do is make vertex properties. i haven't figured that out yet. i was hoping someone might have tips on how to do this in graph-tool. there are ways to do this with other objects (data frames, lists), but i am not sure how to approach it with graph-tool graphs Graph-tool graphs are not any different than any other data structure in Python in this regard. Parallelization with the multiprocessing library is implemented by spawning multiple processes (via fork()), each with its own copy of the data. Most systems have a copy-on-write implementation, where initially the cloned processes point to the same memory. But as soon as this is modified, the memory usage is duplicated. Sometimes in Python it can happen that a read will trigger a copy because of the reference counting, but this should not happen with Graph objects. But I think in your case the memory use that you are experiencing is simply due to the 10 different BlockState objects being created, together the memory usage required for the minimize_blockmodel_dl() algorithm. I don't think there is very much you can do, except use a model that requires less internal book-keeping, e.g. PPBlockState. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: memory handling when using graph-tool's MCMCs and multiprocessing on large networks
Am 22.10.21 um 06:08 schrieb Sam G: thanks for your reply. here's an example running minimize_blockmodel_dl() 10 times on 10 cores. when i run this on a large network (2GB, 2M vertices, 20M edges) i get a MemoryError. It is clear to you that if you run 10 processes in parallel you will need 10 times more memory, right? BTW, your example is not self-contained, because I cannot run it without the 'large_graph' file. Maybe you can reproduce the problem for a smaller graph, or show how much memory you are actually using for a single process. It would also be important to tell us what version of graph-tool you are running. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: memory handling when using graph-tool's MCMCs and multiprocessing on large networks
Am 21.10.21 um 22:38 schrieb Sam G: hi, i was wondering if anyone had any tips on conserving memory when running graph-tool MCMCs on large networks in parallel. i have a large network (2GB), and about 260GB of memory, and was surprised to receive a MemoryError when i ran 10 MCMC chains in parallel using multiprocessing. my impression is that MCMC was relatively light on memory. As usual, it is not possible to say anything concrete without a minimal complete working example that shows the problem. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: error: Received Orphaned Property Map
Am 04.10.21 um 20:27 schrieb sam.gyet...@gmail.com: i wasn't aware of the own_property() function and can't find it in the graph-tool documentation (other than seeing it being used in examples involving visualization). whatever it does, it seems to work just do: help(Graph.own_property) -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: running minimize_blockmodel_dl() in parallel
Am 05.10.21 um 05:06 schrieb Sam G: hi, here is a simple example where i run minimize_blockmodel_dl() 10 times in parallel using multiprocessing and collect the entropy. when i run this, i get the same value of entropy every single time. ``` import multiprocessing as mp import numpy as np import time import graph_tool.all as gt # load graph g = gt.collection.data["celegansneural"] N_iter = 10 def get_sbm_entropy(): np.random.seed() state = gt.minimize_blockmodel_dl(g) return state.entropy() def _parallel_mc(iter=N_iter): pool = mp.Pool(10) future_res = [pool.apply_async(get_sbm_entropy) for _ in range(iter)] res = [f.get() for f in future_res] return res def parallel_monte_carlo(iter=N_iter): entropies = _parallel_mc(iter) return entropies parallel_monte_carlo() ``` result: [8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546, 8331.810102822546] ultimately i would like to use this to keep entropy as well as the block membership vector for each iteration any ideas? The line np.random.seed() initializes numpy's RNG, which different from the one used in graph-tool (which is in C++). You need to replace that line with: gt.seed_rng(0) -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: error: Received Orphaned Property Map
Am 04.10.21 um 10:29 schrieb sam.gyet...@gmail.com:
hi,
i received this error while trying to assign a vertex property map to a graph,
and i'm not sure how to get around it.
context: i am running minimize_blockmodel_dl() many times in parallel using
microprocessing. because my graph is large, i don't want it to save multiple
copies of the graph. so i am trying to only keep the block id each time.
however, when i try to do this, i get the above error
here's a somewhat simplified example:
```
import graph_tool.all as gt
import multiprocessing as mp
pool = mp.Pool(5)
def fit_sbm():
state = gt.minimize_blockmodel_dl(g)
b = state.get_blocks()
return b
blocks = pool.map(fit_sbm, range(5))
pool.close
b0 = blocks[0]
membership = g.new_vertex_property("int")
membership = b0
g.vertex_properties["membership"] = b0
```
could you explain why this error comes up, and also if possible suggest an
alternative way to proceed?
The code you provided does not run. Could you please provide a minimal
_working_ example that shows the problem?
Additionally, you seem confused about how variable assignment works in
Python, since the first line does nothing in:
membership = g.new_vertex_property("int")
membership = b0
And these two lines are also irrelevant to what follows, since you never
use the variable "membership" again.
Presumably, you wanted to do something like:
g.vp["membership"] = g.own_property(blocks[0])
But I can only guess.
Best,
Tiago
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: gpg: keyserver receive failed: General error
Am 01.10.21 um 02:44 schrieb behnam nikparvar: Hi Tiago, I am not sure if this issue is sourced from graph-tool. When I try to download the public key, I get this error: echo "deb http://downloads.skewed.de/apt bionic main" >> /etc/apt/sources.list apt-key adv --keyserver keys.openpgp.org --recv-key 612DEFB798507F25 Error: Executing: /tmp/apt-key-gpghome.wVeQRbY1M1/gpg.1.sh --keyserver keys.openpgp.org --recv-key 612DEFB798507F25 gpg: keyserver receive failed: General error Indeed this has nothing to do with graph-tool and I have no control over the keys.openpgpg.org key server. Please check your network connectivity, try another key server, or try to download the key directly via the browser. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: linux installation: gtk-query-immodules-3.0
Am 30.09.21 um 07:56 schrieb sam.gyet...@gmail.com: hi, installing on linux using conda-forge gives the following error anaconda3/envs/gt/bin/gtk-query-immodules-3.0: error while loading shared libraries: libXcursor.so.1: cannot open shared object file: No such file or directory ERROR: Failed to complete gtk3's post-link script. To fix this, activate the environment and run: glib-compile-schemas "/home/anaconda3/envs/gt/share/glib-2.0/schemas" gtk-update-icon-cache -f -t "/home/anaconda3/envs/gt/share/icons/hicolor" gtk-query-immodules-3.0 --update-cache unfortunately running the three suggested lines gives another error gtk-query-immodules-3.0: error while loading shared libraries: libXcursor.so.1: cannot open shared object file: No such file or directory and running graph-tool also gives an error any ideas? Please report this problem to the conda-forge project: https://github.com/conda-forge/graph-tool-feedstock/issues -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: ValueError: invalid edge descriptor. How to debug?
Am 03.09.21 um 00:18 schrieb Gerion Entrup: My question: How can I debug this and find out, why it is invalid? Can I do/print anything in Python? Should I use GDB? Where would be a good place to set a breakpoint? I cannot print the faulty edge. It is fairly complicate to make an minimal example. My whole code uses no remove_edge() or remove_vertex(). I'm using graph_tool 2.43. What you should do is produce a minimal example that shows the problem... No debugger is going to replace this simple strategy. From the code fragment that you have shown, it's not possible to say much. I notice that you are subclassing Graph, and probably omitting to us specializations that you are making to Graph.vertex() and other methods. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: random graph
Am 01.09.21 um 19:24 schrieb Davide Cittaro: Following up this post, I have large datasets for which I already have a NSBM. I wanted to speed up thinking to an approximate model, so I subsampled a fraction of nodes (randomly chosen) and performed NSBM, then performed a sort of label transfer to the original graph. Except for the fact the partitions at level 0 are now larger than the original ones (as expected) I noticed a general concordance between the communities using subsampled and full graphs. Do you have some literature, ideas or hints about analysis of subsamples? If we assume that the big graph is sampled from a SBM, then the sub-sampled graph would also be sampled from a SBM, but not from the same one, if we are dealing with sparse networks. The sub-sampled SBM would be sparser (smaller average degree), and have a deformed degree distribution in the case of the DC-SBM. The intuition here is that the evidence for the underlying structure will become weaker after sub-sampling, according to how sparser the network becomes. With the MDL/Bayesian approach in graph-tool, you should see fewer groups in the sub-sampled network, but they should otherwise be similar to the full network. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Error with get_edges_prob
Am 01.09.21 um 10:34 schrieb Davide Cittaro: Uhm... I will, in the meantime I realized that I was not specifying the missing edges properly. In fact, testing a working example on gt 2.40 made me realize I have to pass the layer as well for missing edges (source, target, layer), is this correct? Now, on another installation with gt 2.43 that doesn't work anymore. I will open an issue for gt 2.43 only, but as a general question: is it possible to have the probability of a missing edge for a specific layer when LayeredBlockState is used? Not only it's possible, but it is the only way possible. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Error with get_edges_prob
Hi, Please open an issue in the website, including a minimal working example that shows the problem, the graph-tool version used, and all the other requested information... Best, Tiago Am 28.08.21 um 13:26 schrieb Davide Cittaro: Hi all, I have a NestedBlockState with LayeredBlockState as basis, two layers. I would like to predict edges that are missing in one or the other layer (besides, is this even possible?). However, given a list of missing edges (and a list of spurious ones) I tried this a_state.get_edges_prob(missing, spurious) and get this error AttributeErrorTraceback (most recent call last) in > 1 a_state.get_edges_prob([missing], spurious) ~/anaconda3/envs/experimental/lib/python3.8/site-packages/graph_tool/inference/nested_blockmodel.py in get_edges_prob(self, missing, spurious, entropy_args) 441 lstate._state.clear_egroups() 442 --> 443 L += lstate.get_edges_prob(missing, spurious, entropy_args=eargs) 444 if isinstance(self.levels[0], LayeredBlockState): 445 missing = [(lstate.b[u], lstate.b[v], l_) for u, v, l_ in missing] ~/anaconda3/envs/experimental/lib/python3.8/site-packages/graph_tool/inference/blockmodel.py in get_edges_prob(self, missing, spurious, entropy_args) 1204 pos[v] = self.b[v] 1205 -> 1206 self.remove_vertex(pos.keys()) 1207 1208 try: ~/anaconda3/envs/experimental/lib/python3.8/site-packages/graph_tool/inference/blockmodel.py in remove_vertex(self, v) 1144twice. 1145 """ -> 1146 self._state.remove_vertex(int(v)) 1147 1148 def add_vertex(self, v, r): AttributeError: 'graph_tool::BlockState -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Boost error on edge properties
Am 25.08.21 um 05:19 schrieb a...@alan.gold: # print the edge names for e in G.iter_edges(): ...print(e) ...print(G.ep.name[e]) This should have been either: for e in G.edges(): print(G.ep.name[e]) or for s, t, n in G.iter_edges([G.ep.name]): print((s, t)) print(n) Please read carefully the documentation to understand the difference between G.iter_edges() and G.edges(). -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Writing partition labels to state object
Am 24.08.21 um 18:01 schrieb Lietz, Haiko: Dear all, Given a NestedBlockState object called state, I can obtain the list of nested group belongings by x = state.get_bs() I can then order the partitions by y = order_nested_partition_labels(x) Is there a way to store the ordered labels in the original NestedBlockState object? How do I have to write that? Just create a new NestedBlockState (or, equivalently, copy the from the old one): state = NestedBlockState(g, bs=y) or state = state.copy(bs=y) Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 24.08.21 um 12:49 schrieb arttu.malkam...@helsinki.fi: What is the difference between A) fitting a separate model to each layer and B) fitting an overlapping model with layers=True? When interpreting my results, what can I say about the dependencies between layers with discrete types of relationships? With B) the overlapping partition is encoded in a manner that takes correlations between the layers into account, so that the group memberships will vary between layers only if there is enough evidence requiring this. With A) the independent models do not take advantage of any similarity between layers. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: C++ extension: retrieve the filtered graph
Am 20.08.21 um 22:09 schrieb Oliver Baumann:
Tiago de Paula Peixoto wrote:
Am 20.08.21 um 20:51 schrieb oliver.baumann(a)uni-bayreuth.de:
Hi all,
I'm currently offloading construction of the uni-modal projection of a bipartite
graph from Python to C++ (NB: I am not very familiar with C++ or BGL, _yet_).
Thanks to the docs on writing such extensions, I know roughly where I'm
heading.
This is the signature of the projection-function:
template
void projection(const Graph , ProjectedGraph )
Here, g is the bipartite graph, and pg is constructed in Python as:
pg = Graph(GraphView(g, vfilt=lambda v: g.vp.type == 0))
This would remove every vertex from the graph, since g.vp.type == 0 will
always return False.
This was a typo whilst drafting my message, it is of course `lambda v:
g.vp.type[v] == 0`.
I also do not understand why do you construct a new Graph object,
instead of working with the GraphView directly.
To my understanding, modifying the GraphView by adding edges to it would also reflect in
the original, bipartite graph, which I don't want. Would an EdgePropertyMap only on the
view with a binary "exists/missing" be the cleaner approach, you reckon?
If you don't want to modify in place, then you should remove the
filtering altogether by passing "prune=True" to the Graph() constructor.
This will make the code faster.
Slightly hijacking the topic, is there any room for improvement on the
projection code:
#include
namespace gt = graph_tool;
template
void projection(const Graph , ProjectedGraph ) {
// iterate the filtered nodes, type==0
for (const auto v : gt::vertices_range(pg)) {
// iterate the neighbourhood of type == 1
for (const auto direct_neigh : gt::all_neighbors_range(v, g)) {
// iterate this neighbourhood, again of type == 0 to determine edges
between type-0-nodes
for (const auto next_neigh : gt::all_neighbors_range(direct_neigh, g)) {
if (v != nn) {
boost::add_edge(v, next_neigh, pg);
}
}
}
}
}
This looks like it adds every edge twice...
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: C++ extension: retrieve the filtered graph
Am 20.08.21 um 20:51 schrieb oliver.baum...@uni-bayreuth.de: Hi all, I'm currently offloading construction of the uni-modal projection of a bipartite graph from Python to C++ (NB: I am not very familiar with C++ or BGL, _yet_). Thanks to the docs on writing such extensions, I know roughly where I'm heading. This is the signature of the projection-function: template void projection(const Graph , ProjectedGraph ) Here, g is the bipartite graph, and pg is constructed in Python as: pg = Graph(GraphView(g, vfilt=lambda v: g.vp.type == 0)) This would remove every vertex from the graph, since g.vp.type == 0 will always return False. I also do not understand why do you construct a new Graph object, instead of working with the GraphView directly. I am thus passing in the graph containing only the vertices I wish to project onto, and add the edges between them in C++. In Python, pg.num_edges() correctly returns zero. In C++, it returns the same number of edges as the base-graph it was constructed from, g. How can I obtain the filtered view of pg in C++, which I expect to have no edges? Since you have not provided any code, I can only guess what is going on. A typical pitfall is that the C++ function num_edges(g) will always return the number of edges in the _unfiltered_ graph, even if the graph is being filtered. This is due to the BGL semantics that requires the function to be computed in time O(1). But that does not mean that the graph is not being filtered... If you iterate over it, you will get what you expect. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Assortative constraint?
Am 15.08.21 um 21:51 schrieb dadakinda: Hello, Is there any way to bias the nested DC-SBM model towards assortative partitions? I've tested PPBlockState, but as far as I understand it distinguishes groups only by internal vs external density via the Planted Partition model: far more restrictive than the full SBM. I'm wondering if there is a middle-ground where you have a SBM but the groups are biased (required?) to be assortative. I searched for something like this and found a couple papers: "A Regularized Stochastic Block Model for the robust community detection in complex networks" by Lu et al and "Assortative-Constrained Stochastic Block Models" by Gribel et al. Unfortunately neither uses priors to prevent overfitting, a nested formulation to address resolution limit, etc. The latter paper restricts the intra-block connection probabilities to be greater than inter-block connection probabilities. I'm not sure if there is an equivalent idea for the microcanonical model. The only idea I can think of would be to modify the likelihood in some way to make intra-block edges more likely. Such model variations are not implemented in the library. Furthermore, I don't think that the microcanonical priors / integrated likelihoods will be easy to write down in closed form for this kind of constraint. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Creating a layered graph (aka messing with property maps)
Am 06.08.21 um 16:38 schrieb Davide Cittaro:
Hi all,
I'm trying to analyze a graph for which I have multiple layers and I'm finding
difficulties in creating the graph. I have two graphs with the same vertex for
which I define a edge property (the layer) like this:
(g_atac, dist_atac) = gt.generate_knn(A_data, k=n_neighbors)
(g_rna, dist_rna) = gt.generate_knn(R_data, k=n_neighbors)
layer_atac = g_atac.new_edge_property('int')
for e in g_atac.edges():
layer_atac[e] = 1
layer_rna = g_rna.new_edge_property('int')
for e in g_rna.edges():
layer_rna[e] = -1
g_atac.edge_properties['layer'] = layer_atac
g_rna.edge_properties['layer'] = layer_rna
I then merge the graphs by graph union, first defining a node mapping:
rna_mappings = g_rna.new_vertex_property("int")
for x in range(g_atac.num_vertices()):
rna_mappings[x] = x
and then performing the actual union:
gu = gt.graph_union(g_atac, g_rna, intersection=rna_mappings,
internal_props=True)
This indeed creates the final graph, but I noticed that instead of two layers I
have three:
np.unique(gu.edge_properties['layer'].a)
PropertyArray([-1, 0, 1], dtype=int32)
Looking back at the start graphs I have zeros in the edge_properties as well
np.unique(g_rna.edge_properties['layer'].a)
PropertyArray([-1, 0], dtype=int32)
np.unique(g_atac.edge_properties['layer'].a)
PropertyArray([0, 1], dtype=int32)
and, similarly, the elements in the edge property vector are much higher than
the number of edges:
print((len(g_rna.ep['layer'].a), g_rna.num_edges()))
(156672, 14317)
which is different from what I can get from this:
g_rna.get_edges([g_rna.ep['layer']]).shape
(14317, 3)
I'm evidently messing with (internal) properties and I'm clueless, any advice?
As is explained in the documentation, edge indexes need not to be
contiguous (differently from vertex indexes, which are always contiguous).
When accessing the property map values via the ".a" property, this
returns an array indexed by the edge index, which will contain values
for non-existing edges if the indexes are not contiguous. This is what
you are seeing.
If you want to see only the property values for existing edges, you
should get a filtered array with the ".fa" property.
Best,
Tiago
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Fixed vertex size between different layouts
Am 29.07.21 um 13:23 schrieb Botond Molnár: Hello, I am trying to plot a few networks using two different layouts. The vertex sizes and edge width are calculated separately, then each networks is plotted using two different layouts: radial tree layout and sfdp layout. In both cases the same pre-calculated values are used to set the vertex sizes and edge width, in order to maintain the comparability of the plots. However, I end up with different vertex sizes and edge width in function of the chosen layout. I am using the cairo backend, so I can plot both layouts next to each other, but the ax sizes are the exactly the same. Can you please help me how can I sort this out? You can set the vertex size and edge with explicitly, by passing values to the vertex_size and edge_penwidth parameters of the graph_draw() function. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Adding property maps of custom types
Am 03.08.21 um 13:32 schrieb sebyakin.a: One more question: if I want these c++ objects, that are exposed to python with boost::python, to be an internal property maps and so be (de)serialized with the graph (from)into .gt format, should I write a boost::python pickle support functions for these classes? Will it work? Yes, pickling is supported for property maps of type boost::python, provided the values themselves can be pickled. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Hide filtered vertices in interactive display
Am 16.07.21 um 02:27 schrieb anton-...@hotmail.fr: Hi, I want to interactively display a filtered graph whose filter is updated based on mouse events (eg when the user presses a vertex, it then display/remove its neighbors) The problem i have is that when hovering over the window with the mouse, I am still able to select or highlight a filter vertex. Is there an easy way to fully hide the filtered vertices? Without a minimal working example that shows the problem you are describig, there is not much that we can help with... -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Adding property maps of custom types
Am 14.07.21 um 09:46 schrieb sebyakin.a: Thank you for the reply! Do you know, is there's any noticeable overhead of frequent python::extract() calls? Is it urgent to have some pre-extracted data-structure on c++ side, or this is similar to having a pointer? It will incur an overhead, since the python interpreter needs to be involved. So it's better to do this once, ahead of time, and preferably store all property map values in a single python object. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Adding property maps of custom types
Am 13.07.21 um 21:58 schrieb sebyakin.a: Hello, My question is: which parts of a code I should modify to add internal property maps of custom type? Is it possible without ground-breaking modifications, or should I look a way for some workaround? My usecase is to bridge python integer objects and big integer objects (from intx library) on c++ side, as a vertex property. I'm going to perform relatively heavy math operations and graph operations, so I want to write a C++ extension that does it. So I thought, is it possible to add custom property map handlers for big integers, that will convert python long objects to intx type and store it in that type later. The simplest thing for you to do is to use Boost.Python to reflect your custom types to python, which then you can store in a property map of type "python::object". You can access the property map values from your C++ extension by using python::extract(). -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 11.07.21 um 23:31 schrieb arttu.malkam...@helsinki.fi:
Hi,
I'm sorry, I'll try to be more specific.
I eventually need a matrix that I can plot as done in this post by ulgenklc:
https://nabble.skewed.de/Temporal-community-detection-with-DSBM-tt4028581.html#a4028592
Here, I use a subgraph with only two layers to demonstrate this seemingly
trivial issue:
np.random.seed(42)
gt.seed_rng(42)
state = minimize_nested_blockmodel_dl(g, state_args=dict(base_type=LayeredBlockState,
state_args=dict(deg_corr=True, overlap=True, layers=True, ec=g.ep.layer,
recs=[g.ep.weight], rec_types=["discrete-binomial"])))
This yields:
While there are 4610 overlapping blocks in this stage, state.draw() divides the
nodes neatly into three sensible communities. Hence, I know that the snippet of
code that I need already exists somewhere in draw_hierarchy.
Then, I filter the edges in state.g by layer and use get_edge_blocks() to
extract the layer-specific half-edges:
lay = []
for e in g.edges():
lay.append(g.ep.layer[e]==0) #edges in the first layer
lay = np.array(lay, dtype="int")
ft = g.new_edge_property("bool")
ft.a = lay
state.g.set_edge_filter(ft)
be = state.levels[0].get_edge_blocks()
This is where I struggle to find a way to extract the desired information. My
best guess on how to extract the layer-specific node memberships from _be_ is
the following:
s = OverlapBlockState(state.g, b=be) #state.g with filter on
mb = s.get_majority_blocks()
for v in g.vertices():
print(mb[v])
If I compare the _mb_ for the discrete layers, the memberships vary slightly
(this is what I need), but I'm unsure whether this is the correct way to
extract this information?
If "be" is the property map returned by
state.levels[0].get_edge_blocks(), then
bm = g.new_vp("vector", val=[0] * state.levels[0].get_B())
for e in g.edges():
if layer[e] != 0:
continue
u, v = e
u, v = sorted((u, v))
r, s = be[e]
bm[u][r] += 1
bm[v][s] += 1
will give you the memberships in each group at layer 0 in the property
map "bm". Is this what you want?
Best,
Tiago
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 11.07.21 um 17:49 schrieb arttu.malkam...@helsinki.fi: I was referring to the decomposition of layer-specific node membership from get_edge_blocks. So, how do I process the _be_ returned by layers[0].get_edge_blocks() to yield this information as an np.array or vertex_property_map? I'm sorry, but this is not helping. You are not making any substantial effort to be specific, despite my asking, and it's still not clear what you want that is not already given. A half-edge points to a node. Each half-edge belongs to a group, as given by get_edge_blocks(). Each half-edge also belongs to a specific layer. What else do you need? -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 11.07.21 um 17:26 schrieb arttu.malkam...@helsinki.fi: Coming back to my initial question concerning the layer-specific partitions, it seems that I was too optimistic about being able to decompose this information from get_edge_blocks. I have already spent time reading the docs and the use of_be_ in source codes to convert the block membership of each half-edge to layer-specific block membership of each node, so I need to ask for more instructions. There are a few previous posts about extracting this information but none of the answers addresses this decomposition issue beyond get_edge_blocks. I presume there is a relatively straightforward way to do this? Do what? Please ask a specific question. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 09.07.21 um 14:29 schrieb arttu.malkam...@helsinki.fi: Oi! The graph has 220 nodes and 11322 edges divided into 9 layers, so it is not huge. But this seems like a good way forward - I'll keep on experimenting. It seems really strange that it would take _days_ for such a small network! There must be something wrong going on. Try the step-wise approach, maybe it will help you to track it down. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Model specification and extraction of layer-specific partitions in LayeredBlockState when overlap=True
Am 09.07.21 um 12:47 schrieb arttu.malkam...@helsinki.fi: Q1) This is taking very long (days) on a high performance cluster and actually I haven't seen it finishing up for once. Is my model specification correct or simply too complex? If the latter, is there a better strategy to perform my analysis? All I can think of is to revert to a non-nested model and/or reduce the number of layers and/or edges: Very hard to say something, without more concrete information. How large is your network? You are using the most complex combination possible, so indeed it should take longer than the simpler variants. My overall suggestion is to always try the minimize_nested_blockmodel_dl() function first, and (time permitting) use that as a starting point to the MCMC. Furthermore, I would suggest starting simple, using non-overlapping, unweighted and non-layered models first, and seeing how far you get, and building understanding from your data that way. You can then introduce more elaborate models step by step. Q2) Say that the model runs and yields a neat NestedBlockState (or LayeredBlockState) object with X overlapping blocks, 9 layers, degree-corrected, with 2 edge covariates, for graph . Now, in previous posts there are mentions about the number of nodes multiplying by the number of layers when 'overlap=True', but in my experiments with only two layers, this does not seem to be the case and I struggle to find a way to extract the layer-specific block and respective memberships of nodes. I've tried iteration over vertices, but what I eventually need is a 2d_array [y=all nodes in g, x=all layers in g, values=most likely block membership] in order to map how/if the block membership of each node varies across those layers in which they participate (ideally somehow denoting non-participation in the layer with NaN or some unique value). Is there some way to get there? In the overlapping model, group membership is a property of _edges_ (actually, half-edges), not nodes directly. Take a look at OverlapBlockState.get_edge_blocks(), which will give you this information. From this, you should be able to decompose membership across layers, etc. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Question about recent optimizations
Dear Davide, You can take a look at the git commit history for all the gritty details, but in a nutshell: - The agglomeration algorithm has been moved entirely to C++ (some higher level functions were in Python before) - Many data structures have been improved (e.g. the bookkeeping necessary for move proposals) - The initialization of the agglomeration has been changed: when starting with B=N groups, instead of performing merge/sweeps, we just do single-node sweeps, which have the same effect as merges, but are much faster. Only after the number of groups stops decreasing fast enough, we switch to merges. The last modification turned out to have a big relative impact in practice. Best, Tiago Am 09.07.21 um 09:15 schrieb Davide Cittaro: Nevermind, after further investigations I found that the degradation was only apparent. Still, I'd like to know which are the optimizations that have been included and what do they affect ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Nested partition overlap
Dear Haiko, Am 01.07.21 um 16:21 schrieb Lietz, Haiko: Dear Tiago, I just saw your talk at the Networks 2021 satellite symposium on “Community Structure in Networks.” Congratulations on presenting a complicated paper (https://journals.aps.org/prx/abstract/10.1103/PhysRevX.11.021003 <https://journals.aps.org/prx/abstract/10.1103/PhysRevX.11.021003>) in an accessible way. In that paper, you develop ways to return more meaningful (i.e., based on consensus) point estimates from the posterior distribution of the SBM. In the talk, you said the method can just as well be applied to multiple solutions of any clustering algorithm. The partition_overlap_center() function for flat partitions works fine with either lists from a posterior distribution or from multiple runs of an SBM. But the nested_partition_overlap_center() function crashes (version 2.37) when the nested partitions (for which the consensus is to be obtained) do not have the same depth. Please add an issue in the website with a minimal working example that shows the problem, and it will be fixed. Is there a principled problem with making nested_partition_overlap_center() applicable to nested partitions of different depth? Is it something you would like to do for the next release or is there a quick fix? This trivial to fix by adding levels to the shortest hierarchy with a single group each. I probably just forgot to account for this, since it didn't occur to me to pass it two hierarchies of different depths. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: property map values are not loaded for .graphml graph
Am 28.06.21 um 20:48 schrieb bnikp...@gmail.com:
Hi,
I have saved a graph in Networkx with .graphml format. I can successfully load
the graph in graph-tool and I can see the edge property attached to graph.
However, when I try to access the values I get an error that the property does
not exist.
G =
gt.load_graph('gdrive/MyDrive/ColabNotebooks/Thesis/sources/k4p3-2-test.graphml')
G.properties
{('e',
'_graphml_edge_id'): ,
('e',
'etype'): ,
('v',
'_graphml_vertex_id'): }
e = G.edges().next()
etype[e]
NameError: name 'etype' is not defined
You seem unfamiliar with the Python syntax. In Python, objects need to
be declared before they are used.
The correct way to obtain the property map would have been:
etype = G.ep["etype"]
etype[e]
--
Tiago de Paula Peixoto
___
graph-tool mailing list -- graph-tool@skewed.de
To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Maximum-Matching using graph-tool
Am 05.06.21 um 23:12 schrieb chaitanyaagarwal...@gmail.com: But on a separate note, can you suggest me some general tips to optimize memory usage while using this library? For example, I found the shrink to fit function for edgePropertyMap and graph, highly impactful. Are there any other such implementations, tools or tricks that one can use? Again, it is difficult to say without any particular context. The function shrink_to_fit() should help only after the graph has been heavily modified, not otherwise. So in order for us to know if there is any technique that can improve memory use in your particular circumstances, we have to know first what it is... -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Maximum-Matching using graph-tool
Dear Chaitanya, Am 05.06.21 um 21:35 schrieb chaitanyaagarwal...@gmail.com: Hi I am working on a project where I need to use the maximum cardinality matching algorithm. My graphs are complete and weighted. However, even for moderately large graph instances, I am facing a segmentation fault error. I have complete graphs with 400-5000 vertices (I would like to run the algorithm on much bigger graphs too). The algorithm works fine for smaller instances. It seems that this is a memory issue. My machine has more than enough memory to run such instances (16 GB). Further, earlier I wasn't even able to run graphs with 100-200 nodes. I was able to run them after changing the edge weight property type from "double" to "int_16". Please let me know if there is something else I can do to run this code. It's a pretty large and distributed codebase, so it would be difficult for me to post my code for reference over here. On the other hand, if you feel that this is an issue with the library itself, please fix it. If you want us to address any problem in the library, please provide a minimal working example that shows the problem. Just saying you got a segfault is not actionable information, since we cannot easily reproduce it. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Generate SBM with edge weights
Am 01.06.21 um 17:25 schrieb Davide Cittaro: It seems this feature has already been requested :) https://git.skewed.de/count0/graph-tool/-/issues/671 <https://git.skewed.de/count0/graph-tool/-/issues/671> Indeed! :-) -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Generate SBM with edge weights
Dear Davide, Am 28.05.21 um 20:05 schrieb Davide Cittaro: Hello, I need some hints about generating graphs using gt.generate_sbm. I had myself used it to generate a random graph given a predefined modular structure passing blocks and their connectivities. I have now a more complicated problem: I'm inferring communities from a complete weighted graph and I want to generate random graphs using the inferred structure. Is it possible to generate the graph with edge weights modeled after the types used to define the original BlockState? Direct sampling from the SBM with edge covariates is not yet implemented! This is not very difficult to do "by hand", bit it would be a nice addition to the library. Could you please open an issue in the website for this request, and I will try to include it for the next release. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Graph Tool Edge Attribute Usage
Am 17.05.21 um 12:00 schrieb petkov.1...@gmail.com: Hi, I have a question about edge attributes. Currently, I am using graphs that have three edge properties. The first one is a float which serves as the edge weight. The second one is a string that can only be two values and finally, I have an edge id. Does the nested stochastic block model use the weight and the other binary string property for finding the hierarchies? Yes, it's possible to include edge weights and covariates into the nested SBM. Take a look at the documentation here: https://graph-tool.skewed.de/static/doc/demos/inference/inference.html#edge-weights-and-covariates Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: The graph-tool build consumes too much memory
Am 07.05.21 um 20:17 schrieb Marat Idrisov: Yes, I use NJOBS=1 I never needed 10GB to compile it with only one job... But in any case, I'm afraid there is not much that can be done. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: The graph-tool build consumes too much memory
Am 07.05.21 um 19:41 schrieb Marat Idrisov: Hi. I have a device with an Apple M1 chip and I'm trying to build a graph-tool deb package in a debian docker container on an arm64 architecture. I am using the (https://git.skewed.de/count0/graph-tool/-/blob/master/release/debian/Dockerfile) image as the base of the docker file. I understand the dpkg-buildpackage command and the cpp files are being compiled. This process consumes very much memory (about 10 GB). My device doesn't have that much memory and the build crashes. Can we expect there will be a ready version for the arm64 architecture in the near future or can you suggest steps to optimize the build? Change the variable NJOBS=4 to 1. This will reduce the memory usage (but it will increase the build time). -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: issues with multicanonical sampler
Hi, These belong to the issue tracker. Could you please try again to login and post there? Best, Tiago Am 06.05.21 um 10:13 schrieb hazaa: hi, I'm writing to report several issues with the multicanonical sampler, using gt version 2.37. ((haven't been able to login on gitlab tracker) First with the NestedBlockState g = gt.collection.data["celegansneural"] state = gt.NestedBlockState(g) nbins=100 S0 = state.entropy() Smin,Smax = S0*0.90,S0*1.1 ms= gt.MulticanonicalState(state,Smin,Smax, nbins=nbins) gt.multicanonical_equilibrate(ms) Will return: /usr/lib/python3/dist-packages/graph_tool/inference/mcmc.py in sweep(self, **kwargs) 426 427 def sweep(self, **kwargs): --> 428 self._state.multicanonical_sweep(self, **kwargs) 429 430 def get_energies(self): TypeError: multicanonical_sweep() takes 1 positional argument but 2 were given Then with BlockState: state = gt.BlockState(g) nbins=100 S0 = state.entropy() Smin,Smax = S0*0.90,S0*1.1 ms= gt.MulticanonicalState(state,Smin,Smax, nbins=nbins) gt.multicanonical_equilibrate(ms) #THIS IS OK ds,nattempts,nmoves = state.multicanonical_sweep(ms,niter=10 ) The last line fails with the following output: /usr/lib/python3/dist-packages/graph_tool/inference/blockmodel.py in _multicanonical_sweep_dispatch(self, multicanonical_state) 1702 _get_rng()) 1703 else: -> 1704 return libinference.multicanonical_sweep(multicanonical_state, 1705 self._state, _get_rng()) 1706 TypeError: No registered converter was able to extract a C++ reference to type boost::any from this Python object of type NoneType Thanks for this wonderful module! ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Strange outputs for Newman RMI? (graph_tool.inference.partition_centroid.reduced_mutual_information(x, y, norm=False))
Am 03.05.21 um 17:20 schrieb JohannHM: Thank for your comment, Tiago. Would you please recommended me one of the included methods of your gt that is useful for comparing: a. cover, vs partition b. covers, vs covers Nothing in particular comes to mind. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Strange outputs for Newman RMI? (graph_tool.inference.partition_centroid.reduced_mutual_information(x, y, norm=False))
Am 03.05.21 um 12:28 schrieb JohannHM: My final question is about how to deal when one have a node in different partitions. If I have these communities, for example: cover = [ [0,1,2,3], [3,1,5,4] ] How it should be the label partition? This isn't a partition. A partition, by definition, is non-overlapping. RMI is not applicable to this case. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Strange outputs for Newman RMI? (graph_tool.inference.partition_centroid.reduced_mutual_information(x, y, norm=False))
Am 30.04.21 um 18:53 schrieb JohannHM: Hi team. I'm wondering whether you could help me to see what is happening with your reduced_mutual_information() function because of several mismatching outputs I found on this implementation. 1. RMI is a value between [0, 1], but why in your example the output is negative if I compare two partition? x = np.random.randint(0, 10, 1000) y = np.random.randint(0, 10, 1000) gt.reduced_mutual_information(x, y) -0.065562... RMI is _not_ between [0,1]. It can take negative values! The *normalized* value of RMI can take a value of at most one, but it can still be negative. 2. In your example, you create sort of two partitions from a random distribution, Is it not the specific case when RMI is zero, or very close to zero? -0.065562 is close to zero. 3. When I use the exact partitions Newman offer in your own code (wine.txt), your function gives 0.7890319931250596 But the Newman function gives Reduced mutual information M = 1.21946279985 bits per object Why do these results are so different or how can we associate them? Newman's code returns the value in bits (base 2), where in graph-tool the convention is to return the value in nats (base e). Just divide the value obtained via graph-tool by log(2) and the results should match. By the way, for the "wine.txt" example I get: reduced_mutual_information(x, y) -> 0.8452672015130195 reduced_mutual_information(x, y) / log(2) -> 1.2194627998489254 I can only recover your value for norm=True reduced_mutual_information(x, y, norm=True) -> 0.7890319931250596 which is not what is returned by Newman's code. So please pay attention. 4. Finally, what is (or where is) the description of the format one must pass the partitions to the function? I mean, I'm confused about how x (or y) variables should arranged. Each row index is the node label? If so, how to write nodes sharing several partitions? I honestly do not understand the source of confusion. A label partition is a 1D array containing the group labels for each node, indexed by the node index. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Question about entropy
Am 27.04.21 um 14:02 schrieb Lietz, Haiko: The 'polbooks' dataset has 105 nodes. An SBM with one block (B=1) has a DL of about 1550 bits. The DL is minimized (DL_min=1300) for B=5. When each node is in its own block (D=105), DL is maximized (DL_max=1950). Can't I make states of different graphs comparable by taking DL_min/DL_max? It seems like a straightforward application of normalized entropy (https://en.wikipedia.org/wiki/Entropy_(information_theory)#Efficiency_(normalized_entropy)) to me. It's difficult to comment, because I don't know what the objective of the comparison is. If you compute the ratio of the minimum DL with the DL for B=1, this would give you the compression ratio when compared to a baseline random graph model. If you compare this ratio between two networks of two different sizes, this gives you an idea of how more random one is versus the other, when compared to a fully random graph with the same density, but no deeper insight. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: Question about entropy
Am 27.04.21 um 11:50 schrieb Lietz, Haiko: > I am going to compare NestedBlockState.entropy() of the two run, but I am not sure this is correct. > How should I take into account the fact that the networks are slightly different? Would normalization make the two entropies comparable? I’d be interested to hear opinions about using, for normalization, the entropy of a NestedBlockState where each node is in its own group. The description length (DL) tells you how much information is needed to encode both the network and the model parameters. If we compare the DL for the same network but different models, this tells which model most compresses the data. But if we compare two different networks with two different models, this tells us very little, because it mixes a comparison of which network is more regular with the quality of fit of each model. The results of this kind of comparison is often trivial: the more nodes and edges, the higher will be the DL. You *could* compute something like the DL per edge in order to compare two networks, but since the DL is not a linear function of the number of nodes or edges, it is difficult to put this evaluation on solid statistical grounds. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: getting marginals for nested PartitionModeState
Dear Davide, Am 24.04.21 um 16:20 schrieb Davide Cittaro: Hi all, I have my PartitionModeState for multiple NestedBlockState models. Using pmode.get_marginal(g) I can retrieve the 2d matrix of the node marginals for the deepest level of the nested hierarchy. If M contains the marginals for level 0 and C is a binary matrix with correspondences between level 0 (rows) and any other level (columns), then M @ C gives me the node's marginals for any other level in the hierarchy. As NSBM groups blocks hierarchically, how can I get the marginals of blocks identified at any level for the containing level? e.g. if I have 150 blocks at level 0, what is their marginal for the 33 blocks in level 1? Does this makes any sense? This function is not yet implemented! Please open an issue in the website and I will add it to the next release. As a temporary workaround, you can project all the partitions into the lowest level and get the marginals for that. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: How can I extract the positions of the nodes and edges of the dendrogram obtained from minimize_nested_blockmodel_dl?
Am 20.04.21 um 17:32 schrieb messias.phys...@gmail.com: Hi. I'm trying to create an interactive web plot with the result obtained form minimize_nested_blockmodel_dl. Until now, I'm able to draw the edges and nodes positions in webgl. However, I have a issue. How can I get the node positions and edges from the dendrogram graph? graph-tool result https://ibb.co/hLKrcxs webgl result https://ibb.co/YLnhy83 ```python state = gt.inference.minimize.minimize_nested_blockmodel_dl(g, ... pos, tg, tpos = state.draw(**options) ``` # beizer control points cts = gt.draw.get_hierarchy_control_points(g, tg, tpos) ``` This is given by the "tg" (a graph representing the hierarchy tree) and "tpos" parameters above (the position of the tree nodes). -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: importing/installing graphtool without GTK
Am 21.04.21 um 11:03 schrieb Davide Cittaro: Hello there, I have issues with installing GTK3 on a machine but I would like to use graph-tool without any plot support, I only need to fit very large models. I wonder if it is (or if it will be) possible to import graph_tool in a "light" way, without plotting support. It's perfectly possible. GTK is an optional run-time dependency. It does not have to be present during compilation, and if it's not available at run time, the graph-tool import will still succeed, with only a warning that graph_draw() will not work. -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: LayeredBlockState with edge weights and/or different set of nodes across layers
Am 20.04.21 um 12:56 schrieb arttu.malkam...@helsinki.fi: Dear Community, I've been working on community detection for some time already (Louvain/Leiden algorithms), but I got started with graph-tool only recently. After spending some time scrolling the HOWTO, I still haven't found answers to my two concerns that are related to the applicability of SBM to my data. I'm analysing political coalitions by combining data from a survey, newspapers, and Twitter. To study coalitions across the three weighted networks (or more if I include temporal slices), I am using LayeredBlockState through NestedBlockState. Question 1: I haven't found an answer to whether it is possible to run SBM for layered networks with edge weights? I can get around this problem by extracting the noise-corrected 'backbone' of each network, which yields simple graphs, but ideally I wouldn't have to do too much violence to the data but be able to use the raw weighted networks as inputs. Is it possible to run LayeredBlockState with edge weights? Yes, it's perfectly possible. Just pass the edge covariate parameters (recs, rec_types, rec_params) to LayeredBlockState as you would otherwise to BlockState. Question 2: Even if the LayeredBlockState would not (yet) support weighted networks, I also have another, more fundamental question. As my three layers come from different data generation processes, they do not share the exactly same set of nodes. For example, one organisation responded the survey but doesn't necessarily appear in the newspaper data. Is possible to determine constraints for certain nodes in certain layers that would tell the LayeredBlockState to not consider layer-specific isolates? If you used the "layers=True" version of the model, then the nodes of degree zero in a given layer are considered not to be long to that layer, i.e. they have a probability of zero of receiving edges of that type. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
[graph-tool] Re: get_hierarchy_control_points return values
Am 19.04.21 um 21:42 schrieb messias.phys...@gmail.com: I have the same question as Davide Cittaro. I'm trying to create the same plot got from minimize_nested_blockmodel_dl and draw_hyerarch, but using the threejs (https://threejs.org/docs/?q=curve#api/in/extras/curves/Quadratic Bezier Curve) I'm using this code to extract the control points ``` state = gt.inference.minimize.minimize_nested_blockmodel_dl(g) pos, t, tpos = state.draw(**options) cts = gt.draw.get_hierarchy_control_points(g, t, tpos) ``` However I couldn't get how the control points are enconded in the cts ds. For example, a sinlge instance of cts object I have this print(list(cts)[100] ) [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.03 0.09 0.05 0.18 0.11 0.36 0.17 0.48 0.23 0.61 0.3 0.69 0.36 0.84 0.42 0.99 0.46 1.21 0.5 1.23 0.55 1.25 0.6 1.06 0.65 0.87 0.7 0.69 0.75 0.5 0.81 0.35 0.87 0.21 0.93 0.1 0.97 0.05 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. ] What's the meaning of the first 12 elements? How (x,y) for control points are encoded? From the documentation of graph_draw(): control_points: Control points of a Bézier spline used to draw the edge. Each spline segment requires 6 values corresponding to the (x,y) coordinates of the two intermediary control points and the final point. In the coordinate system, (0,0) is the start of the edge, and (1, 0) is the end of the edge. The last point at (1,0) is always assumed, even if not given. (Note that the get_hierarchy_control_points() is wasteful, since it includes duplicated points in the beginning and end. This is an outcome of the translation between B-splines, used internally, and the Bezier splines that are used for drawing. This is something I intended to fix, but it seems I forgot.) Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list -- graph-tool@skewed.de To unsubscribe send an email to graph-tool-le...@skewed.de
Re: [graph-tool] Question about entropy
Am 16.04.21 um 13:10 schrieb Filippo Valle: Hello, I am running /minimize_nested_blockmodel_dl/ on a certain network and obtaining some partition, good. Then I add some nodes to the network and I do another run of /minimize_nested_blockmodel_dl/and obtain another partition. The question I want to answer is: *does adding those nodes help me finding a better partition?* I am going to compare NestedBlockState.entropy() of the two run, but I am not sure this is correct. How should I take into account the fact that the networks are slightly different? This comparison is not meaningful. From the point of view of model selection, the description length can only be used to compare identical networks. Before you conduct your analysis, you should have a clear criterion of what you consider a "better" partition. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Weird error when interfaced with joblib
Am 01.04.21 um 14:31 schrieb Davide Cittaro: # test with default backend in joblib pstates = [gt.PPBlockState(g) for x in range(n_init)] pstates = Parallel(n_jobs=3)(delayed(fast_min)(state, beta, n_sweep, fast_tol) for state in pstates) selected = pstates[np.argmin([x.entropy() for x in pstates])] print(gt.modularity(g, selected.get_blocks())) My guess here is that joblib is using pickle in the background, which ended up copying the whole states and their graphs, making the property maps incompatible. Try changing the last line to: print(gt.modularity(selected.g, selected.get_blocks())) -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Accessing the internal array of vector property maps
Am 29.03.21 um 15:53 schrieb Rodrigo Leal Cervantes: Say for example that I'm storing the 2D coordinates of the vertices inside the vp `pos`. `g.vp.pos.a` returns `None`, and the way I typically convert this to an array is ``` print(g.vp.pos) # ', for Graph 0x7f02f04d34f0, at 0x7f0297d164f0> pos = np.array(list(g.vp.pos)) print(pos.shape) # (100, 2) ``` but I wonder if there is a better way to do this. What do other people use in this case? Yes, take a look at: https://graph-tool.skewed.de/static/doc/graph_tool.html#graph_tool.PropertyMap.get_2d_array This means you can do: a = g.vp.pos.get_2d_array([0, 1]) The reason why this is needed is because vector properties are *not* internally stored as contiguous 2D arrays. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] FYI: Macport building issue on Big Sur
Hi, This does not belong here; please bring this up with macports. Note also that you are referring to very outdated version of graph-tool (2.33 vs the current which is 2.37). Best, Tiago Am 29.03.21 um 16:51 schrieb Carlos E. Andrade: Dear folks, It may not be a direct issue in graph-tool, but just for information. It does not build on Big Sur either py38, boost +python38, not py39, boost +python39. https://trac.macports.org/ticket/61583 Thanks, Carlos -- Sent from: https://nabble.skewed.de/ ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Which state should I copy when doing merge-split on model with weight transformation
Am 21.03.21 um 15:33 schrieb jstonge: There is a big difference between the description length of the two models. My understanding is that the *state* in the first example comes from the previous `real-exponential` model, which means we copy its state then pass the hierarchy levels of the state_ln model. Is it supposed to be so? Shouldn't we always copy the state of the models we ran in the first place to run merge-split algorithm? Yes, this is an error in the howto! Thanks for noticing. It will be fixed in the next version. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Getting shortest distances for a subset of origins - are cython-like speedups worth looking into?
Am 04.03.21 um 16:53 schrieb ignacioRh: Hey, Thanks a lot for the program. I'm using it on an academic project which would have died long ago without it. I'm now running some large simulations and need to speed up how I use the shortest_distance, which is in a python for loop My use case is the following: I have a large graph, and I need to compute the shortest distance between an origin subset of the nodes [A,B], to a target subset [X,Y]. So I want the distances AX, AY, BX, BY. It is infeasible to compute all pairs distances on the graph, its too large I can pass a list to shortest_distance as a target, but I can't pass a list as an origin. Hence I use list comprehension to achieve the result distances = [shortest_distances(origin, target= [X,Y] for origin in [A,B]] This is the main bottleneck in my code. I don't understand why this would be. If your network is very large, surely most of the time is spent in the inner loop, rather than the outer one. I'm wondering if I can somehow put the for loop for the origins in some type of cpython extension, would that result in a speed up gain? So if put the underlying libgraph_tool_topology.get_dists() call into some cython-type code, do you think that would speed up this for loop? If the bottleneck is really there, this could help, but not by much. But again, I think it's unlikely, unless your graph is quite small. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Using graph-tool in Colab after Colab has upgraded to Python 3.7
Dear Edgar, Unfortunately this is bug in colab, and there is nothing much that can be done. Apparently they just switched from 3.6 to 3.7, even though the underlying OS (Ubuntu bionic) still has 3.6 as default. This means that every compiled Python module (cairo, GTK, etc) can only imported from 3.6, not 3.7. Best, Tiago Am 04.03.21 um 01:28 schrieb edgarbelmonte: Hi All, Wonderful to get to know about this mailing list. Thanks to the creators of graph-tool for writing a wonderful library!! I was seamlessly using graph-tool inside Colab notebooks until a week ago to visualize large graphs. But about 10 days ago Colab updated to Python 3.7 and my original graph-tool import and usage setup ran into a few hiccups due to me not knowing how to go about dealing with changes due to their update. After installing graph-tool within Colab in the usual way my main script goes as follows... = from graph_tool.all import * line_no (1) for reference from gi.repository import Gtk, Gdk, GdkPixbuf, GObject, GLib line_no (2) pos= sfdp_layout(CG) line_no (30) for reference graph_draw(.. ) line_no (31) for reference = The accompanying G-libs seem necessary along with graph-tool, since without line_no (2) I can't use sfdp_layout later in line_no 30. Error I get is "name 'sfdp_layout' is not defined" But if I do try import those G-libs as in line_no(2), I get an import error. "ImportError: cannot import name '_gi' from 'gi' (/usr/lib/python3/dist-packages/gi/__init__.py)" There doesn't seem to be a way to make Colab use earlier version Python 3.6 by default for a notebook. So I am a bit clueless on how to go about it. Has anyone run into similar problem? Can anyone check this out and share how one can get around this problem ? Thank you so much, Edgar -- Sent from: https://nabble.skewed.de/ ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Temporal community detection with DSBM
Am 01.03.21 um 17:19 schrieb ulgenklc: Ahh, yes I noticed that when I scanned through the previous posts, sorry for replication. I have three more quick questions: 1) If I fitted a different SBM to splitted the graph, wouldn't the communities in individual layers be temporally discrete? That's not quite what I want because I want to track an evolution, but I'm open to process that result in a temporally overlapping fashion(some sort of set matching algorithms between layers.) I'm not sure exactly what you mean, but if you want to "align" the different partitions you can take a look at https://graph-tool.skewed.de/static/doc/inference.html#graph_tool.inference.partition_modes.PartitionModeState There is an example of this alignment being done here: https://graph-tool.skewed.de/static/doc/demos/inference/inference.html#sampling-from-the-posterior-distribution 2) When I pass overlap =True to LayeredBlockState, I have NxT many nodes as a result like you said, that's all fine. However, now I can't get the node membership using .get_blocks(). levels = states[0].get_levels() levels[0].get_blocks() is a list of length M(total number of edges in the network) which should be a list of length NxT, isn't it? Again, it feels like I'm missing something very trivial here... The overlapping block model consists of a labeling of the half-edges of the graph. Since each half-edge can belong to only one layer, you can then tell how the membership of the respective node has changed in the time slice. Please take a look at the documentation on how to extract this information, e.g. via https://graph-tool.skewed.de/static/doc/inference.html#graph_tool.inference.overlap_blockmodel.OverlapBlockState.get_overlap_blocks 3) Also, is there a method for levels[0] that I can call to get the total number of communities up front? I can see the number of blocks when I do states[0].print_summary() but I need the integer value of this number for preprocessing.. It's worthwhile to peruse the documentation, which contains all the answers to questions such as this: https://graph-tool.skewed.de/static/doc/inference.html#graph_tool.inference.blockmodel.BlockState.get_nonempty_B -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] Temporal community detection with DSBM
Am 26.02.21 um 20:00 schrieb ulgenklc: I have a serious problem with getting the node labels. The code below only returns the node membership information for N nodes(size of the aggregated network). But since this is an evolving network, nodes are expected to change communities over time, so below code should return NxT(number of nodes times number of layers) many community labels? levels = state.get_levels() for s in levels: print(s) This returns the network partition at different levels of the nested SBM but only for N many nodes. What am I missing? You are missing the fact that the layered model only has a single partition shared between all the layers. If you want a model that gives a different partition for every layer you can either: 1. Split each layer into a separate graph, and fit a different SBM for each layer. 2. Fit an overlapping SBM (i.e. by passing overlap=True to LayeredBLockState), which will cluster the "half-edges", and hence give partitions that (potentially) vary between the layers. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] minimize_blockmodel_dl taking a long time to run
Am 27.02.21 um 08:21 schrieb sam: hi, i'm running minimize_blockmodel_dl on a somewhat large network (230K vertices, ~1M edges) and it's taking a very long time. hours so far. as i understand it should run in O(N*ln^2N), and it shouldn't depend on properties of the network in the past it's never taken this long on my machine for networks of similar size anything i might be able to change to speed things up? Unfortunately, it's not possible to say much without a concrete example. You might want to try the alternative to minimize_blockmodel_dl() explained here: https://graph-tool.skewed.de/static/doc/demos/inference/inference.html#trade-off-between-memory-usage-and-computation-time -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
Re: [graph-tool] dfs/bfs seem to have overhead depending on total graph size
Am 26.02.21 um 12:08 schrieb My-Tien Nguyen: I noticed that when I call e.g. dfs_iterator or dfs_search on a source vertex, it takes around 1 – 2 seconds to return. The upper bound is depending on the component’s size, but the lower bound seems to be the same for all components. DFS creates and initializes a property map keeping track of the vertices that have been visited, which is an O(N) operation. It's possible to change the code such that the property map can be passed to the algorithm, and then shared between multiple calls, removing this overhead. Please open an issue in the website, and I'll add this for the next release. Best, Tiago -- Tiago de Paula Peixoto ___ graph-tool mailing list graph-tool@skewed.de https://lists.skewed.de/mailman/listinfo/graph-tool
