On Fri, Aug 14, 2015 at 4:58 PM, John Peterson <jwpeter...@gmail.com> wrote:
>
>
> On Fri, Aug 14, 2015 at 2:28 PM, Junchao Zhang <junchao.zh...@gmail.com>
> wrote:
>
>> Hello,
>> I have a simple AMR code using SerialMesh. I run it 70 refinements on 12
>> cores.
>
>
> 2^70 = 1180591620717411303424, do you mean 7?
>
It is adaptive and some elements are not refined after a few iterations.
After 70 iterations, there are about 9.9M elements in total.
>
>
>> In metis_partitioner.C, I print out the partition result as follows.
>>
>> 437 // Otherwise use kway
>> 438 else
>> 439 Metis::METIS_PartGraphKway(&n, &ncon, &csr_graph.offsets[0],
>> &csr_graph.vals[0], &vwgt[0], NULL,
>> 440 NULL, &nparts, NULL, NULL, NULL,
>> 441 &edgecut, &part[0]);
>> 442
>> 443 for (int i = 0; i < mesh.n_processors(); i++) { // my code
>> 444 int cnt = 0;
>> 445 for (int j = 0; j < n; j++) if (part[j] == i) cnt++;
>> 446 printf("METIS assigned %d elements to proc %d\n", cnt, i);
>> 447 }
>>
>> I found in the last few iterations, the partition result is severely
>> imbalanced. For example, in last iteration, proc 11 has 9M+ elements,
>> while
>> others only have hundreds or thousands.
>>
>> METIS assigned 2829 elements to proc 0
>> METIS assigned 270 elements to proc 1
>> METIS assigned 804 elements to proc 2
>> METIS assigned 264 elements to proc 3
>> METIS assigned 263 elements to proc 4
>> METIS assigned 733 elements to proc 5
>> METIS assigned 273 elements to proc 6
>> METIS assigned 407 elements to proc 7
>> METIS assigned 1200 elements to proc 8
>> METIS assigned 4795 elements to proc 9
>> METIS assigned 23404 elements to proc 10
>> METIS assigned 9866034 elements to proc 11
>>
>>
>> My mesh has only triangles, with at most one hanging node.
>
>
> Your *initial* mesh has hanging nodes? We don't support this...
>
My input mesh has four triangles and does not have hanging nodes. After
refinement, hanging nodes appear.
>
>
>> I did not assign
>> any weight to nodes and I found libMesh uses number of nodes as weight (in
>> my case, it should be 3). I tried to changed &vwgt[0] to NULL when calling
>> METIS. From the printf, I got a quite balanced partitioning. However, the
>> program execution time increased a lot.
>>
>> Any clues?
>>
>
> I don't know what METIS does when you don't use any weights. It sounds
> like it employs a more expensive algorithm, but results in a higher-quality
> partitioning. Can you share the stats for this partitioning?
>
Sure. I will do it. Thanks.
>
> For what it's worth, we've noticed that METIS produces very strange
> partitions on even simple geometries, provided that you pick just the right
> (wrong) number of processors, so it's not surprising that it can also have
> trouble when partitioning millions of elements.
>
>
> https://drive.google.com/file/d/0B9BK7pg8se_iTXpJak1fZWwySm8/view?usp=sharing
>
> https://drive.google.com/file/d/0B9BK7pg8se_ibUJCdTBzR3h1UTQ/view?usp=sharing
>
> https://drive.google.com/file/d/0B9BK7pg8se_ia1F5ZE0zTF9pdFE/view?usp=sharing
>
> https://drive.google.com/file/d/0B9BK7pg8se_iY2hxQkl6ZHlFTmM/view?usp=sharing
>
> https://drive.google.com/file/d/0B9BK7pg8se_iNFI0bDNlUXJwVlE/view?usp=sharing
>
> --
> John
>
------------------------------------------------------------------------------
_______________________________________________
Libmesh-users mailing list
Libmesh-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-users
