Re: [gem5-users] Garnet 2.0 + How the dir_nodes are connected in Mesh_XY network

[Tushar Krishna]

Let me re-phrase your question:
if a topology has more than one link in the same “direction” (say flattened 
butterfly where there might be multiple links going “east”), then your question 
is whether they need to be given unique port names or not.

By default, the port names are not used by Garnet for any routing. Each port 
adds a unique id and routing tables are populated based on distance and 
weights. So it is fine to have two “East” links from the same source router to 
the same or different destination routers.
We added the notion of port names in case someone wanted to hack garnet and use 
their own custom routing algorithm.
So you can use “east1” and “east2” if you want and then use the custom routing 
option inside garnet to route on one or the other based on your scheme.

Cheers,
Tushar



> On Sep 7, 2017, at 3:16 AM, F. A. Faisal  wrote:
> 
> Dear Professor,
> 
> Many many thanks for your earliest reply.
> 
> Howerver, I like to ask more detailed about the 2 point.
> 
> 
>> 2. Now, as mentioned in the figure of Mesh_XY at homepage of garnet 2.0, two 
>> links are connected between two routers (one incomming and outgoing). 
>> However, if we see the below code for Mesh_XY, we can see both links are 
>> connected to the same port number or string. Hence, it means as the 
>> bandwidth factor is fixed, either one of the router can send a packet at a 
>> time, not both.
> 
> > I would like to make this question more clear. 
> Suppose, we have 4 routers in each direction (X-Y: (0-3) in X-direction) and 
> each router in each horizontal or vertical direction is connected to each 
> other. 
> Hence, router 0 requires 3 links for its X-directional routers and another 3 
> links for Y-directional routers. Now, my question is do we need to make 
> different string name
> for each port, which is connected to each directional routers or same port 
> name will not have any effect(as below same outport string for Router_0). 
> 
> In case of MESH/TORUS we have 2 links in each direction. So, it is enough as 
> North or south. However, in the upper situation of multiple same directional 
> links
> will i need to make different port name for each router. 
> 
> ROTUER_0 --> (outport) 0 -> 1 (inport) Router_1  
> ROTUER_0 --> (outport) 0 -> 2 (inport) Router_1 
> ROTUER_0 --> (outport) 0 -> 3 (inport) Router_1 
> 
> Thanks again.
> 
> Best Regards,
> 
> F. A. Faisal
> 
> 
> On Thu, Aug 31, 2017 at 3:44 PM, Tushar Krishna  > wrote:
> Hi Faisal,
> 
>> On Aug 31, 2017, at 2:17 AM, F. A. Faisal > > wrote:
>> 
>> Dear All,
>> 
>> 1. I cross-checked the code for Mesh_XY.py and MeshDirCorners_XY.py file.
>> Where, in the MeshDirCorners_XY.py has the proper implementation of the 
>> connected dir_nodes. 
>> However, I couldn't find any code that connectes the dir_nodes with each 
>> router inside the Mesh_XY.py file.
> 
> In Mesh_XY.py, *all* controllers (L1s, then L2s, then directories) are 
> connected one by one to each router.
> This topology inherently assumes same number of directories as CPUs, so that 
> each core is connected to both a CPU and a directory.
> 
> Dir Corners on the other hand has 4 directory controllers, explicitly 
> connected to the corner routers.
> 
>> 
>> 2. Now, as mentioned in the figure of Mesh_XY at homepage of garnet 2.0, two 
>> links are connected between two routers (one incomming and outgoing). 
>> However, if we see the below code for Mesh_XY, we can see both links are 
>> connected to the same port number or string. Hence, it means as the 
>> bandwidth factor is fixed, either one of the router can send a packet at a 
>> time, not both.
> 
> I didn’t understand your point.
> What do you mean by same port number?
> Two links are created - they both have different “src_node” and “dst_node” 
> routers.
> For example, between routers 5 and 6, two links are created, one from 5 (src) 
> to 6 (dst), and one from 6 (src) to 5 (dst).
> Internally each will instantiate an output port at the source router and 
> input port at the destination router, and can both operate in parallel.
> 
>> 
>> # East output to West input links (weight = 1)
>> for row in xrange(num_rows):
>> for col in xrange(num_columns):
>> if (col + 1 < num_columns):
>> east_out = col + (row * num_columns)
>> west_in = (col + 1) + (row * num_columns)
>> int_links.append(IntLink(link_id=link_count,
>>  src_node=routers[east_out],
>>  dst_node=routers[west_in],
>>  src_outport="East",
>>  dst_inport="West",
>>  latency = link_latency,
>>  weight=1))

Re: [gem5-users] Garnet 2.0 + How the dir_nodes are connected in Mesh_XY network

[F. A. Faisal]

Dear Professor,

Many many thanks for your earliest reply.

Howerver, I like to ask more detailed about the 2 point.


2. Now, as mentioned in the figure of Mesh_XY at homepage of garnet 2.0,
two links are connected between two routers (one incomming and outgoing).
However, if we see the below code for Mesh_XY, we can see both links are
connected to the same port number or string. Hence, it means as the
bandwidth factor is fixed, either one of the router can send a packet at a
time, not both.


> I would like to make this question more clear.
Suppose, we have 4 routers in each direction (X-Y: (0-3) in X-direction)
and each router in each horizontal or vertical direction is connected to
each other.
Hence, router 0 requires 3 links for its X-directional routers and another
3 links for Y-directional routers. Now, my question is do we need to make
different string name
for each port, which is connected to each directional routers or same port
name will not have any effect(as below same outport string for Router_0).

In case of MESH/TORUS we have 2 links in each direction. So, it is enough
as North or south. However, in the upper situation of multiple same
directional links
will i need to make different port name for each router.

ROTUER_0 --> (outport) 0 -> 1 (inport) Router_1
ROTUER_0 --> (outport) 0 -> 2 (inport) Router_1
ROTUER_0 --> (outport) 0 -> 3 (inport) Router_1

Thanks again.

Best Regards,

F. A. Faisal


On Thu, Aug 31, 2017 at 3:44 PM, Tushar Krishna 
wrote:

> Hi Faisal,
>
> On Aug 31, 2017, at 2:17 AM, F. A. Faisal  wrote:
>
> Dear All,
>
> 1. I cross-checked the code for Mesh_XY.py and MeshDirCorners_XY.py file.
> Where, in the MeshDirCorners_XY.py has the proper implementation of the
> connected dir_nodes.
> However, I couldn't find any code that connectes the dir_nodes with each
> router inside the Mesh_XY.py file.
>
>
> In Mesh_XY.py, *all* controllers (L1s, then L2s, then directories) are
> connected one by one to each router.
> This topology inherently assumes same number of directories as CPUs, so
> that each core is connected to both a CPU and a directory.
>
> Dir Corners on the other hand has 4 directory controllers, explicitly
> connected to the corner routers.
>
>
> 2. Now, as mentioned in the figure of Mesh_XY at homepage of garnet 2.0,
> two links are connected between two routers (one incomming and outgoing).
> However, if we see the below code for Mesh_XY, we can see both links are
> connected to the same port number or string. Hence, it means as the
> bandwidth factor is fixed, either one of the router can send a packet at a
> time, not both.
>
>
> I didn’t understand your point.
> What do you mean by same port number?
> Two links are created - they both have different “src_node” and “dst_node”
> routers.
> For example, between routers 5 and 6, two links are created, one from 5
> (src) to 6 (dst), and one from 6 (src) to 5 (dst).
> Internally each will instantiate an output port at the source router and
> input port at the destination router, and can both operate in parallel.
>
>
> # East output to West input links (weight = 1)
> for row in xrange(num_rows):
> for col in xrange(num_columns):
> if (col + 1 < num_columns):
> east_out = col + (row * num_columns)
> west_in = (col + 1) + (row * num_columns)
> int_links.append(IntLink(link_id=link_count,
>  src_node=routers[east_out],
>  dst_node=routers[west_in],
>  src_outport="East",
>  dst_inport="West",
>  latency = link_latency,
>  weight=1))
> link_count += 1
>
> # West output to East input links (weight = 1)
> for row in xrange(num_rows):
> for col in xrange(num_columns):
> if (col + 1 < num_columns):
> east_in = col + (row * num_columns)
> west_out = (col + 1) + (row * num_columns)
> int_links.append(IntLink(link_id=link_count,
>  src_node=routers[west_out],
>  dst_node=routers[east_in],
>  src_outport="West",
>  dst_inport="East",
>  latency = link_latency,
>  weight=1))
> link_count += 1
>
>
>
>
> 3. I also like to know how much minimum clock cycle is required to
> transmit a complete packet between the two interconnected routers.
>
> For each flit, it takes link_latency cycles to transmit.
> Default value of link_latency is 

Re: [gem5-users] Garnet 2.0 + How the dir_nodes are connected in Mesh_XY network

[Tushar Krishna]

Hi Faisal,

> On Aug 31, 2017, at 2:17 AM, F. A. Faisal  wrote:
> 
> Dear All,
> 
> 1. I cross-checked the code for Mesh_XY.py and MeshDirCorners_XY.py file.
> Where, in the MeshDirCorners_XY.py has the proper implementation of the 
> connected dir_nodes. 
> However, I couldn't find any code that connectes the dir_nodes with each 
> router inside the Mesh_XY.py file.

In Mesh_XY.py, *all* controllers (L1s, then L2s, then directories) are 
connected one by one to each router.
This topology inherently assumes same number of directories as CPUs, so that 
each core is connected to both a CPU and a directory.

Dir Corners on the other hand has 4 directory controllers, explicitly connected 
to the corner routers.

> 
> 2. Now, as mentioned in the figure of Mesh_XY at homepage of garnet 2.0, two 
> links are connected between two routers (one incomming and outgoing). 
> However, if we see the below code for Mesh_XY, we can see both links are 
> connected to the same port number or string. Hence, it means as the bandwidth 
> factor is fixed, either one of the router can send a packet at a time, not 
> both.

I didn’t understand your point.
What do you mean by same port number?
Two links are created - they both have different “src_node” and “dst_node” 
routers.
For example, between routers 5 and 6, two links are created, one from 5 (src) 
to 6 (dst), and one from 6 (src) to 5 (dst).
Internally each will instantiate an output port at the source router and input 
port at the destination router, and can both operate in parallel.

> 
> # East output to West input links (weight = 1)
> for row in xrange(num_rows):
> for col in xrange(num_columns):
> if (col + 1 < num_columns):
> east_out = col + (row * num_columns)
> west_in = (col + 1) + (row * num_columns)
> int_links.append(IntLink(link_id=link_count,
>  src_node=routers[east_out],
>  dst_node=routers[west_in],
>  src_outport="East",
>  dst_inport="West",
>  latency = link_latency,
>  weight=1))
> link_count += 1
> 
> # West output to East input links (weight = 1)
> for row in xrange(num_rows):
> for col in xrange(num_columns):
> if (col + 1 < num_columns):
> east_in = col + (row * num_columns)
> west_out = (col + 1) + (row * num_columns)
> int_links.append(IntLink(link_id=link_count,
>  src_node=routers[west_out],
>  dst_node=routers[east_in],
>  src_outport="West",
>  dst_inport="East",
>  latency = link_latency,
>  weight=1))
> link_count += 1
> 
> 
> 
> 
> 3. I also like to know how much minimum clock cycle is required to transmit a 
> complete packet between the two interconnected routers. 
> 
For each flit, it takes link_latency cycles to transmit.
Default value of link_latency is one, but can be overwritten for each link 
separately in the topology file.

For a N-flit packet, it will thus take N cycles.


> 4. And is it possible to make the packet level tracing in Garnet 2.0.
> 

Not yet.
I plan to upload a patch for that next week - keep a look out on the Garnet2.0 
wiki page.



> Please let me know the details to increase the understanding.
> 
> Thanks a lot in advance for your kind help.
> 
> Best regards,
> 
> F. A. Faisal
> 

- Tushar

> 
> ___
> gem5-users mailing list
> gem5-users@gem5.org
> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users

___
gem5-users mailing list
gem5-users@gem5.org
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users