> -----Original Message-----
> From: Jeff Squyres [mailto:jsquy...@cisco.com]
> Sent: ב 25 פברואר 2008 16:52
> To: de...@open-mpi.org
> Cc: Lenny Verkhovsky
> Subject: Re: [OMPI svn-full] svn:open-mpi r17584
>
> Lenny --
>
> Is this the patch that Sharon was working on? I literally just
> created a new branch for bringing in plpa v1.1. Should I do it on
> your rank_file branch instead?
>
No need to.
Yes, it's Sharon's patch,
I made a new branch for Rank Mapping RMAPS component that is a copy of Ralph's
rhc-step2b branch with a new PLPA1.1
I also had to patch the files due to new API of PLPA with Sharon's patch.
Are you planning to merge new PLPA1.1 with the trunk ?
>
> On Feb 25, 2008, at 9:46 AM, lenn...@osl.iu.edu wrote:
>
> > Author: lennyve
> > Date: 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > New Revision: 17584
> > URL: https://svn.open-mpi.org/trac/ompi/changeset/17584
> >
> > Log:
> > Added patched files due to PLPA.1.1 API
> >
> > Added:
> > tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
> > tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
> > tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
> > tmp-public/rank_file/opal/mca/paffinity/linux/paffinity_linux.h
> > tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_component.c
> > tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_module.c
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/Makefile.am
> > 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,53 @@
> > +#
> > +# Copyright (c) 2004-2005 The Trustees of Indiana University and
> > Indiana
> > +# University Research and Technology
> > +# Corporation. All rights reserved.
> > +# Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > +# of Tennessee Research Foundation. All
> > rights
> > +# reserved.
> > +# Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > +# University of Stuttgart. All rights
> > reserved.
> > +# Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > +# All rights reserved.
> > +# Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
> > +# $COPYRIGHT$
> > +#
> > +# Additional copyrights may follow
> > +#
> > +# $HEADER$
> > +#
> > +
> > +SUBDIRS = plpa
> > +
> > +# To find plpa_bottom.h
> > +AM_CPPFLAGS = -I$(top_srcdir)/opal/mca/paffinity/linux/plpa/src/
> > libplpa
> > +
> > +sources = \
> > + paffinity_linux.h \
> > + paffinity_linux_component.c \
> > + paffinity_linux_module.c
> > +
> > +# Make the output library in this directory, and name it either
> > +# mca_<type>_<name>.la (for DSO builds) or libmca_<type>_<name>.la
> > +# (for static builds).
> > +
> > +if OMPI_BUILD_paffinity_linux_DSO
> > +component_noinst =
> > +component_install = mca_paffinity_linux.la
> > +else
> > +component_noinst = libmca_paffinity_linux.la
> > +component_install =
> > +endif
> > +
> > +mcacomponentdir = $(pkglibdir)
> > +mcacomponent_LTLIBRARIES = $(component_install)
> > +mca_paffinity_linux_la_SOURCES = $(sources)
> > +mca_paffinity_linux_la_LDFLAGS = -module -avoid-version
> > +mca_paffinity_linux_la_LIBADD = \
> > + $(top_ompi_builddir)/opal/mca/paffinity/linux/plpa/src/
> > libplpa/libplpa_included.la
> > +
> > +noinst_LTLIBRARIES = $(component_noinst)
> > +libmca_paffinity_linux_la_SOURCES =$(sources)
> > +libmca_paffinity_linux_la_LDFLAGS = -module -avoid-version
> > +libmca_paffinity_linux_la_LIBADD = \
> > + $(top_ompi_builddir)/opal/mca/paffinity/linux/plpa/src/
> > libplpa/libplpa_included.la
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/configure.m4
> > 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,34 @@
> > +# -*- shell-script -*-
> > +#
> > +# Copyright (c) 2004-2005 The Trustees of Indiana University and
> > Indiana
> > +# University Research and Technology
> > +# Corporation. All rights reserved.
> > +# Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > +# of Tennessee Research Foundation. All
> > rights
> > +# reserved.
> > +# Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > +# University of Stuttgart. All rights
> > reserved.
> > +# Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > +# All rights reserved.
> > +# Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
> > +#
> > +# $COPYRIGHT$
> > +#
> > +# Additional copyrights may follow
> > +#
> > +# $HEADER$
> > +#
> > +
> > +# MCA_paffinity_linux_CONFIG([action-if-found], [action-if-not-
> > found])
> > +# -----------------------------------------------------------
> > +m4_include(opal/mca/paffinity/linux/plpa/config/plpa.m4)
> > +
> > +AC_DEFUN([MCA_paffinity_linux_POST_CONFIG],[
> > + PLPA_DO_AM_CONDITIONALS
> > +])dnl
> > +
> > +AC_DEFUN([MCA_paffinity_linux_CONFIG],[
> > + PLPA_SET_SYMBOL_PREFIX([opal_paffinity_linux_plpa_])
> > + PLPA_INCLUDED([opal/mca/paffinity/linux/plpa])
> > + PLPA_INIT([$1],[$2])
> > +])dnl
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/configure.params
> > 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,22 @@
> > +# -*- shell-script -*-
> > +#
> > +# Copyright (c) 2004-2005 The Trustees of Indiana University and
> > Indiana
> > +# University Research and Technology
> > +# Corporation. All rights reserved.
> > +# Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > +# of Tennessee Research Foundation. All
> > rights
> > +# reserved.
> > +# Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > +# University of Stuttgart. All rights
> > reserved.
> > +# Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > +# All rights reserved.
> > +# Copyright (c) 2007 Los Alamos National Security, LLC. All
> > rights
> > +# reserved.
> > +# $COPYRIGHT$
> > +#
> > +# Additional copyrights may follow
> > +#
> > +# $HEADER$
> > +#
> > +
> > +PARAM_CONFIG_FILES="Makefile"
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/paffinity_linux.h
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/paffinity_linux.h
> > 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,69 @@
> > +/*
> > + * Copyright (c) 2004-2005 The Trustees of Indiana University and
> > Indiana
> > + * University Research and Technology
> > + * Corporation. All rights reserved.
> > + * Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > + * of Tennessee Research Foundation. All
> > rights
> > + * reserved.
> > + * Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > + * University of Stuttgart. All rights
> > reserved.
> > + * Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > + * All rights reserved.
> > + * Copyright (c) 2006-2007 Cisco Systems, Inc. All rights reserved.
> > + *
> > + * $COPYRIGHT$
> > + *
> > + * Additional copyrights may follow
> > + *
> > + * $HEADER$
> > + */
> > +
> > +/**
> > + * @file
> > + *
> > + * Processor affinity for Linux.
> > + *
> > + * Linux sucks. There are at least 3 different ways that
> > + * sched_setaffinity is implemented.
> > + *
> > + * Fortunately we have an independent project called Portable Linux
> > + * Processor Affinity (PLPA) which allows us to do processor affinity
> > + * without knowing which flavor of afffinity is installed on the
> > + * system a priori - PLPA does a few probes behind the scenes and
> > + * utilizes the correct syntax to the correct system call to set
> > + * or get processor affinity for us.
> > + *
> > + */
> > +
> > +
> > +#ifndef MCA_PAFFINITY_LINUX_EXPORT_H
> > +#define MCA_PAFFINITY_LINUX_EXPORT_H
> > +
> > +#include "opal_config.h"
> > +
> > +#include "opal/mca/mca.h"
> > +#include "opal/mca/paffinity/paffinity.h"
> > +#include "opal/mca/paffinity/linux/plpa/src/libplpa/plpa.h"
> > +
> > +
> > +#if defined(c_plusplus) || defined(__cplusplus)
> > +extern "C" {
> > +#endif
> > +
> > + /**
> > + * Globally exported variable
> > + */
> > + OPAL_DECLSPEC extern const opal_paffinity_base_component_1_1_0_t
> > + mca_paffinity_linux_component;
> > +
> > +
> > + /**
> > + * paffinity query API function
> > + */
> > + const opal_paffinity_base_module_1_1_0_t *
> > + opal_paffinity_linux_component_query(int *query);
> > +
> > +#if defined(c_plusplus) || defined(__cplusplus)
> > +}
> > +#endif
> > +#endif /* MCA_PAFFINITY_LINUX_EXPORT_H */
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_component.c
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_component.c 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,91 @@
> > +/*
> > + * Copyright (c) 2004-2007 The Trustees of Indiana University and
> > Indiana
> > + * University Research and Technology
> > + * Corporation. All rights reserved.
> > + * Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > + * of Tennessee Research Foundation. All
> > rights
> > + * reserved.
> > + * Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > + * University of Stuttgart. All rights
> > reserved.
> > + * Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > + * All rights reserved.
> > + * Copyright (c) 2007 Cisco, Inc. All rights reserved.
> > + * $COPYRIGHT$
> > + *
> > + * Additional copyrights may follow
> > + *
> > + * $HEADER$
> > + *
> > + * These symbols are in a file by themselves to provide nice linker
> > + * semantics. Since linkers generally pull in symbols by object
> > + * files, keeping these symbols as the only symbols in this file
> > + * prevents utility programs such as "ompi_info" from having to
> > import
> > + * entire components just to query their version and parameters.
> > + */
> > +
> > +#include "opal_config.h"
> > +
> > +#include "opal/constants.h"
> > +#include "opal/mca/paffinity/paffinity.h"
> > +#include "paffinity_linux.h"
> > +
> > +/*
> > + * Public string showing the paffinity ompi_linux component version
> > number
> > + */
> > +const char *opal_paffinity_linux_component_version_string =
> > + "OPAL linux paffinity MCA component version " OPAL_VERSION;
> > +
> > +/*
> > + * Local function
> > + */
> > +static int linux_open(void);
> > +
> > +/*
> > + * Instantiate the public struct with all of our public information
> > + * and pointers to our public functions in it
> > + */
> > +
> > +const opal_paffinity_base_component_1_1_0_t
> > mca_paffinity_linux_component = {
> > +
> > + /* First, the mca_component_t struct containing meta information
> > + about the component itself */
> > +
> > + {
> > + /* Indicate that we are a paffinity v1.1.0 component (which
> > also
> > + implies a specific MCA version) */
> > +
> > + OPAL_PAFFINITY_BASE_VERSION_1_1_0,
> > +
> > + /* Component name and version */
> > +
> > + "linux",
> > + OPAL_MAJOR_VERSION,
> > + OPAL_MINOR_VERSION,
> > + OPAL_RELEASE_VERSION,
> > +
> > + /* Component open and close functions */
> > +
> > + linux_open,
> > + NULL
> > + },
> > + {
> > +
> > + /* The component is checkpoint ready */
> > + MCA_BASE_METADATA_PARAM_CHECKPOINT
> > + },
> > +
> > + /* Query function */
> > +
> > + opal_paffinity_linux_component_query
> > +};
> > +
> > +
> > +static int linux_open(void)
> > +{
> > +
> > mca_base_param_reg_int
> > (&mca_paffinity_linux_component.paffinityc_version,
> > + "priority",
> > + "Priority of the linux paffinity
> > component",
> > + false, false, 10, NULL);
> > +
> > + return OPAL_SUCCESS;
> > +}
> >
> > Added: tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_module.c
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > =
> > ======================================================================
> > --- (empty file)
> > +++ tmp-public/rank_file/opal/mca/paffinity/linux/
> > paffinity_linux_module.c 2008-02-25 09:46:28 EST (Mon, 25 Feb 2008)
> > @@ -0,0 +1,185 @@
> > +/*
> > + * Copyright (c) 2004-2005 The Trustees of Indiana University and
> > Indiana
> > + * University Research and Technology
> > + * Corporation. All rights reserved.
> > + * Copyright (c) 2004-2005 The University of Tennessee and The
> > University
> > + * of Tennessee Research Foundation. All
> > rights
> > + * reserved.
> > + * Copyright (c) 2004-2005 High Performance Computing Center
> > Stuttgart,
> > + * University of Stuttgart. All rights
> > reserved.
> > + * Copyright (c) 2004-2005 The Regents of the University of
> > California.
> > + * All rights reserved.
> > + * Copyright (c) 2006-2007 Cisco Systems, Inc. All rights reserved.
> > + *
> > + * $COPYRIGHT$
> > + *
> > + * Additional copyrights may follow
> > + *
> > + * $HEADER$
> > + */
> > +
> > +#include "opal_config.h"
> > +
> > +/* This component will only be compiled on Linux, where we are
> > + guaranteed to have <unistd.h> and friends */
> > +#include <stdio.h>
> > +
> > +#include <unistd.h>
> > +#include <stdlib.h>
> > +#include <string.h>
> > +#include <errno.h>
> > +
> > +#include "opal/constants.h"
> > +#include "opal/mca/base/mca_base_param.h"
> > +#include "opal/mca/paffinity/paffinity.h"
> > +#include "opal/mca/paffinity/base/base.h"
> > +#include "paffinity_linux.h"
> > +#include "plpa/src/libplpa/plpa.h"
> > +
> > +
> > +/*
> > + * Local functions
> > + */
> > +static int linux_module_init(void);
> > +static int linux_module_set(opal_paffinity_base_cpu_set_t cpumask);
> > +static int linux_module_get(opal_paffinity_base_cpu_set_t *cpumask);
> > +static int linux_module_map_to_processor_id(int socket, int core,
> > int *processor_id);
> > +static int linux_module_map_to_socket_core(int processor_id, int
> > *socket, int *core);
> > +static int linux_module_get_processor_info(int *num_processors, int
> > *max_processor_id);
> > +static int linux_module_get_socket_info(int *num_sockets, int
> > *max_socket_num);
> > +static int linux_module_get_core_info(int socket, int *num_cores,
> > int *max_core_num);
> > +
> > +/*
> > + * Linux paffinity module
> > + */
> > +static const opal_paffinity_base_module_1_1_0_t module = {
> > +
> > + /* Initialization function */
> > +
> > + linux_module_init,
> > +
> > + /* Module function pointers */
> > +
> > + linux_module_set,
> > + linux_module_get,
> > + linux_module_map_to_processor_id,
> > + linux_module_map_to_socket_core,
> > + linux_module_get_processor_info,
> > + linux_module_get_socket_info,
> > + linux_module_get_core_info,
> > + NULL
> > +};
> > +
> > +
> > +const opal_paffinity_base_module_1_1_0_t *
> > +opal_paffinity_linux_component_query(int *query)
> > +{
> > + int param;
> > +
> > + param = mca_base_param_find("paffinity", "linux", "priority");
> > + mca_base_param_lookup_int(param, query);
> > +
> > + return &module;
> > +}
> > +
> > +
> > +static int linux_module_init(void)
> > +{
> > + /* Nothing to do */
> > +
> > + return OPAL_SUCCESS;
> > +}
> > +
> > +
> > +
> > +/
> > ************************************************************************
> > + See the note in paffinity_linux.h -- there are at least 3
> > different
> > + ways that Linux's sched_setaffinity()/sched_getaffinity() are
> > + implemented. Thankfully there is the Portable Linux Processor
> > + Affinity project which determines the flavor of affinity at
> > runtime
> > + and takes care of of the problem.
> > +
> > + Using get/set affinity functions from plpa - configured with an
> > + opal prefix.
> > +
> > + User needs to set a mask with the bit number of the cpu set. We
> > provide
> > + macros to do this.
> > +
> > +
> >
> ************************************************************************/
> > +
> > +static int linux_module_set(opal_paffinity_base_cpu_set_t mask)
> > +{
> > +
> > + opal_paffinity_linux_plpa_cpu_set_t plpa_mask;
> > + unsigned int i;
> > +
> > + if (sizeof(mask) > sizeof(plpa_mask)) {
> > + return OPAL_ERR_BAD_PARAM;
> > + } else {
> > + PLPA_CPU_ZERO(&plpa_mask);
> > + for (i = 0; i < sizeof(plpa_mask) ; i++) {
> > + if (PLPA_CPU_ISSET(i,&mask)) {
> > + PLPA_CPU_SET(i,&plpa_mask);
> > + }
> > + }
> > + }
> > +
> > + if (0 != opal_paffinity_linux_plpa_sched_setaffinity(getpid(),
> > +
> > sizeof(plpa_mask),
> > +
> > &plpa_mask)) {
> > + return OPAL_ERR_IN_ERRNO;
> > + }
> > + return OPAL_SUCCESS;
> > +}
> > +
> > +
> > +static int linux_module_get(opal_paffinity_base_cpu_set_t *mask)
> > +{
> > + opal_paffinity_linux_plpa_cpu_set_t plpa_mask;
> > + unsigned int i;
> > +
> > + if (NULL == mask) {
> > + return OPAL_ERR_BAD_PARAM;
> > + }
> > +
> > + if (sizeof(*mask) > sizeof(plpa_mask)) {
> > + return OPAL_ERR_BAD_PARAM; /* look up in header file */
> > + }
> > +
> > + if (0 != opal_paffinity_linux_plpa_sched_getaffinity(getpid(),
> > sizeof(plpa_mask), &plpa_mask)) {
> > + return OPAL_ERR_IN_ERRNO;
> > + }
> > + for (i = 0; i < sizeof(mask); i++) {
> > + if (PLPA_CPU_ISSET(i,&plpa_mask)) {
> > + PLPA_CPU_SET(i,mask);
> > + }
> > + }
> > +
> > + return OPAL_SUCCESS;
> > +}
> > +
> > +static int linux_module_map_to_processor_id(int socket, int core,
> > int *processor_id)
> > +{
> > + return opal_paffinity_linux_plpa_map_to_processor_id(socket,
> > core, processor_id);
> > +}
> > +
> > +static int linux_module_map_to_socket_core(int processor_id, int
> > *socket, int *core)
> > +{
> > + return
> > opal_paffinity_linux_plpa_map_to_socket_core(processor_id, socket,
> > core);
> > +}
> > +
> > +static int linux_module_get_processor_info(int *num_processors, int
> > *max_processor_id)
> > +{
> > + return
> > opal_paffinity_linux_plpa_get_processor_info(num_processors,
> > max_processor_id);
> > +}
> > +
> > +static int linux_module_get_socket_info(int *num_sockets, int
> > *max_socket_num)
> > +{
> > + return opal_paffinity_linux_plpa_get_socket_info(num_sockets,
> > max_socket_num);
> > +}
> > +
> > +static int linux_module_get_core_info(int socket, int *num_cores,
> > int *max_core_num)
> > +{
> > + return opal_paffinity_linux_plpa_get_core_info(socket,
> > num_cores, max_core_num);
> > +}
> > +
> > _______________________________________________
> > svn-full mailing list
> > svn-f...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/svn-full
>
>
> --
> Jeff Squyres
> Cisco Systems
