Dear Colleagues:
Earlier this year we requested input to help us shape our plans for
Globus' future. (http://www.mail-archive.com/gt-user@lists.globus.org/msg00927.html
). Since then we have been listening, discussing, exploring
alternatives, and finally making decisions about our future efforts in
Globus. This email is a long-overdue follow-up to that previous note,
summarizing our plans.
Note that this mail describes plans only for those portions of Globus
led primarily by U Chicago, Argonne, and USC/ISI. The Globus
community is larger than just these institutions, with substantial
contributions from many individuals and organizations based on many
distinct project and funding priorities. Our plans highlighted here
includes much input from these contributors, but do not speak for
those individuals and organizations.
A) EVENTS
We are planning several events in the coming months where Globus'
future (and present) can be further explained, discussed, and refined,
including:
* SC'09 (http://sc09.supercomputing.org/), November 15-20, Portland,
Oregon, USA: Please come visit us at the Argonne National Laboratory
booth.
* Viva Globus, December 16 at Argonne (near Chicago, Illinois, USA):
It has been a while since our last "Viva Globus" contributors
meeting. We will revive this meeting series on December 16. More
information will be forthcoming soon. This meeting is intended for
substantial contributors to the Globus software, as an opportunity to
update each other on our respective activities, discuss future plans
and collaborations, and make decisions regarding community governance
and processes. Of particular focus at this next Viva Globus will be
discussions and decisions regarding the future of the Globus Alliance
governance.
* GlobusWorld, March 2-4 at Argonne (near Chicago Illinois, USA):
This latest installment of the GlobusWorld user conference series will
be focused on users of the Globus software, including talks and
tutorials on using new (and old) Globus software, and presentations
and discussions by Globus users on their experiences. More
information about this conference will become available starting at
SC'09.
B) DECISIONS
The following points summarize the decisions that we have made over
the past 6 months during the re-evaluation process:
1. Of paramount importance is improving the quality of the Globus
software and the service we provide to Globus users. We will continue
to maintain and support the existing Globus Toolkit version 4.x (GT4)
software as long as demand exists and funding allows, minimally
through the end of the CDIGS project in December 2010. And in all
changes and enhancements we are making to Globus going forward, we are
placing a strong emphasis on issues related to helping our community
migrate forward.
2. We have made some long-overdue, difficult decisions about some of
the more problematic portions of Globus, including:
2.a. There have been long struggles and confusion over the GRAM2 vs.
GRAM4 components. We have resolved these problems by re-investing in,
fixing, and enhancing GRAM2. This new version, called GRAM5, is fully
backward compatible with GRAM2 (with two minor exceptions*), but
solves its scalability issues and adds numerous frequently requested
features. It is currently in alpha testing by several major users,
who are using it with existing GT4 Java jGlobus/COG clients, C
clients, and Condor-G. It will be released soon as part of GT5. We
will continue to support GRAM4 at least through December 2010 (perhaps
longer, depending upon demand and funding), but have begun to assist
GRAM4 users in migrating to GRAM5. If you are an existing GRAM4 user
and would like to discuss migration issues to GRAM5, please contact
us. For a more complete description of the GRAM5 alpha release, see http://dev.globus.org/wiki/GRAM/GRAM5
. We welcome additional testing and bug reports on this GRAM5 alpha
release.
(*The two exceptions to GRAM5's backward compatibility with GRAM2 are:
(i) no support for MPICH-G/MPIG job rendezvous; and (ii) GRAM5 stages
out stdout/err at the end of the job rather than streaming them out
while the job runs.)
2.b. The Reliable File Transfer (RFT) service has been of
considerable interest to many Globus users, but in practice has
suffered from difficulties in both use and operation. We have decided
to replace the RFT functionality with a new Globus.org service,
described below.
2.c. GT4 Java Core is based heavily on obsolete technology (Apache
Axis 1.x) and standards (WSRF), yet nonetheless continues to provide
tremendous value-add to Web Services-based Grid builders, particularly
in the area of security and stateful resource management. With the
urging of, and in partnership with, some of our large Java Core users
such as the caGrid team at Ohio State University, we have begun the
Globus Crux effort to update our Java Web Services stack to newer
technologies (e.g., Apache CXF), while preserving and enhancing our
core value-add security capabilities as a plug-in to CXF and allowing
for WSRF protocol compatibility. We expect to release an alpha version
of Crux by the end of 2009. See http://confluence.globus.org/display/whi/Crux+for+GT+Developers
for details.
2.d. While MDS is applicable to a broad range of monitoring and
discovery tasks, in practice its predominant use has been to build
service registries/catalogs for TeraGrid, caBIG, and BIRN, with
limited adoption for systems monitoring in a few other communities.
Since MDS4 is intimately intertwined with GT4 Java Core, it would
require a substantial reimplementation effort to update it to Crux.
Meanwhile, the state of monitoring tools has evolved considerably
since MDS4 was first conceived, with the widespread adoption of highly
capable, open source monitoring tools such as Nagios. Therefore we
have begun work on a more focused effort to design and implement next
generation service registry capabilities using Crux, which we are
calling our Integrated Information Services (IIS) effort. This IIS
effort is still in the requirements gathering phase, with no releases
planned until sometime in 2010. We recommend that monitoring needs be
met using other tools such as Nagios.
3. Development continues unabated for the other components of the
Globus Toolkit (e.g., GridFTP, RLS, Myproxy, GSI-OpenSSH, etc) and the
many other active dev.globus components (http://dev.globus.org).
4. We plan to release Globus Toolkit version 5 (GT5) in late 2009, as
recently announced (http://www.mail-archive.com/gt-user@lists.globus.org/msg01311.html
). Like previous versions of GT, this version will continue to offer a
collection of tools that Grid builders can use to create a wide
variety of Grid solutions for specific communities. GT5 will include
GridFTP, GRAM5, RSL, Myproxy, GSI-OpenSSH, and the relevant underlying
C libraries such as GSSAPI, XIO, C Core, etc. Note that GT5 will not
include Java Core. Instead, we will continue to support GT4 Java Core,
and will work with our users to migrate GT4 Java Core services to Crux
when it becomes available.
5. Globus Toolkit version 5.2, targeted for Q1 2010, will focus on
repackaging the GT5 components into independent component releases
that leverage OS-native packaging approaches (e.g., RPM), with
assistance from other groups (e.g., KnowARC) who have already blazed
this trail. Subsequent GT releases in the remainder of 2010 will focus
primarily on usability and reliability, along with features required
by Globus.org. The repackaging effort will not impact backward
compatibility with GT 5.0. We expect GT 5.2 clients and services to
be fully compatible with GT 5.0.
C) THE GLOBUS.ORG ONLINE SERVICE
We are creating a new Globus.org online, hosted service (i.e.,
Software-as-a-Service), to provide higher-level, end-to-end Grid
capabilities, targeted to end users, as well as Grid builders looking
for more complete solutions to build upon. Initial functionality of Globus.org
will focus on replacing and enhancing the RFT functionality of
reliable, high-performance, fire-and-forget data transfer, but over
time will grow to include more “collective layer” functionality (as
described in the “Anatomy of the Grid” paper 1]). We plan to debut and
demonstrate Globus.org at SC'09 next month. We will begin operating a
beta version of this service in November 2009 to a limited set of
initial users, with a substantial ramp-up starting in early 2010.
During the remaining 14 months of the CDIGS project, we intend to
focus more resources toward Grid data management problems. We have
seen tremendous growth in GridFTP usage over the past 2 years. Through
a combination of usability and packaging improvements to GridFTP,
along with the introduction of end-to-end Grid data management
capabilities in Globus.org, we intend to substantially increase the
value and resulting usage of the Globus data management software.
D) NEW LEADERSHIP
We are pleased to welcome returning and new leadership to our Globus
team. Steve Tuecke, who was the Globus lead architect for its first
10 years before leaving five years ago to start a company, returned to
the University of Chicago in January 2009, and has resumed both
technical and project leadership for our Globus activities.
Additionally, the University of Chicago recruited Paul Dave’ to a new
Director of User Services position in August 2009, with a mandate of
dramatically improving the quality and methods by which we provide
services to Globus users, including support, consulting and
operations. We have also re-organized our software development team
and processes, with the introduction of widely adopted Agile Scrum
development practices.
We are excited about this re-invogoration of the Globus software and
community, and the increasing value the Globus community can bring to
the many multi-institutional scientific and biomedical communities
whose need for robust Grid computing middleware continues to grow
unabated.
Viva Globus!
The Globus Team
[1] “The Anatomy of the Grid: Enabling Scalable Virtual
Organizations,” Ian Foster, Carl Kesselman, Steven Tuecke.
International Journal of Supercomputer Applications, 15(3), 2001.
