Everyone,
There have been a number of threads that have been running on the list over
the past couple of weeks that all revolve around the new overlay feature of
7.6.4.
The goal of this note is not to try and address every specific issue or to
argue one way or the other on every given topic. It is to put forth the basic
ideas and philosophy for this feature.
To start, overall we are having customers have a very good experience with the
feature. It is working. It is delivering the capability that it is designed
for. It is indeed keeping user customizations separate from original
definitions and the applications are running correctly with the definitions in
place. I think that after reading through all the comments and notes that
folks have posted on issues or concerns about this feature you will find that
ALL of them are about issues on the definition side and that there have not
been any issues reported from the execution side or about the run-time
capability of the system. In addition, there have not been any issues from
users (OK, we have received a few "polish and efficiency of use" enhancement
requests) who are coming into the feature new without a history. However,
there have been some issues on the converting into side -- some because of
issues and some because of not approaching the feature correctly.
What I want to do in this note is to walk through the intention of how the
feature is intended to be used and to provide clarity and "one voice"
(subject to update if I make a mistake but that would just be a correction to
still provide a single position/answer from BMC) about the feature and its
intended use. During this discussion, I hope to address the themes (if not
the specific point items) from the various thread that have been posted on the
list recently.
The goal of the feature is to allow you to isolate out-of-the-box application
definitions from your enhancements/customizations. This is accomplished by
putting definitions into two layers -- a base layer for out-of-the-box and
an overlay layer for both changes to out-of-the-box definitions and for new
definitions custom to your site. In addition, to put infrastructure in place
in the system to assist you with following the rules and to prevent you from
changes that will cause upgrade issues or produce incompatibilities going
forward into the future. The result of this is better understanding of the
application changes and extensions made and the ability to dramatically improve
the upgrade, and subsequent recustomize, operation.
All out-of-the-box definitions are loaded into the base layer. This works
effectively for all BMC applications -- new and old versions. Your changes
and customizations are in another layer and will not be touched by the changes
in the BMC applications yet all changes to the BMC applications will show
through for every object except where you have explicitly changed them for your
own requirements.
This feature is not a "dev to production" feature. The purpose is not to
make test changes and then "promote to base". Base is the out of the box
application and you should not change it. Overlay is your changes. There
should not be a need to "move something from overlay to base" in the use of
this feature in its intended use case.
As with any feature, it is important to understand the parameters of the
feature and what it is intended to be used for. Use outside those parameters
often does not work like expected.
The interaction through Dev Studio is generally effective and is working well
for customers. There are a few things that are showing up where we can make
things more efficient (like allowing you to overlay a field and us helping by
overlaying the form automatically if it is not already overlaid rather than
making you overlay the form first and then the field).
The one area where there has been some confusion is the area of "converting into
the use of the feature" in an existing environment. This is an important
area. Originally, there was a discussion about how much we could help here and
concern that we could not get 100% right so maybe we shouldn't do anything other
than document how you convert. But, we decided that this wasn't enough so we
built a conversion utility (BPCU) to provide that conversion assistance. It
cannot be 100% perfect but we did run tests on a number of large customer
environments during development and resolved the vast majority of issues so
that it was converting those environments (and they were large ones with
various levels of the applications and various levels of customization)
effectively with minimal manual cleanup needed.
This leads to point #1 -- the use of the overlay feature is optional. You do
not have to use it. You can set the configuration of the system into base mode,
leave everything in base mode, not use overlays at all. The sytem will perform
EXACTLY like it always had from a definition perspective. All your definitions
are in the same layer. You have to worry about conflicts. You have to deal
with rules for customizations and restoring things around upgrades.
This can be done now. The only suggestion is to set the default overlay
characteristic to base so that all references base at all times.
You can follow the directions to convert into using the feature at any point in
the future. You would follow the same directions later as someone would follow
now to convert into using the feature. Now, in the future, we are going to
optimize our packaged application installers to assume you are using the feature
to protect your customizations and extensions so we do recommend that you do
convert into the use of this feature over time. The key is that it does not
have to be now if you don't want to.
So, for customers who want to upgrade to 7.6.4 AR System but don't want to use
this new capability for now, this is possible (and easy).
Now, we get to point #2 -- the BPCU utility. The goal of this utility is to be
a ONE TIME run utility that will take a look at every definition within your
system, check to see if that definition matches the image of any flavor of the
application from 7.0 through 7.6.4, if it does, to take no action, if it does
not match in any way, to create an overlay for that defintion in the overlay
layer that is an exact copy of the definition. The expectation is that this
utility is run ONE TIME (OK, unless you use the option to just specify a
specific set of forms because for some reason you want to move some forms now
but others later but that is still "one time" per definition).
Yes, the documentation and some comments have said otherwise. But, that is all
about responding/reacting to a situation where you could try and take advantage
of a utility to work around something or cause something to occur. That doesn't
change the overall model that this utility is intended to be a one time run
utility that you will not use again after the initial conversion.
Closely related is point #3 -- the result of the BPCU run. As noted above in
point #2, the utility looks at every definition and if it does not match in
every particular with one of the definitions from some release from 7.0 through
7.6.4, it will flag it as different. We took this approach to make sure that
we correctly made a copy of any definition that you have changed in any way.
That change could be anything from a serious change to a filter by adding 5
new actions and changing the qualification to something as subtle as changing
the x/y coordinate by 1 pixel on a field. We have no idea how important the
difference is and whether you want to make sure that that difference is
preserved as a change to the definition so we err on the side of making an
overlay for it. Then, you can review the things that overlays were made for
and if you don't want to consider any difference as a custom change and you are
OK for any difference that does exist be overwritten by a future app upgrade,
you can simply remove the overlay definition. For those who have seen a few
extra things created as overlays, I think you will agree that this conservative
model for making overlays just in case when there is a difference is better
than ignoring a difference and possibly having customizations overwritten in
the future because there is not an overlay.
Of course there is another option... You don't have to run BPCU at all. You can
simply go into the system and create overlays for the things you have modified
yourself and not use the utility at all. This will create only the overlays
you want and will not create any extras. You have to know what you have
modified (and want to preserve) of course, but this is an option.
So, we are now to the point of
Understanding that I don't have to use the feature right away, I can use it when
I am ready.
Understanding the role of BPCU -- a ONE TIME conversion utility.
Understanding the action of BPCU when finding a difference and the conservative
action it takes. And, that you may have a minor amount of correction to do --
that might include deleting the overlay even if there is change because you
don't want to preserve that change going forward and are OK if it is
overwritten.
There is an area that has come up where there are several different approaches
that can be taken -- and each is valid to some degree -- where we have not been
careful to document the ideas. And, in one case, where we have not properly
exposed a feature (it exists, but there is no Dev Studio access to it) to allow
you to follow the choice you made.
This area is all about what to do with imports of forms/definitions into a
system where you are using overlays and the import definitions are from a
release pre-overlay. This means that the definitions themselves have no
concept of base or overlay or custom in them.
For backward compatibility and consistency with the past, we have all imports
where there is no identification come into the base layer. This means older
out-of-the-box applications work the same way they always have. This means
that older upgrades work the same way they always have.
The challenge is what if that thing being imported is
-- from a partner or third party
-- custom development that you did not have on the system when you converted
-- customizations to out of the box applications that you want to put into
the new system as overlays
I will take the last one first -- this is simply not designed for. There are no
plans at this time to add any assistance for that layer. Any customizations
you had are assumed to be on the system before the conversion and the conversion
utility would have taken care of creating the overlays. We do not intend to
support some form of importing customization (by this I mean changes to existing
defintions NOT new extensions which are now called custom objects which are
really type 2 from the list) to existing objects into the system as overlays
from an older environment. We do fully support exporting overlays from a
system and importing them to another system as an overlay.
The good news is that to this point, we have not had a single request for this
type of functionality which justifies our decision.
Continuing backward, you have the custom development (new objects/definitions)
that were not on the system. These will be imported into the base mode as they
are unidentified. Since they are custom objects really, they need to get into
the overlay layer as custom objects.
If the objects were on the system before the one time run of BPCU, they would
have been converted to custom objects. But, they were not. The expectation is
not to run BPCU again -- it is for one time conversion use. What you need is a
way to promote a definition from base to a custom object.
We actually have logic to do that promotion (we also have logic to move
something from custom to base but that is not really something you should be
needing to do unless you made a mistake in a promotion). Unfortunately, we did
not expose that operation in Dev Studio. This is an oversight that we plan to
correct in a future release. You would then be able to simply go into base
mode, select the object(s) to promote and promote them.
The good news is that since it is an API call and the driver program that is
supplied with all installs allows you to call the API call that does the
work, there is a way you can use driver to promote things from base to custom.
ARSList doesn't allow me to attach things from what I understand so I have
included the text of a driver script that shows the calls for each type of
object to promote it. You could create a simple file that loops across a list
of things and promotes them. Again, this is a workaround, but it closes the
gap in Dev Studio. It is not for everyone; but for those experienced with using
driver and needing a solution and willing to put in a little extra work today,
it is available.
Finally, we have the first category, a partner/third party app. For this type
of app, there is one of two directions you can head short term and then in the
future when we enhance the overlay feature to the next level there is a best
practice for the future that every partner/third party should take advantage of
to give the best result.
Today, Option 1 -- The goal of the base is to hold out-of-the-box applications.
Well, it doesn't have to be just applications from BMC. As long as the third
party app is a well behaved app (i.e. it doesn't modify BMC code but is just its
application), it can be installed in and live in Base. You shouldn't be
modifying their out-of-the-box app just like you shouldn't be modifying BMC's.
You may want to overlay some of their definitions to tailor them just like you
would BMC's. In general, I would argue that this is the best practice for today
for a third party app that is well behaved and does not modify BMC app
definitions.
Note that there is a way in BPCU to exclude objects from the evaluation. It is
an XML file format and you have to list all the objects you want to exclude so
it may be a bit of a pain to build up the list (of course the third party
provider could do it for you and they know the objects that are theirs so this
should not be hard for them). This would mean that if you have the app already
installed, you can specify the exclude file when running BPCU to prevent it from
converting objects that you want to stay as base objects that BMC doesn't know
about.
Today, Option 2 -- Put the third-party app into the overlay. To do this, you
really need the third party to either be overlay aware and have built it this
way or you need to promote the definitions from base to overlay. That is what
the missing feature and driver script workaround that I discussed earlier is
about. You would import the definitions into base and then promote them to
overlay. There are two main challenges. One, you cannot overlay anything in
the third-party app because it is already in the overlay layer. So, you are
back in the old model of modifying an app or you cannot modify it. Two, what
do you do when an upgrade comes along? The upgrade will work on things in the
base layer (assuming the third-party has not made their app overlay aware) but
your definitions are in the overlay layer.
Future Best Practice -- In the future, we will be introducing overlay layers
so that you can have multiple levels of overlays with each level independent
of each other and overlaying each other. When that is available, the best
practice will be for the third party to have their entire app in an overlay
layer as custom objects. Then, that layer can be added/upgraded/whatever in
the system completely independently of base or of other overlay layers that
may in fact overlay things in that third-party app as well. Then there will be
a BMC base layer, a third-party app custom overlay layer (and maybe more than
one), and then a customer app custom/overlay overlay layer. But, this is more
than a year away and third parties have to build their apps to the concept to
be seamless so you will need to use one of the "Today" options for now.
One final topic. In the documentation there is a discussion of reloading the
application after running BPCU and getting all the overlays created for your
custom objects. This step is highly recommended at some point to restore the
original out-of-the-box definitions in to the base layer. Without this, you
don't have the original definition on the system as the base layer is still your
modified version for anything that an overlay was automatically created. The
biggest benefit of the feature is to have the original definition in place in
the base and your changes in the overlay layer. Again, this is not required
for things to work, but it is highly and strongly recommended to best put the
system in a mode to fully and completely realize the value of the overlay
feature. You can defer it if you like, but you should have it in your plans
as one of the steps toward fully adopting overlays.
So, a long message, but I wanted to make sure to capture a holistic discussion
of the feature, what it is used for, the thinking and intention behind it, and
the best options for how it should be used. Through it all, we did identify
one feature that is not properly exposed that would really help if it was (and
provided a workaround you can use now).
I hope this note is useful and helps address some of the concerns and issues
that have been being discussed. The note has definitely not addressed all
aspects of every question -- that was not the goal. But, I hope it has given
some insight into this feature that addresses the topics behind many of the
questions that have been discussed.
We are committed to this capability. We know that it will fundamentally change
the way you can manage your applications and will give you a much more effective
and powerful model. If there are aspects we can polish, please let us know and
we will consider them for the future.
Doug Mueller
PS For the person who was getting a crash in BPCU -- that is definitely
something to report and work with the support team on. You should not be
having the utility crash on you.
Sample driver commands to move things from base to custom. There is really
one command -- cofo. Once you set things up, you run this command with the
appropriate parameters for the type of object you are changing (there are
examples for each type of object), and you have moved something from base to
a custom object in the overlay layer. You could move one thing, create a file
of things to move and build a batch file to run across the list or whatever.
It is a workaround and you should only have to move the things one time.
(For those who want to move custom to base, change the first parameter of the
cofo call below from a 1 to a 3 and that should move it that direction.)
ssc
1961
4
1
ssc
1962
4
1
#convert form to custom
cofo
1
___Get
1
#convert field to custom
cofo
1
536870928
15
___Get
536870928
#
#convert view to custom
cofo
1
536870930
14
___Get
536870930
#
#convert filter to custom
cofo
1
___Get
5
#convert Active Link to custom
cofo
1
___Get
6
#
#convert Escalation to custom
cofo
1
___Get
9
#
#convert Menu to custom
cofo
1
___Get
8
#
#convert Container(ALGuide,FilterGuide,WebService) to custom
cofo
1
___Get
12
#
#convert Image to custom
cofo
1
___Get
17
#
_______________________________________________________________________________
UNSUBSCRIBE or access ARSlist Archives at www.arslist.org
attend wwrug11 www.wwrug.com ARSList: "Where the Answers Are"
