Alex Karasulu wrote:
Thinking more about how to update the schema and the mechanics involved
along with the various trade offs. Here's how this should progress:
Schema Change Steps
(1) A schema change operation arrives and changes are performed on a cloned
set of registries where the impact of the change is calculated in terms of
the total number of indirect dependents affected.
When you talk about 'indirect dependents affected', you mean affected
SchemaObjects, I guess.
(2a) If the operation is consistent then a schema-txn-begin message is sent
to all listeners informing them of the OIDs impacted by the schema change
calculated in the first step. This lets objects dependent on these schema
entities to respond to the situation. This message is delivered
synchronously. Access to the old Registries is provided before the swap in
case the object needs them via the SchemaManager handle if they are exposed.
There is an issue here, if the message is synchronous, as many of the
ServerEntries (or constants used all over the code) may be in memory,
without any thread using them. Some of the use cases are :
- objects in cache (we have many different cache, like the referral
cache, the alias cache, etc)
- objects currently being built, but not completed, waiting for some
more data from the client side (in the codec)
- objects which has already been checked against the old schema, but are
not yet processed by the backend, and may become invalid
There is also another potential problem if the modification impacts the
index (like if we removed an indexed AttributeType).
Anyway, sending synchronous messages is simply impossible.
(2b) If the operation leaves the schema in an inconsistent state then no
message is sent to schema listeners and the schema change is rejected.
That I agree on.
(3) The schema registries are swapped out.
(4) The SchemaManager then sends a schema-txn-commit message to all
listeners to now be able to access the modified schema elements they're
interested in from the newly swapped in registries. This message is
delivered synchronously.
See point 2a above.
Schema Change Responses
(1) Invalidate the object in question so all using/run processes and caches
can reset or update themselves via the acquisition of a new valid object to
continue the process.
Clearing the cache is an option, but for aliases or referals, it will
request a new search to get back all the aliases or referrals. Can be
costly... But maybe the problem is not that this operation is costly,
but that it's the way we handle aliases and referrals which is wrong...
(2) Make the object depending on schema objects valid by updating stale
references and hope for the best. Sometimes timing will cause certain
failures but this is really OK with us since this is a rare situation to
update the schema.
Assuming that an synchronous message being sent to the objects can't be
done, the other option would be to send this synchronous message to the
session and to the cache managers, as they have direct access to the
objects. Let's say the session is not active (no thread to process it),
and let's say it's reactivated at some point : the first thing to do
would be to check if the schema has changed, and if so, propagate the
new schema to all the related objects. That mean we must register
aother listener in the objects to react to this session modification.
the very same for the cache managers.
About the cache, we have to use a better strategy than the one we are
using? It's insane to have so many caches all over the server. We should
use one generic cache system where we store all the different elements
we want to cache, and have a dedicated thread to manage the global
cache. But the first step would be to list all the caches we are using...
Interesting times !
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
