Send kea-dev mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/kea-dev
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of kea-dev digest..."
Today's Topics:
1. On lease expiration (Tomek Mrugalski)
2. Re: On lease expiration (Shawn Routhier)
----------------------------------------------------------------------
Message: 1
Date: Tue, 08 Jul 2014 22:36:36 +0200
From: Tomek Mrugalski <[email protected]>
To: [email protected]
Subject: On lease expiration
Message-ID: <[email protected]>
Content-Type: text/plain; charset=ISO-8859-1
This is another mail preparing us for tasks that are planned past 0.9
release.
Currently Kea does not take any specific actions when a lease expires.
However, when picking new leases for a new client, it can reuse a lease
if it is expired. This approach is not ideal, it was just the best we
could do given the schedules we had. We know that this is not ideal,
hence we plan to implement it properly after 0.9 release.
REQUIREMENTS
Upon lease expiration, we need to initiate one or more actions. Those
actions are (the list is not exhaustive and may grow over time):
- logging the expiration (that sometimes is a requirement from the law
enforcement perspective)
- triggering DDNS removal (required every time DDNS is used)
- calling hooks (new hook point will be defined for this)
- notifying failover partner (we don't have failover support yet, but
we will get it one day)
- notifying active leasequery requestors (we don't have active LQ
support, but we will get it one day)
- remove the lease from the DB (that may be optional, as the lease
could be fixed or maybe sysadmin prefers to keep expired leases, so
the returning client will get the same address if it's still
available)
POTENTIAL SOLUTIONS
In all cases we need to implement house-keeping routine that will pick
expired leases, perform all actions on it and possibly remove it. That
routine will look the same, regardless of how we choose to trigger it.
How this could work? There are couple possible approaches:
a) Implement house-keeping process that runs in the background. It will
be periodically sweeping the database and will pick expired leases. The
biggest advantage of this approach is its relative simplicity.
Implementation is not related to any existing code, so the likelihood of
any side effects are minimal. It seems easy to implement, but in fact it
is quite complex, as it will need to implement its own configuration
parser. Once we decide to go multi-process, it would be reasonable to
have a single house-keeping process. These are all benefits I can think
of. Unfortunately, there are numerous disadvantages. First, it would be
inherently incompatible with memfile, our default backend. This is a big
problem for me as I think that memfile will also be the most popular
backend due to its installation simplicity. Second, stand alone process
would call all external actions in a separate process, so users writing
hook libraries will need to develop multiprocess capabilities, even if
we have only one Kea server. Third, I have no idea how to implement
failover with stand alone house keeper. It will have to notify somehow
the Kea server that a lease expired and its partner must be notified (or
other failover actions must take place). So we're talking about
inter-process notification protocol here on top of failover. Finally,
this is a new process that needs to be maintained (new makefiles, new
man page, new binaries to install, new documentation etc.). The bottom
line for me is that if we decide to do stand-alone process, we're
getting short term benefits, but we'll pay for it with much larger
development and maintenance costs in the long term.
b) implement house-keeping as part of the existing process. There would
be a configuration parameter that would tell the server to trigger the
routine every X seconds. The server would call select() for at most X
seconds, run house-keeping and then call select() again. This approach
seems very flexible. If you really need a notification the very second
when the lease expires, set X to 1. If you're experiencing an event
where extra performance is needed, set it to some large value, so
expiration will be checked upon the next day. Of course, there are
tradeoffs. The longer value set means that, although the procedure will
not happen that often, each house-keeping will take longer as it will
have more leases to process. The benefit of this approach is that
logging, hooks, ddns happen in the same process, so no multi-process
hassles for us and for hook lib developers. Also, failover does not
become more complicated. It is easy to implement as the whole design of
existing code is ready for this (calculating timeout + calling select()
with that timeout).
c) that's really b) + extra capability of being able to disable
house-keeping, at least during normal operation. House-keeping would be
called during startup and shutdown. So if you really need the absolute
max performance and intermediate pauses needed for house-keeping are a
problem for you, use this. You will get the lease expiration
notification after some time ("Least A expired Y minutes ago"), but
that's ok if max performance is your goal.
d) That's b) + an optimization. When assigning or updating a lease, we
can keep the timestamp to the shortest expiration event and dynamically
set X to that value. So if there are no leases, we'll select() for
MAX_UINT. If there are 1000s of leases, we'll select() until the
shortest (or oldest) one expires. This is how expiration is implemented
in Dibbler. Seems to be working fine.
Solutions b) and c) have a small disadvantage when we decide to enable
Kea to run multiple instances. That will mean that if you have X
processes, the house-keeping will be run X times. In principle that's
not optimal, but it isn't that bad. Although there will be multiple
checks run, but on average each routine will have to only process X
times less leases. Also, this can be easily solved by having a
designated process (e.g. only the first instance runs house-keeping,
additional instances have house-keeping disabled).
e) Over time, we will also need to implement a new command: lease-expire
or db-cleanup. It would add extra capability for admins who tweaked
something in the DB and want Kea to process the changes. Also, some
admins would possibly want to use their external scripts to do lease
expirations, e.g. during low traffic at 3am.
Actual house-keeping routine implementation is lease database specific.
For SQL-based backends, we'll probably implement a query that returns
expired lease. For memfile, we'll either add an index or will have to
traverse the leases. It would be good if we could do that using
bisection. I haven't looked at what's available in multi-index containers.
Thoughts? Comments?
Tomek
------------------------------
Message: 2
Date: Tue, 8 Jul 2014 18:18:10 -0700
From: Shawn Routhier <[email protected]>
To: Tomek Mrugalski <[email protected]>
Cc: [email protected]
Subject: Re: On lease expiration
Message-ID: <[email protected]>
Content-Type: text/plain; charset=us-ascii
A couple of quick comments about what the current code does.
The current code schedules a routine to be run when the next lease
should expire. As this can be delayed while we process requests we
may end up processing a number of expires before returning to processing
requests. A customer has mentioned this to us multiple times as a problem as it
results in a wave effect. Spend time doing requests then time doing
expires but no requests then back to requests. I imagine they would
suggest trying to move the expiration work into a different task.
While we do some cleanup on the v4 lease on expiration the lease
still retains some information to allow us to prefer it for the previous client.
For v6 the current code picks the NA address by using a hash of the
DUID so there is a built in preference without needing to maintain the
lease information.
**
For failover I'm curious if we could push the failover work into a separate
task and have the server and the expire send events to it to update the
peer. I think this might be an idea that's better in theory than in practice
though. As I think about it more I think the amount of information flow out
of that task into other areas (for example - the peer has issued a lease,
now we need to rebalance) is probably prohibitive.
Shawn
On Jul 8, 2014, at 1:36 PM, Tomek Mrugalski wrote:
> This is another mail preparing us for tasks that are planned past 0.9
> release.
>
> Currently Kea does not take any specific actions when a lease expires.
> However, when picking new leases for a new client, it can reuse a lease
> if it is expired. This approach is not ideal, it was just the best we
> could do given the schedules we had. We know that this is not ideal,
> hence we plan to implement it properly after 0.9 release.
>
> REQUIREMENTS
> Upon lease expiration, we need to initiate one or more actions. Those
> actions are (the list is not exhaustive and may grow over time):
> - logging the expiration (that sometimes is a requirement from the law
> enforcement perspective)
> - triggering DDNS removal (required every time DDNS is used)
> - calling hooks (new hook point will be defined for this)
> - notifying failover partner (we don't have failover support yet, but
> we will get it one day)
> - notifying active leasequery requestors (we don't have active LQ
> support, but we will get it one day)
> - remove the lease from the DB (that may be optional, as the lease
> could be fixed or maybe sysadmin prefers to keep expired leases, so
> the returning client will get the same address if it's still
> available)
>
> POTENTIAL SOLUTIONS
> In all cases we need to implement house-keeping routine that will pick
> expired leases, perform all actions on it and possibly remove it. That
> routine will look the same, regardless of how we choose to trigger it.
>
> How this could work? There are couple possible approaches:
> a) Implement house-keeping process that runs in the background. It will
> be periodically sweeping the database and will pick expired leases. The
> biggest advantage of this approach is its relative simplicity.
> Implementation is not related to any existing code, so the likelihood of
> any side effects are minimal. It seems easy to implement, but in fact it
> is quite complex, as it will need to implement its own configuration
> parser. Once we decide to go multi-process, it would be reasonable to
> have a single house-keeping process. These are all benefits I can think
> of. Unfortunately, there are numerous disadvantages. First, it would be
> inherently incompatible with memfile, our default backend. This is a big
> problem for me as I think that memfile will also be the most popular
> backend due to its installation simplicity. Second, stand alone process
> would call all external actions in a separate process, so users writing
> hook libraries will need to develop multiprocess capabilities, even if
> we have only one Kea server. Third, I have no idea how to implement
> failover with stand alone house keeper. It will have to notify somehow
> the Kea server that a lease expired and its partner must be notified (or
> other failover actions must take place). So we're talking about
> inter-process notification protocol here on top of failover. Finally,
> this is a new process that needs to be maintained (new makefiles, new
> man page, new binaries to install, new documentation etc.). The bottom
> line for me is that if we decide to do stand-alone process, we're
> getting short term benefits, but we'll pay for it with much larger
> development and maintenance costs in the long term.
>
> b) implement house-keeping as part of the existing process. There would
> be a configuration parameter that would tell the server to trigger the
> routine every X seconds. The server would call select() for at most X
> seconds, run house-keeping and then call select() again. This approach
> seems very flexible. If you really need a notification the very second
> when the lease expires, set X to 1. If you're experiencing an event
> where extra performance is needed, set it to some large value, so
> expiration will be checked upon the next day. Of course, there are
> tradeoffs. The longer value set means that, although the procedure will
> not happen that often, each house-keeping will take longer as it will
> have more leases to process. The benefit of this approach is that
> logging, hooks, ddns happen in the same process, so no multi-process
> hassles for us and for hook lib developers. Also, failover does not
> become more complicated. It is easy to implement as the whole design of
> existing code is ready for this (calculating timeout + calling select()
> with that timeout).
>
> c) that's really b) + extra capability of being able to disable
> house-keeping, at least during normal operation. House-keeping would be
> called during startup and shutdown. So if you really need the absolute
> max performance and intermediate pauses needed for house-keeping are a
> problem for you, use this. You will get the lease expiration
> notification after some time ("Least A expired Y minutes ago"), but
> that's ok if max performance is your goal.
>
> d) That's b) + an optimization. When assigning or updating a lease, we
> can keep the timestamp to the shortest expiration event and dynamically
> set X to that value. So if there are no leases, we'll select() for
> MAX_UINT. If there are 1000s of leases, we'll select() until the
> shortest (or oldest) one expires. This is how expiration is implemented
> in Dibbler. Seems to be working fine.
>
> Solutions b) and c) have a small disadvantage when we decide to enable
> Kea to run multiple instances. That will mean that if you have X
> processes, the house-keeping will be run X times. In principle that's
> not optimal, but it isn't that bad. Although there will be multiple
> checks run, but on average each routine will have to only process X
> times less leases. Also, this can be easily solved by having a
> designated process (e.g. only the first instance runs house-keeping,
> additional instances have house-keeping disabled).
>
> e) Over time, we will also need to implement a new command: lease-expire
> or db-cleanup. It would add extra capability for admins who tweaked
> something in the DB and want Kea to process the changes. Also, some
> admins would possibly want to use their external scripts to do lease
> expirations, e.g. during low traffic at 3am.
>
> Actual house-keeping routine implementation is lease database specific.
> For SQL-based backends, we'll probably implement a query that returns
> expired lease. For memfile, we'll either add an index or will have to
> traverse the leases. It would be good if we could do that using
> bisection. I haven't looked at what's available in multi-index containers.
>
> Thoughts? Comments?
>
> Tomek
> _______________________________________________
> kea-dev mailing list
> [email protected]
> https://lists.isc.org/mailman/listinfo/kea-dev
------------------------------
_______________________________________________
kea-dev mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/kea-dev
End of kea-dev Digest, Vol 4, Issue 1
*************************************
