This is an automated email from the ASF dual-hosted git repository.
zrhoffman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/trafficcontrol.git
The following commit(s) were added to refs/heads/master by this push:
new d640e91 TR: cache and reuse RRSIG records to avoid unnecessary
re-signing (#5598)
d640e91 is described below
commit d640e9151fa626a9aa9b3aab8f4a8e2faa1e6ff9
Author: Rawlin Peters <[email protected]>
AuthorDate: Thu Mar 4 14:55:02 2021 -0700
TR: cache and reuse RRSIG records to avoid unnecessary re-signing (#5598)
* TR: cache and reuse RRSIG records to avoid unnecessary re-signing
Add a new TR profile parameter: 'dnssec.rrsig.cache.enabled'. When set
to 'true', RRSIG records will be cached and reused when the same RRset
is re-signed. This heavily reduces the amount of CPU used to re-sign
zones after a change is made that requires re-signing large numbers of
zones (e.g. ONLINING/OFFLINING Traffic Routers). Also, this causes
Traffic Router to startup much quicker, because duplicate work is
eliminated during the initial zone signing process.
As DNSSEC keys are removed/replaced, the RRSIG cache is cleaned up, and
entries that are no longer necessary are removed. RRsets that have a
cached RRSIG record that is more than halfway to expiring will be
re-signed.
Setting 'dnssec.rrsig.cache.enabled' to 'false' will clear the cache and
prevent it from being used for new requests until re-enabled again.
* Remove unused getters, collapse lambda expressions
---
CHANGELOG.md | 1 +
docs/source/admin/traffic_router.rst | 5 +
.../traffic_router/core/dns/RRSIGCacheKey.java | 48 +++++++
.../traffic_router/core/dns/RRsetKey.java | 59 ++++++++
.../traffic_router/core/dns/SignatureManager.java | 82 +++++++++--
.../traffic_router/core/dns/SignedZoneKey.java | 28 +++-
.../traffic_router/core/dns/ZoneManager.java | 7 +
.../traffic_router/core/dns/ZoneSigner.java | 8 +-
.../traffic_router/core/dns/ZoneSignerImpl.java | 46 +++++-
.../traffic_router/core/router/TrafficRouter.java | 1 +
.../core/dns/ZoneSignerImplTest.java | 160 +++++++++++++++++++++
11 files changed, 419 insertions(+), 26 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b2e5f5c..5e9bc0b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,7 @@ The format is based on [Keep a
Changelog](http://keepachangelog.com/en/1.0.0/).
- Traffic Portal: upgraded delivery service UI tables to use more
powerful/performant ag-grid component
- Traffic Ops: added a feature so that the user can specify
`maxRequestHeaderBytes` on a per delivery service basis
- Traffic Router: log warnings when requests to Traffic Monitor return a 503
status code
+- Traffic Router: added new 'dnssec.rrsig.cache.enabled' profile parameter to
enable new DNSSEC RRSIG caching functionality. Enabling this greatly reduces
CPU usage during the DNSSEC signing process.
- [#5316](https://github.com/apache/trafficcontrol/issues/5316) - Add router
host names and ports on a per interface basis, rather than a per server basis.
- [#5344](https://github.com/apache/trafficcontrol/issues/5344) - Add a page
that addresses migrating from Traffic Ops API v1 for each endpoint
- [#5296](https://github.com/apache/trafficcontrol/issues/5296) - Fixed a bug
where users couldn't update any regex in Traffic Ops/ Traffic Portal
diff --git a/docs/source/admin/traffic_router.rst
b/docs/source/admin/traffic_router.rst
index 77acbd4..90002ab 100644
--- a/docs/source/admin/traffic_router.rst
+++ b/docs/source/admin/traffic_router.rst
@@ -256,6 +256,11 @@ Much of a Traffic Router's configuration can be obtained
through the :term:`Para
| dnssec.zone.diffing.enabled | CRConfig.json
| If DNSSEC is enabled, enabling this parameter allows Traffic Router to diff
existing zones with newly generated zones. If the newly |
| |
| generated zone is the same as the existing zone, Traffic Router will simply
reuse the existing signed zone instead of signing the |
| |
| same new zone. This reduces the CPU time taken to process new snapshots and
new DNSSEC keys. Defaults to "false". |
+ | |
| NOTE: this may be removed in favor of the ``dnssec.rrsig.cache.enabled``
setting in a future release. |
+
+-----------------------------------------+------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
+ | dnssec.rrsig.cache.enabled | CRConfig.json
| If DNSSEC is enabled, enabling this parameter allows Traffic Router to cache
RRSIG records for reuse during DNSSEC signing. |
+ | |
| This greatly reduces the CPU time taken to sign DNS zones. Defaults to
"false". |
+ | |
| NOTE: this may supersede the ``dnssec.zone.diffing.enabled`` setting in a
future release. |
+-----------------------------------------+------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| dnssec.allow.expired.keys | CRConfig.json
| Allow Traffic Router to use expired DNSSEC keys to sign zones; default is
"true". This helps prevent DNSSEC related outages due to |
| |
| failed Traffic Control components or connectivity issues.
|
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRSIGCacheKey.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRSIGCacheKey.java
new file mode 100644
index 0000000..d266eba
--- /dev/null
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRSIGCacheKey.java
@@ -0,0 +1,48 @@
+/*
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.dns;
+
+import java.util.Arrays;
+
+public class RRSIGCacheKey {
+ final private byte[] privateKeyBytes;
+ final private int algorithm;
+
+ public RRSIGCacheKey(final byte[] privateKeyBytes, final int algorithm) {
+ this.privateKeyBytes = privateKeyBytes;
+ this.algorithm = algorithm;
+ }
+
+ @Override
+ public boolean equals(final Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+ return false;
+ }
+ final RRSIGCacheKey that = (RRSIGCacheKey) o;
+ return algorithm == that.algorithm && Arrays.equals(privateKeyBytes,
that.privateKeyBytes);
+ }
+
+ @Override
+ public int hashCode() {
+ int result = algorithm;
+ result = 31 * result + Arrays.hashCode(privateKeyBytes);
+ return result;
+ }
+
+}
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRsetKey.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRsetKey.java
new file mode 100644
index 0000000..96db4db
--- /dev/null
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/RRsetKey.java
@@ -0,0 +1,59 @@
+/*
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.dns;
+
+import org.xbill.DNS.RRset;
+import org.xbill.DNS.Record;
+
+import java.util.Iterator;
+
+public class RRsetKey {
+ final private RRset rrset;
+
+ public RRsetKey(final RRset rrset) {
+ this.rrset = rrset;
+ }
+
+ @Override
+ public boolean equals(final Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+ return false;
+ }
+ final RRsetKey that = (RRsetKey) o;
+ final Iterator thisIterator = rrset.rrs(false);
+ final Iterator thatIterator = that.rrset.rrs(false);
+ while (thisIterator.hasNext() && thatIterator.hasNext()) {
+ if (!thisIterator.next().equals(thatIterator.next())) {
+ return false;
+ }
+ }
+ return !thisIterator.hasNext() && !thatIterator.hasNext();
+ }
+
+ @Override
+ public int hashCode() {
+ int hashCode = 1;
+ final Iterator it = rrset.rrs(false);
+ while (it.hasNext()) {
+ final Record r = (Record) it.next();
+ hashCode = 31*hashCode + (r==null? 0 : r.hashCode());
+ }
+ return hashCode;
+ }
+}
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
index dc5f75a..1695dd8 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignatureManager.java
@@ -22,9 +22,13 @@ import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.HashMap;
+import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
@@ -39,6 +43,7 @@ import org.apache.log4j.Logger;
import org.xbill.DNS.DSRecord;
import org.xbill.DNS.Name;
import org.xbill.DNS.Record;
+import org.xbill.DNS.RRSIGRecord;
import org.xbill.DNS.TextParseException;
import com.comcast.cdn.traffic_control.traffic_router.core.edge.CacheRegister;
import
com.comcast.cdn.traffic_control.traffic_router.core.dns.ZoneManager.ZoneCacheType;
@@ -50,6 +55,9 @@ public final class SignatureManager {
private static final Logger LOGGER =
Logger.getLogger(SignatureManager.class);
private int expirationMultiplier;
private CacheRegister cacheRegister;
+ private static ConcurrentMap<RRSIGCacheKey, ConcurrentMap<RRsetKey,
RRSIGRecord>> RRSIGCache = new ConcurrentHashMap<>();
+ private static final Object RRSIGCacheLock = new Object(); // to ensure
that the RRSIGCache is totally empty if disabled
+ private boolean RRSIGCacheEnabled = false;
private static ScheduledExecutorService keyMaintenanceExecutor;
private TrafficOpsUtils trafficOpsUtils;
private boolean dnssecEnabled = false;
@@ -64,6 +72,7 @@ public final class SignatureManager {
this.setCacheRegister(cacheRegister);
this.setTrafficOpsUtils(trafficOpsUtils);
this.setZoneManager(zoneManager);
+ setRRSIGCacheEnabled(cacheRegister.getConfig());
initKeyMap();
}
@@ -73,6 +82,19 @@ public final class SignatureManager {
}
}
+ private void setRRSIGCacheEnabled(final JsonNode config) {
+ RRSIGCacheEnabled = JsonUtils.optBoolean(config,
TrafficRouter.DNSSEC_RRSIG_CACHE_ENABLED, false);
+ if (!RRSIGCacheEnabled) {
+ synchronized (RRSIGCacheLock) {
+ RRSIGCache = new ConcurrentHashMap<>();
+ }
+ }
+ }
+
+ private boolean isRRSIGCacheEnabled() {
+ return this.RRSIGCacheEnabled;
+ }
+
private void initKeyMap() {
synchronized(SignatureManager.class) {
final JsonNode config = cacheRegister.getConfig();
@@ -154,6 +176,7 @@ public final class SignatureManager {
}
}
}
+ cleanRRSIGCache(keyMap,
newKeyMap);
if (keyMap == null) {
// initial startup
@@ -177,16 +200,44 @@ public final class SignatureManager {
};
}
- private boolean hasNewKeys(final Map<String, List<DnsSecKeyPair>>
keyMap, final Map<String, List<DnsSecKeyPair>> newKeyMap) {
- for (final String key : newKeyMap.keySet()) {
- if (!keyMap.containsKey(key)) {
- return true;
+ private void cleanRRSIGCache(final Map<String, List<DnsSecKeyPair>>
oldKeyMap, final Map<String, List<DnsSecKeyPair>> newKeyMap) {
+ synchronized (RRSIGCacheLock) {
+ if (RRSIGCache.isEmpty() || oldKeyMap == null ||
getKeyDifferences(oldKeyMap, newKeyMap).isEmpty()) {
+ return;
+ }
+ final int oldKeySize = RRSIGCache.size();
+ final int oldRRSIGSize =
RRSIGCache.values().stream().map(Map::size).reduce(0, Integer::sum);
+ final long now = new Date().getTime();
+ final ConcurrentMap<RRSIGCacheKey,
ConcurrentMap<RRsetKey, RRSIGRecord>> newRRSIGCache = new ConcurrentHashMap<>();
+ newKeyMap.forEach((name, keyPairs) ->
keyPairs.forEach(keypair -> {
+ final RRSIGCacheKey cacheKey = new
RRSIGCacheKey(keypair.getPrivate().getEncoded(),
keypair.getDNSKEYRecord().getAlgorithm());
+ final ConcurrentMap<RRsetKey, RRSIGRecord>
cacheValue = RRSIGCache.get(cacheKey);
+ if (cacheValue != null) {
+ cacheValue.entrySet().removeIf(e ->
e.getValue().getExpire().getTime() <= now);
+ newRRSIGCache.put(cacheKey, cacheValue);
+ }
+ }));
+ RRSIGCache = newRRSIGCache;
+ final int keySize = RRSIGCache.size();
+ final int RRSIGSize =
RRSIGCache.values().stream().map(Map::size).reduce(0, Integer::sum);
+ LOGGER.info("DNSSEC keys were changed or removed so
RRSIG cache was cleaned. Old key size: " + oldKeySize +
+ ", new key size: " + keySize + ", old
RRSIG size: " + oldRRSIGSize + ", new RRSIG size: " + RRSIGSize);
+ }
+ }
+
+ // return the key names from newKeyMap that are different or missing
from oldKeyMap
+ private Set<String> getKeyDifferences(final Map<String,
List<DnsSecKeyPair>> newKeyMap, final Map<String, List<DnsSecKeyPair>>
oldKeyMap) {
+ final Set<String> newKeyNames = new HashSet<>();
+ for (final String newName : newKeyMap.keySet()) {
+ if (!oldKeyMap.containsKey(newName)) {
+ newKeyNames.add(newName);
+ continue;
}
- for (final DnsSecKeyPair newKeyPair :
newKeyMap.get(key)) {
+ for (final DnsSecKeyPair newKeyPair :
newKeyMap.get(newName)) {
boolean matched = false;
- for (final DnsSecKeyPair keyPair :
keyMap.get(key)) {
+ for (final DnsSecKeyPair keyPair :
oldKeyMap.get(newName)) {
if (newKeyPair.equals(keyPair)) {
matched = true;
break;
@@ -194,12 +245,20 @@ public final class SignatureManager {
}
if (!matched) {
- LOGGER.info("Found new or changed key
for " + newKeyPair.getName());
- return true; // has a new key because
we didn't find a match
+ newKeyNames.add(newKeyPair.getName());
+ break;
}
}
}
+ return newKeyNames;
+ }
+ private boolean hasNewKeys(final Map<String, List<DnsSecKeyPair>>
oldKeyMap, final Map<String, List<DnsSecKeyPair>> newKeyMap) {
+ final Set<String> newOrChangedKeyNames =
getKeyDifferences(newKeyMap, oldKeyMap);
+ if (!newOrChangedKeyNames.isEmpty()) {
+ newOrChangedKeyNames.forEach(name -> LOGGER.info("Found
new or changed key for " + name));
+ return true;
+ }
return false;
}
@@ -407,7 +466,7 @@ public final class SignatureManager {
sb.append(" is ");
sb.append(zoneKey.getTimestampDate());
sb.append("; expires ");
- sb.append(zoneKey.getSignatureExpiration().getTime());
+ sb.append(zoneKey.getMinimumSignatureExpiration().getTime());
if (needsRefresh) {
sb.append("; refresh needed");
@@ -447,9 +506,10 @@ public final class SignatureManager {
final ZoneSigner zoneSigner = new
ZoneSignerImpl();
- signedRecords = zoneSigner.signZone(name,
records, kskPairs, zskPairs, start.getTime(), signatureExpiration.getTime(),
true, DSRecord.SHA256_DIGEST_ID);
+ signedRecords = zoneSigner.signZone(records,
kskPairs, zskPairs, start.getTime(),
+ signatureExpiration.getTime(),
isRRSIGCacheEnabled() ? RRSIGCache : null);
-
zoneKey.setSignatureExpiration(signatureExpiration);
+
zoneKey.setMinimumSignatureExpiration(signedRecords, signatureExpiration);
zoneKey.setKSKExpiration(kskExpiration);
zoneKey.setZSKExpiration(zskExpiration);
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignedZoneKey.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignedZoneKey.java
index 61fd439..9918afd 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignedZoneKey.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/SignedZoneKey.java
@@ -17,12 +17,17 @@ package
com.comcast.cdn.traffic_control.traffic_router.core.dns;
import java.util.Calendar;
import java.util.List;
+import java.util.OptionalLong;
+import org.apache.log4j.Logger;
import org.xbill.DNS.Name;
+import org.xbill.DNS.RRSIGRecord;
import org.xbill.DNS.Record;
public class SignedZoneKey extends ZoneKey {
- private Calendar signatureExpiration;
+ private static final Logger LOGGER =
Logger.getLogger(SignedZoneKey.class);
+
+ private Calendar minimumSignatureExpiration;
private Calendar kskExpiration;
private Calendar zskExpiration;
@@ -31,16 +36,27 @@ public class SignedZoneKey extends ZoneKey {
super(name, records);
}
- public Calendar getSignatureExpiration() {
- return signatureExpiration;
+ public Calendar getMinimumSignatureExpiration() {
+ return minimumSignatureExpiration;
}
- public void setSignatureExpiration(final Calendar signatureExpiration) {
- this.signatureExpiration = signatureExpiration;
+ public void setMinimumSignatureExpiration(final List<Record>
signedRecords, final Calendar defaultExpiration) {
+ final OptionalLong minSignatureExpiration =
signedRecords.stream()
+ .filter(r -> r instanceof RRSIGRecord)
+ .mapToLong(r -> ((RRSIGRecord)
r).getExpire().getTime())
+ .min();
+ if (!minSignatureExpiration.isPresent()) {
+ LOGGER.error("unable to calculate minimum signature
expiration: no RRSIG records given");
+ this.minimumSignatureExpiration = defaultExpiration;
+ return;
+ }
+ final Calendar tmp = Calendar.getInstance();
+ tmp.setTimeInMillis(minSignatureExpiration.getAsLong());
+ this.minimumSignatureExpiration = tmp;
}
public long getSignatureDuration() {
- return this.signatureExpiration.getTimeInMillis() -
getTimestamp();
+ return this.minimumSignatureExpiration.getTimeInMillis() -
getTimestamp();
}
public long getRefreshHorizon() {
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
index e53845e..9b82493 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneManager.java
@@ -212,6 +212,11 @@ public class ZoneManager extends Resolver {
ZoneManager.dynamicZoneCache = dzc;
ZoneManager.zoneCache = zc;
+ final long oldZCSize = tzc == null ? 0 :
tzc.size();
+ final long oldDCZSize = tzc == null ? 0 :
tdzc.size();
+ LOGGER.info("old static zone cache size: " +
oldZCSize + ", new static zone cache size: " + zc.size() +
+ ", old dynamic zone cache size:
" + oldDCZSize + ", new dynamic zone cache size: " + dzc.size());
+
ZoneManager.domainsToZoneKeys =
newDomainsToZoneKeys;
if (tze != null) {
@@ -278,6 +283,7 @@ public class ZoneManager extends Resolver {
private static Runnable getMaintenanceRunnable(final
LoadingCache<ZoneKey, Zone> cache, final ZoneCacheType type, final int
refreshInterval) {
return new Runnable() {
public void run() {
+ LOGGER.info("starting maintenance on " +
type.toString() + " zone cache: " + Integer.toHexString(cache.hashCode()) + ".
Current size: " + cache.size());
cache.cleanUp();
for (final ZoneKey zoneKey :
cache.asMap().keySet()) {
@@ -289,6 +295,7 @@ public class ZoneManager extends Resolver {
LOGGER.fatal("RuntimeException
caught on " + zoneKey.getClass().getSimpleName() + " for " + zoneKey.getName(),
ex);
}
}
+ LOGGER.info("completed maintenance on " +
type.toString() + " zone cache: " + Integer.toHexString(cache.hashCode()));
}
};
}
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSigner.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSigner.java
index c2fa4d9..9d17024 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSigner.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSigner.java
@@ -17,16 +17,18 @@ package
com.comcast.cdn.traffic_control.traffic_router.core.dns;
import org.xbill.DNS.DNSKEYRecord;
import org.xbill.DNS.DSRecord;
-import org.xbill.DNS.Name;
import org.xbill.DNS.Record;
+import org.xbill.DNS.RRSIGRecord;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.Date;
import java.util.List;
+import java.util.concurrent.ConcurrentMap;
public interface ZoneSigner {
- List<Record> signZone(Name name, List<Record> records,
List<DnsSecKeyPair> kskPairs, List<DnsSecKeyPair> zskPairs,
- Date inception, Date expiration, boolean
fullySignKeySet, int digestId) throws IOException, GeneralSecurityException;
+ List<Record> signZone(List<Record> records, List<DnsSecKeyPair>
kskPairs,
+ List<DnsSecKeyPair> zskPairs,
Date inception, Date expiration,
+ ConcurrentMap<RRSIGCacheKey,
ConcurrentMap<RRsetKey, RRSIGRecord>> RRSIGCache) throws IOException,
GeneralSecurityException;
DSRecord calculateDSRecord(DNSKEYRecord dnskeyRecord, int digestId,
long ttl);
}
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImpl.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImpl.java
index b23e2a9..743c88b 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImpl.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImpl.java
@@ -37,6 +37,8 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
@@ -57,21 +59,53 @@ public class ZoneSignerImpl implements ZoneSigner {
return StreamSupport.stream(iterable.spliterator(), false);
}
- private RRset signRRset(final RRset rrSet, final List<DnsSecKeyPair>
kskPairs, final List<DnsSecKeyPair> zskPairs, final Date inception, final Date
expiration) {
+ private RRSIGRecord sign(final RRset rrset, final DNSKEYRecord
dnskeyRecord, final PrivateKey privateKey, final Date inception, final Date
expiration) {
+ try {
+ return DNSSEC.sign(rrset, dnskeyRecord, privateKey,
inception, expiration);
+ } catch (DNSSEC.DNSSECException e) {
+ final String message = String.format("Failed to sign
Resource Record Set for %s %d %d %d : %s",
+ dnskeyRecord.getName(),
dnskeyRecord.getDClass(), dnskeyRecord.getType(), dnskeyRecord.getTTL(),
e.getMessage());
+ LOGGER.error(message, e);
+ return null;
+ }
+ }
+
+ private boolean isSignatureAlmostExpired(final Date inception, final
Date expiration, final Date now) {
+ // now is over halfway through validity period
+ return now.getTime() > inception.getTime() +
((expiration.getTime() - inception.getTime())/2);
+ }
+
+ private RRset signRRset(final RRset rrSet, final List<DnsSecKeyPair>
kskPairs, final List<DnsSecKeyPair> zskPairs,
+ final Date inception,
final Date expiration,
+ final
ConcurrentMap<RRSIGCacheKey, ConcurrentMap<RRsetKey, RRSIGRecord>> RRSIGCache) {
final List<RRSIGRecord> signatures = new ArrayList<>();
final List<DnsSecKeyPair> pairs = rrSet.getType() ==
Type.DNSKEY ? kskPairs : zskPairs;
+ final Date now = new Date();
pairs.forEach(pair -> {
final DNSKEYRecord dnskeyRecord =
pair.getDNSKEYRecord();
final PrivateKey privateKey = pair.getPrivate();
+ RRSIGRecord signature = null;
try {
- signatures.add(DNSSEC.sign(rrSet, dnskeyRecord,
privateKey, inception, expiration));
+ if (RRSIGCache == null) {
+ signature = sign(rrSet, dnskeyRecord,
privateKey, inception, expiration);
+ } else {
+ final ConcurrentMap<RRsetKey,
RRSIGRecord> sigMap = RRSIGCache.computeIfAbsent(new
RRSIGCacheKey(privateKey.getEncoded(), dnskeyRecord.getAlgorithm()),
rrsigCacheKey -> new ConcurrentHashMap<>());
+ signature = sigMap.computeIfAbsent(new
RRsetKey(rrSet), k -> sign(rrSet, dnskeyRecord, privateKey, inception,
expiration));
+
+ if (signature != null &&
isSignatureAlmostExpired(signature.getTimeSigned(), signature.getExpire(),
now)) {
+ signature = sigMap.compute(new
RRsetKey(rrSet), (k, v) -> sign(rrSet, dnskeyRecord, privateKey, inception,
expiration));
+ }
+ }
} catch (Exception e) {
final String message = String.format("Failed to
sign Resource Record Set for %s %d %d %d : %s",
dnskeyRecord.getName(),
dnskeyRecord.getDClass(), dnskeyRecord.getType(), dnskeyRecord.getTTL(),
e.getMessage());
LOGGER.error(message, e);
}
+ if (signature != null) {
+ signatures.add(signature);
+ }
});
final RRset signedRRset = new RRset();
@@ -124,9 +158,9 @@ public class ZoneSignerImpl implements ZoneSigner {
@Override
- public List<Record> signZone(final Name name, final List<Record>
records, final List<DnsSecKeyPair> kskPairs, final List<DnsSecKeyPair> zskPairs,
- final Date inception, final Date expiration, final boolean
fullySignKeySet, final int digestId) throws IOException,
GeneralSecurityException {
- LOGGER.info("Signing records, name for first record is " +
records.get(0).getName());
+ public List<Record> signZone(final List<Record> records, final
List<DnsSecKeyPair> kskPairs,
+ final
List<DnsSecKeyPair> zskPairs, final Date inception, final Date expiration,
+ final
ConcurrentMap<RRSIGCacheKey, ConcurrentMap<RRsetKey, RRSIGRecord>> RRSIGCache)
throws IOException, GeneralSecurityException {
final List<NSECRecord> nsecRecords = createNsecRecords(records);
records.addAll(nsecRecords);
@@ -158,7 +192,7 @@ public class ZoneSignerImpl implements ZoneSigner {
final List<RRset> rrSets = new RRSetsBuilder().build(records);
final List<RRset> signedRrSets = rrSets.stream()
- .map(rRset -> signRRset(rRset, kskPairs, zskPairs,
inception, expiration))
+ .map(rRset -> signRRset(rRset, kskPairs, zskPairs,
inception, expiration, RRSIGCache))
.sorted((rRset1, rRset2) ->
rRset1.getName().compareTo(rRset2.getName()))
.collect(toList());
diff --git
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
index 7a45d1f..592272a 100644
---
a/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
+++
b/traffic_router/core/src/main/java/com/comcast/cdn/traffic_control/traffic_router/core/router/TrafficRouter.java
@@ -104,6 +104,7 @@ public class TrafficRouter {
public static final String CLIENT_STEERING_DIVERSITY =
"client.steering.forced.diversity";
public static final String DNSSEC_ENABLED = "dnssec.enabled";
public static final String DNSSEC_ZONE_DIFFING =
"dnssec.zone.diffing.enabled";
+ public static final String DNSSEC_RRSIG_CACHE_ENABLED =
"dnssec.rrsig.cache.enabled";
private static final long DEFAULT_EDGE_NS_TTL = 3600;
private static final int DEFAULT_EDGE_TR_LIMIT = 4;
diff --git
a/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImplTest.java
b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImplTest.java
new file mode 100644
index 0000000..f5ceb46
--- /dev/null
+++
b/traffic_router/core/src/test/java/com/comcast/cdn/traffic_control/traffic_router/core/dns/ZoneSignerImplTest.java
@@ -0,0 +1,160 @@
+/*
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.comcast.cdn.traffic_control.traffic_router.core.dns;
+
+import java.net.InetAddress;
+import java.security.PrivateKey;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Date;
+import java.util.List;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.notNullValue;
+import static org.mockito.Matchers.any;
+import static org.mockito.Matchers.argThat;
+import static org.mockito.Matchers.eq;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.mockito.ArgumentMatcher;
+import org.powermock.api.mockito.PowerMockito;
+import org.powermock.core.classloader.annotations.PrepareForTest;
+import org.powermock.modules.junit4.PowerMockRunner;
+import org.xbill.DNS.ARecord;
+import org.xbill.DNS.DClass;
+import org.xbill.DNS.DNSKEYRecord;
+import org.xbill.DNS.Name;
+import org.xbill.DNS.RRSIGRecord;
+import org.xbill.DNS.RRset;
+import org.xbill.DNS.Record;
+import org.xbill.DNS.Type;
+
+@RunWith(PowerMockRunner.class)
+@PrepareForTest(ZoneSignerImpl.class)
+public class ZoneSignerImplTest {
+
+ static class IsRRsetTypeA extends ArgumentMatcher<RRset> {
+ public boolean matches(Object rrset) {
+ return ((RRset) rrset).getType() == Type.A;
+ }
+ }
+
+ static class IsRRsetTypeNSEC extends ArgumentMatcher<RRset> {
+ public boolean matches(Object rrset) {
+ return ((RRset) rrset).getType() == Type.NSEC;
+ }
+ }
+
+ @Test
+ public void signZoneWithRRSIGCacheTest() throws Exception {
+ ZoneSignerImpl zoneSigner = PowerMockito.spy(new ZoneSignerImpl());
+ List<Record> records = new ArrayList<>();
+ Record ARecord1 = new ARecord(new Name("foo.example.com."), DClass.IN,
60, InetAddress.getByName("1.2.3.4"));
+ Record ARecord2 = new ARecord(new Name("foo.example.com."), DClass.IN,
60, InetAddress.getByName("1.2.3.5"));
+ Record ARecord3 = new ARecord(new Name("foo.example.com."), DClass.IN,
60, InetAddress.getByName("1.2.3.6"));
+ Record ARecord4 = new ARecord(new Name("foo.example.com."), DClass.IN,
60, InetAddress.getByName("1.2.3.7"));
+ DnsSecKeyPair zskPair = mock(DnsSecKeyPairImpl.class);
+ DNSKEYRecord zskDnskey = mock(DNSKEYRecord.class);
+ when(zskPair.getDNSKEYRecord()).thenReturn(zskDnskey);
+ PrivateKey zskKey = mock(PrivateKey.class);
+ when(zskPair.getPrivate()).thenReturn(zskKey);
+ when(zskKey.getEncoded()).thenReturn(new byte[] {1});
+ when(zskDnskey.getAlgorithm()).thenReturn(1);
+ List<DnsSecKeyPair> kskPairs = new ArrayList<>();
+ List<DnsSecKeyPair> zskPairs = Collections.singletonList(zskPair);
+
+ Date inception = new Date();
+ Date expire = Date.from(inception.toInstant().plusSeconds(100000));
+ RRSIGRecord aRRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.A, 1, 60, inception, expire, 1,
new Name("example.com."), new byte[]{1});
+ RRSIGRecord nsecRRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.NSEC, 1, 60, inception, expire,
1, new Name("example.com."), new byte[]{2});
+ PowerMockito.doReturn(aRRSigRecord).when(zoneSigner, "sign",
argThat(new IsRRsetTypeA()), any(DNSKEYRecord.class), any(PrivateKey.class),
eq(inception), eq(expire));
+ PowerMockito.doReturn(nsecRRSigRecord).when(zoneSigner, "sign",
argThat(new IsRRsetTypeNSEC()), any(DNSKEYRecord.class), any(PrivateKey.class),
eq(inception), eq(expire));
+
+ Date newInception = Date.from(inception.toInstant().plusSeconds(100));
+ Date newExpire =
Date.from(newInception.toInstant().plusSeconds(100000));
+ RRSIGRecord newARRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.A, 1, 60, newInception,
newExpire, 1, new Name("example.com."), new byte[]{3});
+ RRSIGRecord newNSECRRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.NSEC, 1, 60, newInception,
newExpire, 1, new Name("example.com."), new byte[]{4});
+ PowerMockito.doReturn(newARRSigRecord).when(zoneSigner, "sign",
argThat(new IsRRsetTypeA()), any(DNSKEYRecord.class), any(PrivateKey.class),
eq(newInception), eq(newExpire));
+ PowerMockito.doReturn(newNSECRRSigRecord).when(zoneSigner, "sign",
argThat(new IsRRsetTypeNSEC()), any(DNSKEYRecord.class), any(PrivateKey.class),
eq(newInception), eq(newExpire));
+
+ Date expiresSoonInception =
Date.from(inception.toInstant().minusSeconds(100));
+ Date expiresSoonExpire =
Date.from(inception.toInstant().plusSeconds(50));
+ RRSIGRecord expiresSoonARRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.A, 1, 60, expiresSoonInception,
expiresSoonExpire, 1, new Name("example.com."), new byte[]{5});
+ RRSIGRecord expiresSoonNSECRRSigRecord = new RRSIGRecord(new
Name("foo.example.com."), DClass.IN, 60, Type.NSEC, 1, 60,
expiresSoonInception, expiresSoonExpire, 1, new Name("example.com."), new
byte[]{6});
+ PowerMockito.doReturn(expiresSoonARRSigRecord).when(zoneSigner,
"sign", argThat(new IsRRsetTypeA()), any(DNSKEYRecord.class),
any(PrivateKey.class), eq(expiresSoonInception), eq(expiresSoonExpire));
+ PowerMockito.doReturn(expiresSoonNSECRRSigRecord).when(zoneSigner,
"sign", argThat(new IsRRsetTypeNSEC()), any(DNSKEYRecord.class),
any(PrivateKey.class), eq(expiresSoonInception), eq(expiresSoonExpire));
+
+ ConcurrentMap<RRSIGCacheKey, ConcurrentMap<RRsetKey, RRSIGRecord>>
RRSIGCache = new ConcurrentHashMap<>();
+
+ records.add(ARecord1);
+ records.add(ARecord2);
+ List<Record> signedRecords = zoneSigner.signZone(records, kskPairs,
zskPairs, inception, expire, RRSIGCache);
+ RRSIGRecord ret = (RRSIGRecord) signedRecords.stream().filter(r -> r
instanceof RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(aRRSigRecord));
+
+ // re-signing the same RRset with new timestamps should reuse the
cached RRSIG record
+ records.clear();
+ records.add(ARecord1);
+ records.add(ARecord2);
+ signedRecords = zoneSigner.signZone(records, kskPairs, zskPairs,
newInception, newExpire, RRSIGCache);
+ ret = (RRSIGRecord) signedRecords.stream().filter(r -> r instanceof
RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(aRRSigRecord));
+
+ // changed RRset should be re-signed
+ records.clear();
+ records.add(ARecord1);
+ records.add(ARecord2);
+ records.add(ARecord3);
+ records.add(ARecord4);
+ signedRecords = zoneSigner.signZone(records, kskPairs, zskPairs,
newInception, newExpire, RRSIGCache);
+ ret = (RRSIGRecord) signedRecords.stream().filter(r -> r instanceof
RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(newARRSigRecord));
+
+ // re-signing 1st RRset again should reuse the cached RRSIG record
+ records.clear();
+ records.add(ARecord1);
+ records.add(ARecord2);
+ signedRecords = zoneSigner.signZone(records, kskPairs, zskPairs,
newInception, newExpire, RRSIGCache);
+ ret = (RRSIGRecord) signedRecords.stream().filter(r -> r instanceof
RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(aRRSigRecord));
+
+ // re-signing RRset that has a cached RRSIG record that is close to
expiring should be re-signed
+ records.clear();
+ records.add(ARecord3);
+ records.add(ARecord4);
+ signedRecords = zoneSigner.signZone(records, kskPairs, zskPairs,
expiresSoonInception, expiresSoonExpire, RRSIGCache);
+ ret = (RRSIGRecord) signedRecords.stream().filter(r -> r instanceof
RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(expiresSoonARRSigRecord));
+ records.clear();
+ records.add(ARecord3);
+ records.add(ARecord4);
+ signedRecords = zoneSigner.signZone(records, kskPairs, zskPairs,
newInception, newExpire, RRSIGCache);
+ ret = (RRSIGRecord) signedRecords.stream().filter(r -> r instanceof
RRSIGRecord && ((RRSIGRecord) r).getTypeCovered() ==
Type.A).findFirst().orElse(null);
+ assertThat(ret, notNullValue());
+ assertThat(ret, equalTo(newARRSigRecord));
+ }
+}
