This is an automated email from the ASF dual-hosted git repository. cmcfarlen pushed a commit to branch 10.2.x in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 7ee76e3b2dda580d11d01dbcdfd86d9897666d48 Author: Leif Hedstrom <[email protected]> AuthorDate: Fri Mar 13 23:17:03 2026 -0700 Cache volumes: RAM cache settings and remap option (#12717) * First cut at new volume configs for RAM cache * Added support for @volume= in remap.config This gets a bit complicated because we allow for ATS to start up with remap before the cache is properly started. * Adds a default volume list to records.yaml * Adds some autests * Address Chris' review comment * Address Copilot review comments - foreach_mapping: remove const (Trie only has const_iterator, so const_cast is still needed but function is now semantically correct) - Cache.cc: document default_volumes_host_rec as intentional singleton - CacheHosting.cc: restore tmp++ (tmp=line_end broke multi-param lines), remove unreachable null check after new, use char array for "volume" literal instead of const_cast - ReverseProxy.cc: add acquire()/release() around rewrite_table access in init_remap_volume_host_records, document startup-only constraint - Test files: remove trailing commas from ATSReplayTest calls * Address review comments from bryancall - CacheHosting.cc: restrict suffix skip loops to uppercase KMGT to match ink_atoi64's uppercase-only handling - RemapConfig.cc: use swoc::TextView to validate @volume= segments (rejects empty, zero, out-of-range, trailing comma) - ReverseProxy.h/cc, HttpSM.cc: use std::atomic<UrlRewrite*> for rewrite_table to make atomic access explicit - cache_volume_features.replay.yaml: remove misleading @volume=99 remap entry and its corresponding session test * Try to prove the code correctly parses configs (cherry picked from commit 8ea9548fbe7619f5aebf7346cb26fffd4c471c61) --- configs/volume.config.default | 33 ++++- doc/admin-guide/files/records.yaml.en.rst | 38 ++++++ doc/admin-guide/files/remap.config.en.rst | 48 +++++++ doc/admin-guide/files/volume.config.en.rst | 86 ++++++++++-- include/iocore/cache/Cache.h | 5 +- include/proxy/ReverseProxy.h | 7 +- include/proxy/http/HttpTransact.h | 8 +- include/proxy/http/remap/RemapConfig.h | 1 + include/proxy/http/remap/UrlMapping.h | 37 +++++- include/proxy/http/remap/UrlMappingPathIndex.h | 13 ++ src/iocore/cache/Cache.cc | 102 ++++++++++----- src/iocore/cache/CacheHosting.cc | 120 +++++++++++++++-- src/iocore/cache/CacheProcessor.cc | 121 +++++++++++++---- src/iocore/cache/CacheVC.cc | 12 +- src/iocore/cache/P_CacheHosting.h | 49 +++++-- src/iocore/cache/P_CacheInternal.h | 11 +- src/iocore/cache/Stripe.h | 2 + src/proxy/ReverseProxy.cc | 76 +++++++++-- src/proxy/http/HttpCacheSM.cc | 20 ++- src/proxy/http/HttpSM.cc | 2 +- src/proxy/http/remap/RemapConfig.cc | 76 ++++++++++- src/proxy/http/remap/RemapProcessor.cc | 3 + src/proxy/http/remap/UrlMapping.cc | 33 +++++ src/records/RecordsConfig.cc | 2 + src/traffic_server/traffic_server.cc | 4 + tests/gold_tests/autest-site/ats_replay.test.ext | 12 ++ .../cache/cache_volume_defaults.replay.yaml | 144 +++++++++++++++++++++ .../gold_tests/cache/cache_volume_defaults.test.py | 24 ++++ .../cache/cache_volume_features.replay.yaml | 141 ++++++++++++++++++++ .../gold_tests/cache/cache_volume_features.test.py | 25 ++++ 30 files changed, 1133 insertions(+), 122 deletions(-) diff --git a/configs/volume.config.default b/configs/volume.config.default index c1aa03644a..6eafd85aed 100644 --- a/configs/volume.config.default +++ b/configs/volume.config.default @@ -29,6 +29,37 @@ # disk (assuming each disk has enough free space available). # # To create one volume of size 10% of the total cache space and -# another 1 Gig volume, +# another 1 Gig volume, # volume=1 scheme=http size=10% # volume=2 scheme=http size=1024 +# +# Additional optional parameters: +# +# ramcache=true/false +# Enable or disable RAM cache for this volume (default: true) +# +# ram_cache_size=<size> +# Allocate a dedicated RAM cache pool for this volume (e.g., 512M, 2G) +# This amount is automatically subtracted from the global ram_cache.size +# setting, with the remainder shared among other volumes. +# +# ram_cache_cutoff=<size> +# Override the global ram_cache_cutoff for this volume (e.g., 64K, 1M) +# Objects larger than this will not be stored in RAM cache. +# +# avg_obj_size=<size> +# Override the global min_average_object_size for this volume +# +# fragment_size=<size> +# Override the global target_fragment_size for this volume (max: 4MB) +# +# Advanced RAM cache configuration examples: +# +# Example 1: Volume with dedicated 2GB RAM cache +# volume=1 scheme=http size=40% ram_cache_size=2G +# +# Example 2: Small objects with custom cutoff and dedicated RAM +# volume=2 scheme=http size=20% ram_cache_size=512M ram_cache_cutoff=64K +# +# Example 3: Large media with higher cutoff (shares remaining RAM pool) +# volume=3 scheme=http size=40% ram_cache_cutoff=1M diff --git a/doc/admin-guide/files/records.yaml.en.rst b/doc/admin-guide/files/records.yaml.en.rst index 74aba11c22..50379396fd 100644 --- a/doc/admin-guide/files/records.yaml.en.rst +++ b/doc/admin-guide/files/records.yaml.en.rst @@ -2677,6 +2677,34 @@ Cache Control used in determining the number of :term:`directory buckets <directory bucket>` to allocate for the in-memory cache directory. +.. ts:cv:: CONFIG proxy.config.cache.default_volumes STRING "" + + Specifies a comma-separated list of cache volume numbers to use as the default + for cache stripe selection when no more specific volume configuration applies. + For example, ``"1,2"`` would use volumes 1 and 2 as the default. + + The volume selection priority order is: + + 1. ``@volume=`` directive in :file:`remap.config` (highest priority) + 2. Hostname matching in :file:`hosting.config` + 3. ``proxy.config.cache.default_volumes`` (if non-empty) + 4. All available cache volumes (lowest priority) + + An empty string (the default) disables this feature, causing |TS| to fall + back directly to using all available volumes when no other configuration + matches. + + This is useful for scenarios where you want to restrict default caching to + specific volumes without configuring hostname patterns in :file:`hosting.config`. + For example, you might want to reserve certain volumes for specific remap rules + while having a different set of default volumes for all other traffic. + +.. topic:: Example + + Assign volumes 1 and 2 as defaults for general traffic :: + + CONFIG proxy.config.cache.default_volumes STRING "1,2" + .. ts:cv:: CONFIG proxy.config.cache.permit.pinning INT 0 :reloadable: @@ -2817,6 +2845,11 @@ RAM Cache Alternatively, it can be set to a fixed value such as **20GB** (21474836480) + This global setting can be overridden on a per-volume basis using the + ``ram_cache_size`` parameter in :file:`volume.config`. Per-volume + allocations are subtracted from the total RAM cache size before + distributing the remainder among volumes without explicit settings. + .. ts:cv:: CONFIG proxy.config.cache.ram_cache_cutoff INT 4194304 Objects greater than this size will not be kept in the RAM cache. @@ -2824,6 +2857,11 @@ RAM Cache in memory in order to improve performance. **4MB** (4194304) + This global setting can be overridden on a per-volume basis using the + ``ram_cache_cutoff`` parameter in :file:`volume.config`. When set, + the per-volume cutoff takes precedence over this global setting for + that specific volume. + .. ts:cv:: CONFIG proxy.config.cache.ram_cache.algorithm INT 1 Two distinct RAM caches are supported, the default (1) being the simpler diff --git a/doc/admin-guide/files/remap.config.en.rst b/doc/admin-guide/files/remap.config.en.rst index 7578750216..7301282373 100644 --- a/doc/admin-guide/files/remap.config.en.rst +++ b/doc/admin-guide/files/remap.config.en.rst @@ -455,6 +455,54 @@ will pass "1" and "2" to plugin1.so and "3" to plugin2.so. This will pass "1" and "2" to plugin1.so and "3" to plugin2.so +.. _remap-config-cache-volume-selection: + +Cache Volume Selection +====================== + +The ``@volume`` directive allows you to override the default cache volume selection +for specific remap rules, bypassing the hostname-based volume selection configured in +:file:`hosting.config`. This provides fine-grained control over which cache volumes +are used for different URL patterns. + +Format +------ + +:: + + @volume=<volume_list> + +Where ``<volume_list>`` can be either: + +- A single volume number: ``@volume=4`` +- Multiple comma-separated volume numbers: ``@volume=3,4,5`` + +Volume numbers must be between 1 and 255 (volume 0 is reserved and not usable). +All specified volumes must be defined in :file:`volume.config`. + +Examples +-------- + +:: + + # Single volume for API requests (backward compatibility) + map https://api.example.com/ https://api-origin.example.com/ @volume=4 + + # Multiple volumes for load distribution across SSD volumes + map https://cdn.example.com/ https://cdn-origin.example.com/ @volume=2,3,4 + + # Single high-performance volume for critical services + map https://checkout.example.com/ https://checkout-origin.example.com/ @volume=1 + + # Everything else gets the default volume allocations (hosting.config rules) + map https://www.example.com/ https://origin.example.com/ + +.. note:: + + When using ``@volume``, ensure that the target volumes have appropriate disk space and + performance characteristics for the expected traffic patterns. For multiple volumes, + consider the combined capacity and performance of all specified volumes. + .. _remap-config-named-filters: NextHop Selection Strategies diff --git a/doc/admin-guide/files/volume.config.en.rst b/doc/admin-guide/files/volume.config.en.rst index 70ec2725af..f26a122eef 100644 --- a/doc/admin-guide/files/volume.config.en.rst +++ b/doc/admin-guide/files/volume.config.en.rst @@ -73,20 +73,67 @@ Optional directory entry sizing You can also add an option ``avg_obj_size=<size>`` to the volume configuration line. This overrides the global :ts:cv:`proxy.config.cache.min_average_object_size` -configuration for this volume. This is useful if you have a volume that is dedicated -for say very small objects, and you need a lot of directory entries to store them. +configuration for this volume. The size supports multipliers (K, M, G, T) for +convenience (e.g., ``avg_obj_size=64K`` or ``avg_obj_size=1M``). This is useful +if you have a volume that is dedicated for say very small objects, and you need +a lot of directory entries to store them. Optional fragment size setting ------------------------------ You can also add an option ``fragment_size=<size>`` to the volume configuration line. This overrides the global :ts:cv:`proxy.config.cache.target_fragment_size` -configuration for this volume. This allows for a smaller, or larger, fragment size -for a particular volume. This may be useful together with ``avg_obj_size`` as well, -since a larger fragment size could reduce the number of directory entries needed -for a large object. - -Note that this setting has a maximmum value of 4MB. +configuration for this volume. The size supports multipliers (K, M, G, T) for +convenience (e.g., ``fragment_size=512K`` or ``fragment_size=2M``). This allows +for a smaller, or larger, fragment size for a particular volume. This may be +useful together with ``avg_obj_size`` as well, since a larger fragment size could +reduce the number of directory entries needed for a large object. + +Note that this setting has a maximum value of 4MB. + +Optional RAM cache size allocation +----------------------------------- + +You can add an option ``ram_cache_size=<size>`` to the volume configuration line +to allocate a dedicated RAM cache pool for this volume. The size supports +multipliers (K, M, G, T) for convenience (e.g., ``ram_cache_size=512M`` or +``ram_cache_size=2G``). Setting ``ram_cache_size=0`` disables the RAM cache +for this volume, which is equivalent to ``ramcache=false``. + +When ``ram_cache_size`` is specified for a volume, that amount is **automatically +subtracted** from the global :ts:cv:`proxy.config.cache.ram_cache.size` setting, +and the remainder is shared among volumes without private allocations. This ensures +total RAM cache usage never exceeds the configured global limit. + +For example, if the global RAM cache size is 4GB and you allocate 1GB to volume 1 +and 512MB to volume 2, the remaining 2.5GB will be distributed among other volumes +using the normal proportional allocation based on disk space. + +**Important notes:** + +* If the sum of all ``ram_cache_size`` allocations exceeds the global RAM cache size, + Traffic Server will fail to start with a fatal error. Increase + :ts:cv:`proxy.config.cache.ram_cache.size` or reduce the per-volume allocations. +* If ``ramcache=false`` is set alongside ``ram_cache_size``, the ``ram_cache_size`` + is ignored (with a warning) since the RAM cache is disabled for that volume. +* This setting only takes effect when :ts:cv:`proxy.config.cache.ram_cache.size` + is set to a positive value (not ``-1`` for automatic sizing). + +Optional RAM cache cutoff override +----------------------------------- + +You can add an option ``ram_cache_cutoff=<size>`` to the volume configuration line +to override the global :ts:cv:`proxy.config.cache.ram_cache_cutoff` setting for +this specific volume. The size supports multipliers (K, M, G, T) for convenience +(e.g., ``ram_cache_cutoff=64K`` or ``ram_cache_cutoff=1M``). + +This cutoff determines the maximum object size that will be stored in the RAM cache. +Objects larger than this size will only be stored on disk. Setting different cutoffs +per volume allows you to: + +* Use larger cutoffs for volumes serving frequently accessed large objects +* Use smaller cutoffs for volumes with many small objects to maximize RAM cache hits +* Disable RAM caching entirely for certain objects by setting a very low cutoff Exclusive spans and volume sizes ================================ @@ -126,5 +173,24 @@ ramcache has been disabled.:: volume=1 scheme=http size=20% volume=2 scheme=http size=20% volume=3 scheme=http size=20% - volume=4 scheme=http size=20% avg_obj_size=4096 - volume=5 scheme=http size=20% ramcache=false fragment_size=524288 + volume=4 scheme=http size=20% avg_obj_size=4K + volume=5 scheme=http size=20% ramcache=false fragment_size=512K + +The following example shows advanced RAM cache configuration with dedicated +allocations and custom cutoffs:: + + # Volume 1: General content with 2GB dedicated RAM cache + volume=1 scheme=http size=40% ram_cache_size=2G + + # Volume 2: Small API responses with custom cutoff and 512MB RAM cache + volume=2 scheme=http size=20% ram_cache_size=512M ram_cache_cutoff=64K + + # Volume 3: Large media with higher cutoff for thumbnails + volume=3 scheme=http size=40% ram_cache_cutoff=1M + +In this example, assuming a global ``proxy.config.cache.ram_cache.size`` of 4GB: + +* Volume 1 gets a dedicated 2GB RAM cache allocation +* Volume 2 gets a dedicated 512MB RAM cache allocation and only caches objects up to 64KB +* Volume 3 shares from the remaining 1.5GB pool (4GB - 2GB - 512MB) and caches objects up to 1MB +* The automatic subtraction ensures total RAM usage stays within the 4GB limit diff --git a/include/iocore/cache/Cache.h b/include/iocore/cache/Cache.h index 2e8369c909..3a7da523b5 100644 --- a/include/iocore/cache/Cache.h +++ b/include/iocore/cache/Cache.h @@ -52,6 +52,7 @@ struct CacheDisk; class URL; class HTTPHdr; class HTTPInfo; +struct CacheHostRecord; using CacheHTTPHdr = HTTPHdr; using CacheURL = URL; @@ -83,9 +84,9 @@ struct CacheProcessor : public Processor { Action *scan(Continuation *cont, std::string_view hostname = std::string_view{}, int KB_per_second = SCAN_KB_PER_SECOND); Action *lookup(Continuation *cont, const HttpCacheKey *key, CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP); Action *open_read(Continuation *cont, const HttpCacheKey *key, CacheHTTPHdr *request, const HttpConfigAccessor *params, - CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP); + CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP, const CacheHostRecord *volume_host_rec = nullptr); Action *open_write(Continuation *cont, const HttpCacheKey *key, CacheHTTPInfo *old_info, time_t pin_in_cache = 0, - CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP); + CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP, const CacheHostRecord *volume_host_rec = nullptr); Action *remove(Continuation *cont, const HttpCacheKey *key, CacheFragType frag_type = CACHE_FRAG_TYPE_HTTP); /** Mark physical disk/device/file as offline. diff --git a/include/proxy/ReverseProxy.h b/include/proxy/ReverseProxy.h index e7aca26ca0..c24748b796 100644 --- a/include/proxy/ReverseProxy.h +++ b/include/proxy/ReverseProxy.h @@ -32,6 +32,8 @@ #pragma once +#include <atomic> + #include "records/RecProcess.h" #include "tscore/ink_defs.h" @@ -45,7 +47,7 @@ class url_mapping; struct host_hdr_info; -extern UrlRewrite *rewrite_table; +extern std::atomic<UrlRewrite *> rewrite_table; // API Functions int init_reverse_proxy(); @@ -57,4 +59,5 @@ bool response_url_remap(HTTPHdr *response_header, UrlRewrite *table); bool reloadUrlRewrite(); bool urlRewriteVerify(); -int url_rewrite_CB(const char *name, RecDataT data_type, RecData data, void *cookie); +void init_remap_volume_host_records(); +int url_rewrite_CB(const char *name, RecDataT data_type, RecData data, void *cookie); diff --git a/include/proxy/http/HttpTransact.h b/include/proxy/http/HttpTransact.h index 0e05c1e14b..52c92a9cb7 100644 --- a/include/proxy/http/HttpTransact.h +++ b/include/proxy/http/HttpTransact.h @@ -102,6 +102,7 @@ using ink_time_t = time_t; struct HttpConfigParams; class HttpSM; +struct CacheHostRecord; #include "iocore/net/ConnectionTracker.h" #include "tscore/InkErrno.h" @@ -494,6 +495,8 @@ public: URL *parent_selection_url = nullptr; URL parent_selection_url_storage; + const CacheHostRecord *volume_host_rec = nullptr; + _CacheLookupInfo() {} }; @@ -715,8 +718,9 @@ public: MgmtByte cache_open_write_fail_action = 0; - HttpConfigParams *http_config_param = nullptr; - CacheLookupInfo cache_info; + HttpConfigParams *http_config_param = nullptr; + CacheLookupInfo cache_info; + ResolveInfo dns_info; RedirectInfo redirect_info; ConnectionTracker::TxnState outbound_conn_track_state; diff --git a/include/proxy/http/remap/RemapConfig.h b/include/proxy/http/remap/RemapConfig.h index 8456dd846c..559604e560 100644 --- a/include/proxy/http/remap/RemapConfig.h +++ b/include/proxy/http/remap/RemapConfig.h @@ -40,6 +40,7 @@ class UrlRewrite; #define REMAP_OPTFLG_INTERNAL 0x0080u /* only allow internal requests to hit this remap */ #define REMAP_OPTFLG_IN_IP 0x0100u /* "in_ip=" option (used for ACL filtering)*/ #define REMAP_OPTFLG_STRATEGY 0x0200u /* "strategy=" the name of the nexthop selection strategy */ +#define REMAP_OPTFLG_VOLUME 0x0400u /* "volume=" cache volume override */ #define REMAP_OPTFLG_MAP_ID 0x0800u /* associate a map ID with this rule */ #define REMAP_OPTFLG_INVERT 0x80000000u /* "invert" the rule (for src_ip and src_ip_category at least) */ #define REMAP_OPTFLG_ALL_FILTERS \ diff --git a/include/proxy/http/remap/UrlMapping.h b/include/proxy/http/remap/UrlMapping.h index dabab07118..8888e1568f 100644 --- a/include/proxy/http/remap/UrlMapping.h +++ b/include/proxy/http/remap/UrlMapping.h @@ -24,6 +24,8 @@ #pragma once +#include <atomic> +#include <string> #include <vector> #include "tscore/ink_config.h" @@ -36,6 +38,7 @@ #include "tscore/List.h" class NextHopSelectionStrategy; +struct CacheHostRecord; /** * Used to store http referrer strings (and/or regexp) @@ -112,20 +115,41 @@ public: bool ip_allow_check_enabled_p = false; acl_filter_rule *filter = nullptr; // acl filtering (linked list of rules) LINK(url_mapping, link); // For use with the main Queue linked list holding all the mapping - NextHopSelectionStrategy *strategy = nullptr; - std::string remapKey; - std::atomic<uint64_t> _hitCount = 0; // counter can overflow + NextHopSelectionStrategy *strategy = nullptr; + std::string remapKey; + std::atomic<uint64_t> _hitCount = 0; // counter can overflow + std::atomic<CacheHostRecord *> volume_host_rec = nullptr; + + CacheHostRecord * + getVolumeHostRec() const + { + return volume_host_rec.load(std::memory_order_acquire); + } + + void + setVolume(const char *str) + { + if (str && *str) { + _volume_str = str; + } + } + + const std::string & + getVolume() const + { + return _volume_str; + } int getRank() const { return _rank; - }; + } void setRank(int rank) { _rank = rank; - }; + } void setRemapKey() @@ -145,9 +169,12 @@ public: _hitCount++; } + bool initVolumeHostRec(char *errbuf, size_t errbufsize); + private: std::vector<RemapPluginInst *> _plugin_inst_list; int _rank = 0; + std::string _volume_str; }; /** diff --git a/include/proxy/http/remap/UrlMappingPathIndex.h b/include/proxy/http/remap/UrlMappingPathIndex.h index 63bc90cd6e..a74156c9f2 100644 --- a/include/proxy/http/remap/UrlMappingPathIndex.h +++ b/include/proxy/http/remap/UrlMappingPathIndex.h @@ -40,6 +40,19 @@ public: void Print() const; std::string PrintUrlMappingPathIndex() const; + // Apply a function to each url_mapping in this index. + // Note: Trie only exposes const_iterator, so const_cast is required. + template <typename Func> + void + foreach_mapping(Func &&f) + { + for (auto &trie_pair : m_tries) { + for (auto const &mapping : *trie_pair.second) { + f(const_cast<url_mapping &>(mapping)); + } + } + } + private: using UrlMappingTrie = Trie<url_mapping>; diff --git a/src/iocore/cache/Cache.cc b/src/iocore/cache/Cache.cc index 512a5e7bf7..14855e98fa 100644 --- a/src/iocore/cache/Cache.cc +++ b/src/iocore/cache/Cache.cc @@ -100,6 +100,8 @@ ClassAllocator<EvacuationBlock, false> evacuationBlockAllocator("evacuationBl ClassAllocator<CacheRemoveCont, false> cacheRemoveContAllocator("cacheRemoveCont"); ClassAllocator<EvacuationKey, false> evacuationKeyAllocator("evacuationKey"); std::unordered_set<std::string> known_bad_disks; +// Process-lifetime singleton: allocated during cache init and intentionally never freed. +CacheHostRecord *default_volumes_host_rec = nullptr; namespace { @@ -231,7 +233,7 @@ Cache::open_done() } ReplaceablePtr<CacheHostTable>::ScopedReader hosttable(&this->hosttable); - if (hosttable->gen_host_rec.num_cachevols == 0) { + if (hosttable->getGenHostRecCacheVols() == 0) { ready = CacheInitState::FAILED; } else { ready = CacheInitState::INITIALIZED; @@ -242,6 +244,22 @@ Cache::open_done() Emergency("Failed to initialize cache host table"); } + // Initialize default_volumes_host_rec from proxy.config.cache.default_volumes + if (ready == CacheInitState::INITIALIZED && default_volumes_host_rec == nullptr) { + auto default_volumes_str = RecGetRecordStringAlloc("proxy.config.cache.default_volumes"); + + if (default_volumes_str && !default_volumes_str.value().empty()) { + char errbuf[256]; + + default_volumes_host_rec = createCacheHostRecord(default_volumes_str.value().c_str(), errbuf, sizeof(errbuf)); + if (default_volumes_host_rec != nullptr) { + Dbg(dbg_ctl_cache_init, "Initialized default_volumes from '%s'", default_volumes_str.value().c_str()); + } else { + Warning("Failed to parse proxy.config.cache.default_volumes '%s': %s", default_volumes_str.value().c_str(), errbuf); + } + } + } + cacheProcessor.cacheInitialized(); return 0; @@ -529,7 +547,7 @@ Cache::scan(Continuation *cont, std::string_view hostname, int KB_per_second) co Action * Cache::open_read(Continuation *cont, const CacheKey *key, CacheHTTPHdr *request, const HttpConfigAccessor *params, - CacheFragType type, std::string_view hostname) const + CacheFragType type, std::string_view hostname, const CacheHostRecord *volume_host_rec) const { if (!CacheProcessor::IsCacheReady(type)) { cont->handleEvent(CACHE_EVENT_OPEN_READ_FAILED, reinterpret_cast<void *>(-ECACHE_NOT_READY)); @@ -537,7 +555,7 @@ Cache::open_read(Continuation *cont, const CacheKey *key, CacheHTTPHdr *request, } ink_assert(caches[type] == this); - StripeSM *stripe = key_to_stripe(key, hostname); + StripeSM *stripe = key_to_stripe(key, hostname, volume_host_rec); Dir result, *last_collision = nullptr; ProxyMutex *mutex = cont->mutex.get(); OpenDirEntry *od = nullptr; @@ -603,8 +621,8 @@ Lcallreturn: // main entry point for writing of http documents Action * -Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *info, time_t apin_in_cache, CacheFragType type, - std::string_view hostname) const +Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *old_info, time_t pin_in_cache, CacheFragType type, + std::string_view hostname, const CacheHostRecord *volume_host_rec) const { if (!CacheProcessor::IsCacheReady(type)) { cont->handleEvent(CACHE_EVENT_OPEN_WRITE_FAILED, reinterpret_cast<void *>(-ECACHE_NOT_READY)); @@ -613,7 +631,7 @@ Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *info, ink_assert(caches[type] == this); intptr_t err = 0; - int if_writers = reinterpret_cast<uintptr_t>(info) == CACHE_ALLOW_MULTIPLE_WRITES; + int if_writers = reinterpret_cast<uintptr_t>(old_info) == CACHE_ALLOW_MULTIPLE_WRITES; CacheVC *c = new_CacheVC(cont); c->vio.op = VIO::WRITE; c->first_key = *key; @@ -629,10 +647,10 @@ Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *info, } while (DIR_MASK_TAG(c->key.slice32(2)) == DIR_MASK_TAG(c->first_key.slice32(2))); c->earliest_key = c->key; c->frag_type = CACHE_FRAG_TYPE_HTTP; - c->stripe = key_to_stripe(key, hostname); + c->stripe = key_to_stripe(key, hostname, volume_host_rec); StripeSM *stripe = c->stripe; - c->info = info; - if (c->info && reinterpret_cast<uintptr_t>(info) != CACHE_ALLOW_MULTIPLE_WRITES) { + c->info = old_info; + if (c->info && reinterpret_cast<uintptr_t>(old_info) != CACHE_ALLOW_MULTIPLE_WRITES) { /* Update has the following code paths : a) Update alternate header only : @@ -664,9 +682,9 @@ Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *info, c->f.update = 1; c->op_type = static_cast<int>(CacheOpType::Update); DDbg(dbg_ctl_cache_update, "Update called"); - info->object_key_get(&c->update_key); + old_info->object_key_get(&c->update_key); ink_assert(!(c->update_key.is_zero())); - c->update_len = info->object_size_get(); + c->update_len = old_info->object_size_get(); } else { c->op_type = static_cast<int>(CacheOpType::Write); } @@ -674,7 +692,7 @@ Cache::open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *info, ts::Metrics::Gauge::increment(cache_rsb.status[c->op_type].active); ts::Metrics::Gauge::increment(stripe->cache_vol->vol_rsb.status[c->op_type].active); // coverity[Y2K38_SAFETY:FALSE] - c->pin_in_cache = static_cast<uint32_t>(apin_in_cache); + c->pin_in_cache = static_cast<uint32_t>(pin_in_cache); { CACHE_TRY_LOCK(lock, c->stripe->mutex, cont->mutex->thread_holding); @@ -745,41 +763,61 @@ CacheVConnection::CacheVConnection() : VConnection(nullptr) {} // if generic_host_rec.stripes == nullptr, what do we do??? StripeSM * -Cache::key_to_stripe(const CacheKey *key, std::string_view hostname) const +Cache::key_to_stripe(const CacheKey *key, std::string_view hostname, const CacheHostRecord *volume_host_rec) const { ReplaceablePtr<CacheHostTable>::ScopedReader hosttable(&this->hosttable); - uint32_t h = (key->slice32(2) >> DIR_TAG_WIDTH) % STRIPE_HASH_TABLE_SIZE; - unsigned short *hash_table = hosttable->gen_host_rec.vol_hash_table; - const CacheHostRecord *host_rec = &hosttable->gen_host_rec; + uint32_t h = (key->slice32(2) >> DIR_TAG_WIDTH) % STRIPE_HASH_TABLE_SIZE; + const CacheHostRecord *host_rec = hosttable->getGenHostRec(); + unsigned short *hash_table = host_rec->vol_hash_table; + StripeSM *selected_stripe = nullptr; + bool remap_selection = false; - if (hosttable->m_numEntries > 0 && !hostname.empty()) { + // Priority 1: @volume directive (highest priority) + if (volume_host_rec && volume_host_rec->vol_hash_table) { + selected_stripe = volume_host_rec->stripes[volume_host_rec->vol_hash_table[h]]; + remap_selection = true; + Dbg(dbg_ctl_cache_hosting, "@volume directive: using volume hash table for stripe selection"); + } + // Priority 2: Normal hostname-based volume selection (from hosting.config) + if (!selected_stripe && hosttable->getNumEntries() > 0 && !hostname.empty()) { CacheHostResult res; + hosttable->Match(hostname, &res); if (res.record) { unsigned short *host_hash_table = res.record->vol_hash_table; + if (host_hash_table) { - if (dbg_ctl_cache_hosting.on()) { - char format_str[50]; - snprintf(format_str, sizeof(format_str), "Volume: %%xd for host: %%.%ds", static_cast<int>(hostname.length())); - Dbg(dbg_ctl_cache_hosting, format_str, res.record, hostname.data()); - } - return res.record->stripes[host_hash_table[h]]; + Dbg(dbg_ctl_cache_hosting, "Volume: %p for host: %.*s", res.record, static_cast<int>(hostname.length()), hostname.data()); + selected_stripe = res.record->stripes[host_hash_table[h]]; } } } - if (hash_table) { - if (dbg_ctl_cache_hosting.on()) { - char format_str[50]; - snprintf(format_str, sizeof(format_str), "Generic volume: %%xd for host: %%.%ds", static_cast<int>(hostname.length())); - Dbg(dbg_ctl_cache_hosting, format_str, host_rec, hostname.data()); + + // Priority 3: Global default volumes from proxy.config.cache.default_volumes + if (!selected_stripe && default_volumes_host_rec && default_volumes_host_rec->vol_hash_table) { + selected_stripe = default_volumes_host_rec->stripes[default_volumes_host_rec->vol_hash_table[h]]; + Dbg(dbg_ctl_cache_hosting, "Using default_volumes for stripe selection"); + } + + // Priority 4: Generic/default volume selection (fallback) + if (!selected_stripe) { + if (hash_table) { + selected_stripe = host_rec->stripes[hash_table[h]]; + } else { + selected_stripe = host_rec->stripes[0]; } - return host_rec->stripes[hash_table[h]]; - } else { - return host_rec->stripes[0]; } -} + if (dbg_ctl_cache_hosting.on() && selected_stripe && selected_stripe->cache_vol) { + Dbg(dbg_ctl_cache_hosting, "Cache volume selected: %d (%s) for key=%08x%08x hostname='%.*s' %s", + selected_stripe->cache_vol->vol_number, + selected_stripe->cache_vol->ramcache_enabled ? "ramcache_enabled" : "ramcache_disabled", key->slice32(0), key->slice32(1), + static_cast<int>(hostname.length()), hostname.data(), remap_selection ? "(remap)" : "(calculated)"); + } + + return selected_stripe; +} int FragmentSizeUpdateCb(const char * /* name ATS_UNUSED */, RecDataT /* data_type ATS_UNUSED */, RecData data, void * /* cookie ATS_UNUSED */) diff --git a/src/iocore/cache/CacheHosting.cc b/src/iocore/cache/CacheHosting.cc index c8f7d5f679..9f24fb1a91 100644 --- a/src/iocore/cache/CacheHosting.cc +++ b/src/iocore/cache/CacheHosting.cc @@ -38,6 +38,7 @@ namespace DbgCtl dbg_ctl_cache_hosting{"cache_hosting"}; DbgCtl dbg_ctl_matcher{"matcher"}; +constexpr static int MAX_VOLUME_IDX = 255; } // end anonymous namespace /************************************************************* @@ -652,6 +653,8 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) bool ramcache_enabled = true; int avg_obj_size = -1; // Defaults int fragment_size = -1; + int64_t ram_cache_size = -1; // -1 means use shared allocation + int64_t ram_cache_cutoff = -1; // -1 means use global cutoff while (true) { // skip all blank spaces at beginning of line @@ -696,7 +699,7 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) break; } - if (volume_number < 1 || volume_number > 255) { + if (volume_number < 1 || volume_number > MAX_VOLUME_IDX) { err = "Bad Volume Number"; break; } @@ -740,17 +743,33 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) in_percent = 0; } } else if (strcasecmp(tmp, "avg_obj_size") == 0) { // match avg_obj_size - tmp += 13; - avg_obj_size = atoi(tmp); + tmp += 13; + if (!ParseRules::is_digit(*tmp)) { + err = "Invalid avg_obj_size value (must start with a number, e.g., 64K)"; + break; + } + avg_obj_size = static_cast<int>(ink_atoi64(tmp)); - while (ParseRules::is_digit(*tmp)) { + if (avg_obj_size < 0) { + err = "Invalid avg_obj_size value (must be >= 0)"; + break; + } + while (*tmp && (ParseRules::is_digit(*tmp) || strchr("KMGT", *tmp))) { tmp++; } } else if (strcasecmp(tmp, "fragment_size") == 0) { // match fragment_size - tmp += 14; - fragment_size = atoi(tmp); + tmp += 14; + if (!ParseRules::is_digit(*tmp)) { + err = "Invalid fragment_size value (must start with a number, e.g., 1M)"; + break; + } + fragment_size = static_cast<int>(ink_atoi64(tmp)); - while (ParseRules::is_digit(*tmp)) { + if (fragment_size < 0) { + err = "Invalid fragment_size value (must be >= 0)"; + break; + } + while (*tmp && (ParseRules::is_digit(*tmp) || strchr("KMGT", *tmp))) { tmp++; } } else if (strcasecmp(tmp, "ramcache") == 0) { // match ramcache @@ -765,11 +784,42 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) err = "Unexpected end of line"; break; } + } else if (strcasecmp(tmp, "ram_cache_size") == 0) { // match ram_cache_size + tmp += 15; + if (!ParseRules::is_digit(*tmp)) { + err = "Invalid ram_cache_size value (must start with a number, e.g., 10G)"; + break; + } + ram_cache_size = ink_atoi64(tmp); + + if (ram_cache_size < 0) { + err = "Invalid ram_cache_size value (must be >= 0)"; + break; + } + // Note: ram_cache_size=0 disables RAM cache for this volume, same as ramcache=false + while (*tmp && (ParseRules::is_digit(*tmp) || strchr("KMGT", *tmp))) { + tmp++; + } + } else if (strcasecmp(tmp, "ram_cache_cutoff") == 0) { // match ram_cache_cutoff + tmp += 17; + if (!ParseRules::is_digit(*tmp)) { + err = "Invalid ram_cache_cutoff value (must start with a number, e.g., 5M)"; + break; + } + ram_cache_cutoff = ink_atoi64(tmp); + + if (ram_cache_cutoff < 0) { + err = "Invalid ram_cache_cutoff value (must be >= 0)"; + break; + } + while (*tmp && (ParseRules::is_digit(*tmp) || strchr("KMGT", *tmp))) { + tmp++; + } } // ends here if (end < line_end) { - tmp++; + tmp = line_end; } } @@ -791,6 +841,8 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) configp->size = size; configp->avg_obj_size = avg_obj_size; configp->fragment_size = fragment_size; + configp->ram_cache_size = ram_cache_size; + configp->ram_cache_cutoff = ram_cache_cutoff; configp->cachep = nullptr; configp->ramcache_enabled = ramcache_enabled; cp_queue.enqueue(configp); @@ -800,8 +852,10 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) } else { ink_release_assert(!"Unexpected non-HTTP cache volume"); } - Dbg(dbg_ctl_cache_hosting, "added volume=%d, scheme=%d, size=%d percent=%d, ramcache enabled=%d", volume_number, - static_cast<int>(scheme), size, in_percent, ramcache_enabled); + Dbg(dbg_ctl_cache_hosting, + "added volume=%d, scheme=%d, size=%d percent=%d, ramcache enabled=%d, " + "ram_cache_size=%" PRId64 ", ram_cache_cutoff=%" PRId64, + volume_number, static_cast<int>(scheme), size, in_percent, ramcache_enabled, ram_cache_size, ram_cache_cutoff); } tmp = bufTok.iterNext(&i_state); @@ -809,3 +863,49 @@ ConfigVolumes::BuildListFromString(char *config_file_path, char *file_buf) return; } + +// Wrapper function for deleting CacheHostRecord from outside the cache module. +void +destroyCacheHostRecord(CacheHostRecord *rec) +{ + delete rec; +} + +// Build a CacheHostRecord for @volume= directive with comma-separated volumes +// This reuses CacheHostRecord::Init() by constructing a minimal matcher_line +CacheHostRecord * +createCacheHostRecord(const char *volume_str, char *errbuf, size_t errbufsize) +{ + if (!volume_str || !*volume_str) { + snprintf(errbuf, errbufsize, "Empty volume specification"); + return nullptr; + } + + CacheHostRecord *host_rec = new CacheHostRecord(); + + // Build a minimal matcher_line structure with just the volume= directive + char volume_key[] = "volume"; + matcher_line ml; + memset(&ml, 0, sizeof(ml)); + + ml.line[0][0] = volume_key; + ml.line[1][0] = const_cast<char *>(volume_str); + ml.num_el = 1; + ml.dest_entry = -1; + ml.line_num = 0; + ml.type = MATCH_NONE; + ml.next = nullptr; + + int result = host_rec->Init(&ml, CacheType::HTTP); + + if (result != 0) { + delete host_rec; + snprintf(errbuf, errbufsize, "Failed to initialize volume record (check volume.config)"); + return nullptr; + } + + Dbg(dbg_ctl_cache_hosting, "Created remap volume record with %d volumes, %d stripes", host_rec->num_cachevols, + host_rec->num_vols); + + return host_rec; +} diff --git a/src/iocore/cache/CacheProcessor.cc b/src/iocore/cache/CacheProcessor.cc index 8d1faf6871..14d7732097 100644 --- a/src/iocore/cache/CacheProcessor.cc +++ b/src/iocore/cache/CacheProcessor.cc @@ -411,16 +411,16 @@ CacheProcessor::lookup(Continuation *cont, const HttpCacheKey *key, CacheFragTyp Action * CacheProcessor::open_read(Continuation *cont, const HttpCacheKey *key, CacheHTTPHdr *request, const HttpConfigAccessor *params, - CacheFragType type) + CacheFragType frag_type, const CacheHostRecord *volume_host_rec) { - return caches[type]->open_read(cont, &key->hash, request, params, type, key->hostname); + return caches[frag_type]->open_read(cont, &key->hash, request, params, frag_type, key->hostname, volume_host_rec); } Action * CacheProcessor::open_write(Continuation *cont, const HttpCacheKey *key, CacheHTTPInfo *old_info, time_t pin_in_cache, - CacheFragType type) + CacheFragType frag_type, const CacheHostRecord *volume_host_rec) { - return caches[type]->open_write(cont, &key->hash, old_info, pin_in_cache, type, key->hostname); + return caches[frag_type]->open_write(cont, &key->hash, old_info, pin_in_cache, frag_type, key->hostname, volume_host_rec); } //---------------------------------------------------------------------------- @@ -483,7 +483,7 @@ CacheProcessor::mark_storage_offline(CacheDisk *d, ///< Target disk } else { // check cache types specifically if (theCache) { ReplaceablePtr<CacheHostTable>::ScopedReader hosttable(&theCache->hosttable); - if (!hosttable->gen_host_rec.vol_hash_table) { + if (!hosttable->getGenHostRec()->vol_hash_table) { unsigned int caches_ready = 0; caches_ready = caches_ready | (1 << CACHE_FRAG_TYPE_HTTP); caches_ready = caches_ready | (1 << CACHE_FRAG_TYPE_NONE); @@ -506,8 +506,8 @@ void rebuild_host_table(Cache *cache) { ReplaceablePtr<CacheHostTable>::ScopedWriter hosttable(&cache->hosttable); - build_vol_hash_table(&hosttable->gen_host_rec); - if (hosttable->m_numEntries != 0) { + build_vol_hash_table(const_cast<CacheHostRecord *>(hosttable->getGenHostRec())); + if (hosttable->getNumEntries() != 0) { CacheHostMatcher *hm = hosttable->getHostMatcher(); CacheHostRecord *h_rec = hm->getDataArray(); int h_rec_len = hm->getNumElements(); @@ -1168,6 +1168,8 @@ cplist_update() cp->ramcache_enabled = config_vol->ramcache_enabled; cp->avg_obj_size = config_vol->avg_obj_size; cp->fragment_size = config_vol->fragment_size; + cp->ram_cache_size = config_vol->ram_cache_size; + cp->ram_cache_cutoff = config_vol->ram_cache_cutoff; config_vol->cachep = cp; } else { /* delete this volume from all the disks */ @@ -1440,9 +1442,32 @@ CacheProcessor::cacheInitialized() } } + // Calculate total private RAM allocations from per-volume configurations int64_t http_ram_cache_size = 0; + int64_t total_private_ram = 0; + + if (cache_config_ram_cache_size != AUTO_SIZE_RAM_CACHE) { + CacheVol *cp = cp_list.head; + + for (; cp; cp = cp->link.next) { + if (cp->ram_cache_size > 0 && !cp->ramcache_enabled) { + Warning("Volume %d has ram_cache_size=%" PRId64 " but ramcache=false, ignoring ram_cache_size", cp->vol_number, + cp->ram_cache_size); + cp->ram_cache_size = -1; + } + if (cp->ram_cache_size > 0) { + total_private_ram += cp->ram_cache_size; + Dbg(dbg_ctl_cache_init, "Volume %d has private RAM allocation: %" PRId64 " bytes (%" PRId64 " MB)", cp->vol_number, + cp->ram_cache_size, cp->ram_cache_size / (1024 * 1024)); + } + } + + if (total_private_ram > 0) { + Dbg(dbg_ctl_cache_init, "Total private RAM allocations: %" PRId64 " bytes (%" PRId64 " MB)", total_private_ram, + total_private_ram / (1024 * 1024)); + } + } - // let us calculate the Size if (cache_config_ram_cache_size == AUTO_SIZE_RAM_CACHE) { Dbg(dbg_ctl_cache_init, "cache_config_ram_cache_size == AUTO_SIZE_RAM_CACHE"); } else { @@ -1450,12 +1475,24 @@ CacheProcessor::cacheInitialized() // TODO, should we check the available system memories, or you will // OOM or swapout, that is not a good situation for the server Dbg(dbg_ctl_cache_init, "%" PRId64 " != AUTO_SIZE_RAM_CACHE", cache_config_ram_cache_size); - http_ram_cache_size = - static_cast<int64_t>((static_cast<double>(theCache->cache_size) / total_size) * cache_config_ram_cache_size); + + // Calculate shared pool: global RAM cache size minus private allocations + int64_t shared_pool = cache_config_ram_cache_size - total_private_ram; + + if (shared_pool < 0) { + Fatal("Total private RAM cache allocations (%" PRId64 " bytes) exceed global ram_cache.size (%" PRId64 " bytes). " + "Increase proxy.config.cache.ram_cache.size or reduce per-volume ram_cache_size allocations.", + total_private_ram, cache_config_ram_cache_size); + } else if (total_private_ram > 0) { + Dbg(dbg_ctl_cache_init, "Shared RAM cache pool (after private allocations): %" PRId64 " bytes (%" PRId64 " MB)", + shared_pool, shared_pool / (1024 * 1024)); + } + + http_ram_cache_size = static_cast<int64_t>((static_cast<double>(theCache->cache_size) / total_size) * shared_pool); Dbg(dbg_ctl_cache_init, "http_ram_cache_size = %" PRId64 " = %" PRId64 "Mb", http_ram_cache_size, http_ram_cache_size / (1024 * 1024)); - int64_t stream_ram_cache_size = cache_config_ram_cache_size - http_ram_cache_size; + int64_t stream_ram_cache_size = shared_pool - http_ram_cache_size; Dbg(dbg_ctl_cache_init, "stream_ram_cache_size = %" PRId64 " = %" PRId64 "Mb", stream_ram_cache_size, stream_ram_cache_size / (1024 * 1024)); @@ -1468,23 +1505,55 @@ CacheProcessor::cacheInitialized() uint64_t total_cache_bytes = 0; // bytes that can used in total_size uint64_t total_direntries = 0; // all the direntries in the cache uint64_t used_direntries = 0; // and used - uint64_t total_ram_cache_bytes = 0; + uint64_t total_ram_cache_bytes = 0; // Total RAM cache size across all volumes + uint64_t shared_cache_size = 0; // Total cache size of volumes without explicit RAM allocations + + // Calculate total cache size of volumes without explicit RAM allocations + if (http_ram_cache_size > 0) { + for (int i = 0; i < gnstripes; i++) { + if (gstripes[i]->cache_vol->ram_cache_size <= 0) { + shared_cache_size += (gstripes[i]->len >> STORE_BLOCK_SHIFT); + } + } + Dbg(dbg_ctl_cache_init, "Shared cache size (for RAM pool distribution): %" PRId64 " blocks", shared_cache_size); + } for (int i = 0; i < gnstripes; i++) { StripeSM *stripe = gstripes[i]; int64_t ram_cache_bytes = 0; - if (stripe->cache_vol->ramcache_enabled) { - if (http_ram_cache_size == 0) { + // If RAM cache enabled, check if this volume has a private RAM cache allocation + if (stripe->cache_vol->ramcache_enabled && stripe->cache_vol->ram_cache_size != 0) { + if (stripe->cache_vol->ram_cache_size > 0) { + int64_t volume_stripe_count = 0; + + for (int j = 0; j < gnstripes; j++) { + if (gstripes[j]->cache_vol == stripe->cache_vol) { + volume_stripe_count++; + } + } + + if (volume_stripe_count > 0) { + ram_cache_bytes = stripe->cache_vol->ram_cache_size / volume_stripe_count; + Dbg(dbg_ctl_cache_init, "Volume %d stripe %d using private RAM allocation: %" PRId64 " bytes (%" PRId64 " MB)", + stripe->cache_vol->vol_number, i, ram_cache_bytes, ram_cache_bytes / (1024 * 1024)); + } + } else if (http_ram_cache_size == 0) { // AUTO_SIZE_RAM_CACHE ram_cache_bytes = stripe->dirlen() * DEFAULT_RAM_CACHE_MULTIPLIER; } else { - ink_assert(stripe->cache != nullptr); - - double factor = static_cast<double>(static_cast<int64_t>(stripe->len >> STORE_BLOCK_SHIFT)) / theCache->cache_size; - Dbg(dbg_ctl_cache_init, "factor = %f", factor); - - ram_cache_bytes = static_cast<int64_t>(http_ram_cache_size * factor); + // Use shared pool allocation - distribute only among volumes without explicit allocations + if (shared_cache_size > 0) { + ink_assert(stripe->cache != nullptr); + double factor = static_cast<double>(static_cast<int64_t>(stripe->len >> STORE_BLOCK_SHIFT)) / shared_cache_size; + + Dbg(dbg_ctl_cache_init, "factor = %f (divisor = %" PRId64 ")", factor, shared_cache_size); + ram_cache_bytes = static_cast<int64_t>(http_ram_cache_size * factor); + } else { + ram_cache_bytes = 0; + Dbg(dbg_ctl_cache_init, "Volume %d stripe has no explicit RAM allocation, but shared pool is empty", + stripe->cache_vol->vol_number); + } } stripe->ram_cache->init(ram_cache_bytes, stripe); @@ -1495,19 +1564,19 @@ CacheProcessor::cacheInitialized() ram_cache_bytes, ram_cache_bytes / (1024 * 1024)); } - uint64_t vol_total_cache_bytes = stripe->len - stripe->dirlen(); - total_cache_bytes += vol_total_cache_bytes; + uint64_t vol_total_cache_bytes = stripe->len - stripe->dirlen(); + uint64_t vol_total_direntries = stripe->directory.entries(); + uint64_t vol_used_direntries = stripe->directory.entries_used(); + + total_cache_bytes += vol_total_cache_bytes; ts::Metrics::Gauge::increment(stripe->cache_vol->vol_rsb.bytes_total, vol_total_cache_bytes); ts::Metrics::Gauge::increment(stripe->cache_vol->vol_rsb.stripes); Dbg(dbg_ctl_cache_init, "total_cache_bytes = %" PRId64 " = %" PRId64 "Mb", total_cache_bytes, total_cache_bytes / (1024 * 1024)); - uint64_t vol_total_direntries = stripe->directory.entries(); - total_direntries += vol_total_direntries; + total_direntries += vol_total_direntries; ts::Metrics::Gauge::increment(stripe->cache_vol->vol_rsb.direntries_total, vol_total_direntries); - - uint64_t vol_used_direntries = stripe->directory.entries_used(); ts::Metrics::Gauge::increment(stripe->cache_vol->vol_rsb.direntries_used, vol_used_direntries); used_direntries += vol_used_direntries; } diff --git a/src/iocore/cache/CacheVC.cc b/src/iocore/cache/CacheVC.cc index d230463e1b..e21c7a7028 100644 --- a/src/iocore/cache/CacheVC.cc +++ b/src/iocore/cache/CacheVC.cc @@ -412,15 +412,17 @@ CacheVC::handleReadDone(int event, Event * /* e ATS_UNUSED */) // Put the request in the ram cache only if its a open_read or lookup if (vio.op == VIO::READ && okay) { bool cutoff_check; + // Determine effective cutoff: use per-volume override if set, otherwise use global + int64_t effective_cutoff = + (stripe->cache_vol->ram_cache_cutoff > 0) ? stripe->cache_vol->ram_cache_cutoff : cache_config_ram_cache_cutoff; // cutoff_check : // doc_len == 0 for the first fragment (it is set from the vector) // The decision on the first fragment is based on // doc->total_len // After that, the decision is based of doc_len (doc_len != 0) - // (cache_config_ram_cache_cutoff == 0) : no cutoffs - cutoff_check = - ((!doc_len && static_cast<int64_t>(doc->total_len) < cache_config_ram_cache_cutoff) || - (doc_len && static_cast<int64_t>(doc_len) < cache_config_ram_cache_cutoff) || !cache_config_ram_cache_cutoff); + // (effective_cutoff == 0) : no cutoffs + cutoff_check = ((!doc_len && static_cast<int64_t>(doc->total_len) < effective_cutoff) || + (doc_len && static_cast<int64_t>(doc_len) < effective_cutoff) || !effective_cutoff); if (cutoff_check && !f.doc_from_ram_cache) { uint64_t o = dir_offset(&dir); stripe->ram_cache->put(read_key, buf.get(), doc->len, http_copy_hdr, o); @@ -635,7 +637,7 @@ CacheVC::scanStripe(int /* event ATS_UNUSED */, Event * /* e ATS_UNUSED */) ReplaceablePtr<CacheHostTable>::ScopedReader hosttable(&theCache->hosttable); - const CacheHostRecord *rec = &hosttable->gen_host_rec; + const CacheHostRecord *rec = hosttable->getGenHostRec(); if (!hostname.empty()) { CacheHostResult res; hosttable->Match(hostname, &res); diff --git a/src/iocore/cache/P_CacheHosting.h b/src/iocore/cache/P_CacheHosting.h index d6dc8fd6c2..d36821c51e 100644 --- a/src/iocore/cache/P_CacheHosting.h +++ b/src/iocore/cache/P_CacheHosting.h @@ -62,7 +62,9 @@ struct CacheHostRecord { CacheHostRecord() {} }; -void build_vol_hash_table(CacheHostRecord *cp); +void build_vol_hash_table(CacheHostRecord *cp); +CacheHostRecord *createCacheHostRecord(const char *volume_str, char *errbuf, size_t errbufsize); +void destroyCacheHostRecord(CacheHostRecord *rec); struct CacheHostResult { CacheHostRecord *record = nullptr; @@ -229,18 +231,42 @@ public: void Match(std::string_view rdata, CacheHostResult *result) const; void Print() const; + // Getters for Cache::key_to_stripe access + const CacheHostRecord * + getGenHostRec() const + { + return &gen_host_rec; + } + int - getEntryCount() const + getNumEntries() const { return m_numEntries; } + + int + getGenHostRecCacheVols() const + { + return gen_host_rec.num_cachevols; + } + CacheHostMatcher * getHostMatcher() const { return hostMatch.get(); } - static int config_callback(const char *, RecDataT, RecData, void *); + CacheType + getType() const + { + return type; + } + + Cache * + getCache() const + { + return cache; + } void register_config_callback(ReplaceablePtr<CacheHostTable> *p) @@ -248,12 +274,13 @@ public: RecRegisterConfigUpdateCb("proxy.config.cache.hosting_filename", CacheHostTable::config_callback, (void *)p); } - CacheType type = CacheType::HTTP; - Cache *cache = nullptr; - int m_numEntries = 0; - CacheHostRecord gen_host_rec; - private: + static int config_callback(const char *, RecDataT, RecData, void *); + + CacheType type = CacheType::HTTP; + Cache *cache = nullptr; + int m_numEntries = 0; + CacheHostRecord gen_host_rec; std::unique_ptr<CacheHostMatcher> hostMatch = nullptr; const matcher_tags config_tags = {"hostname", "domain", nullptr, nullptr, nullptr, nullptr, false}; const char *matcher_name = "unknown"; // Used for Debug/Warning/Error messages @@ -276,8 +303,8 @@ struct CacheHostTableConfig : public Continuation { Cache *cache = nullptr; { ReplaceablePtr<CacheHostTable>::ScopedReader hosttable(ppt); - type = hosttable->type; - cache = hosttable->cache; + type = hosttable->getType(); + cache = hosttable->getCache(); } ppt->reset(new CacheHostTable(cache, type)); delete this; @@ -298,6 +325,8 @@ struct ConfigVol { int percent; int avg_obj_size; int fragment_size; + int64_t ram_cache_size; // Per-volume RAM cache size (-1 = use shared allocation) + int64_t ram_cache_cutoff; // Per-volume RAM cache cutoff (-1 = use global cutoff) CacheVol *cachep; LINK(ConfigVol, link); }; diff --git a/src/iocore/cache/P_CacheInternal.h b/src/iocore/cache/P_CacheInternal.h index 08560ba85e..08e77291bb 100644 --- a/src/iocore/cache/P_CacheInternal.h +++ b/src/iocore/cache/P_CacheInternal.h @@ -108,6 +108,9 @@ struct EvacuationBlock; extern CacheStatsBlock cache_rsb; +// Global default volumes host record (initialized from proxy.config.cache.default_volumes) +extern CacheHostRecord *default_volumes_host_rec; + // Configuration extern int cache_config_dir_sync_frequency; extern int cache_config_dir_sync_delay; @@ -485,9 +488,11 @@ struct Cache { Action *scan(Continuation *cont, std::string_view hostname = std::string_view{}, int KB_per_second = 2500) const; Action *open_read(Continuation *cont, const CacheKey *key, CacheHTTPHdr *request, const HttpConfigAccessor *params, - CacheFragType type, std::string_view hostname = std::string_view{}) const; + CacheFragType type, std::string_view hostname = std::string_view{}, + const CacheHostRecord *volume_host_rec = nullptr) const; Action *open_write(Continuation *cont, const CacheKey *key, CacheHTTPInfo *old_info, time_t pin_in_cache = 0, - CacheFragType type = CACHE_FRAG_TYPE_HTTP, std::string_view hostname = std::string_view{}) const; + CacheFragType type = CACHE_FRAG_TYPE_HTTP, std::string_view hostname = std::string_view{}, + const CacheHostRecord *volume_host_rec = nullptr) const; static void generate_key(CryptoHash *hash, CacheURL *url); static void generate_key(HttpCacheKey *hash, CacheURL *url, bool ignore_query = false, cache_generation_t generation = -1); @@ -500,7 +505,7 @@ struct Cache { int open_done(); - StripeSM *key_to_stripe(const CacheKey *key, std::string_view hostname) const; + StripeSM *key_to_stripe(const CacheKey *key, std::string_view hostname, const CacheHostRecord *volume_host_rec = nullptr) const; Cache() {} }; diff --git a/src/iocore/cache/Stripe.h b/src/iocore/cache/Stripe.h index 46ab8201a6..b99b4773fe 100644 --- a/src/iocore/cache/Stripe.h +++ b/src/iocore/cache/Stripe.h @@ -58,6 +58,8 @@ struct CacheVol { int avg_obj_size = -1; // Defer to the records.config if not overriden int fragment_size = -1; // Defer to the records.config if not overriden bool ramcache_enabled = true; + int64_t ram_cache_size = -1; // Per-volume RAM cache size (-1 = use shared allocation) + int64_t ram_cache_cutoff = -1; // Per-volume RAM cache cutoff (-1 = use global cutoff) StripeSM **stripes = nullptr; DiskStripe **disk_stripes = nullptr; LINK(CacheVol, link); diff --git a/src/proxy/ReverseProxy.cc b/src/proxy/ReverseProxy.cc index 341c2c7de4..8deb9d0016 100644 --- a/src/proxy/ReverseProxy.cc +++ b/src/proxy/ReverseProxy.cc @@ -39,6 +39,7 @@ #include "proxy/http/remap/RemapProcessor.h" #include "proxy/http/remap/UrlRewrite.h" #include "proxy/http/remap/UrlMapping.h" +#include "proxy/http/remap/UrlMappingPathIndex.h" namespace { @@ -49,7 +50,7 @@ DbgCtl dbg_ctl_url_rewrite{"url_rewrite"}; } // end anonymous namespace // Global Ptrs -UrlRewrite *rewrite_table = nullptr; +std::atomic<UrlRewrite *> rewrite_table = nullptr; thread_local PluginThreadContext *pluginThreadContext = nullptr; // Tokens for the Callback function @@ -66,12 +67,13 @@ thread_local PluginThreadContext *pluginThreadContext = nullptr; int init_reverse_proxy() { - ink_assert(rewrite_table == nullptr); + ink_assert(rewrite_table.load() == nullptr); reconfig_mutex = new_ProxyMutex(); - rewrite_table = new UrlRewrite(); + rewrite_table.store(new UrlRewrite()); + rewrite_table.load()->acquire(); Note("%s loading ...", ts::filename::REMAP); - if (!rewrite_table->load()) { + if (!rewrite_table.load()->load()) { Emergency("%s failed to load", ts::filename::REMAP); } else { Note("%s finished loading", ts::filename::REMAP); @@ -82,9 +84,6 @@ init_reverse_proxy() RecRegisterConfigUpdateCb("proxy.config.reverse_proxy.enabled", url_rewrite_CB, (void *)REVERSE_CHANGED); RecRegisterConfigUpdateCb("proxy.config.http.referer_default_redirect", url_rewrite_CB, (void *)HTTP_DEFAULT_REDIRECT_CHANGED); - // Hold at least one lease, until we reload the configuration - rewrite_table->acquire(); - return 0; } @@ -148,7 +147,7 @@ reloadUrlRewrite() newTable->acquire(); // Swap configurations - oldTable = ink_atomic_swap(&rewrite_table, newTable); + oldTable = rewrite_table.exchange(newTable); ink_assert(oldTable != nullptr); @@ -168,6 +167,65 @@ reloadUrlRewrite() } } +/** + * Helper function to initialize volume_host_rec for a single url_mapping. + * This is a no-op if the mapping has no volume string or is already initialized. + */ +static void +init_mapping_volume_host_rec(url_mapping &mapping) +{ + char errbuf[256]; + + if (!mapping.initVolumeHostRec(errbuf, sizeof(errbuf))) { + Error("Failed to initialize volume record for @volume=%s: %s", mapping.getVolume().c_str(), errbuf); + } +} + +static void +init_store_volume_host_records(UrlRewrite::MappingsStore &store) +{ + if (store.hash_lookup) { + for (auto &entry : *store.hash_lookup) { + UrlMappingPathIndex *path_index = entry.second; + + if (path_index) { + path_index->foreach_mapping(init_mapping_volume_host_rec); + } + } + } + + for (UrlRewrite::RegexMapping *reg_map = store.regex_list.head; reg_map; reg_map = reg_map->link.next) { + if (reg_map->url_map) { + init_mapping_volume_host_rec(*reg_map->url_map); + } + } +} + +// This is called after the cache is initialized, since we may need the volume_host_records. +// Must only be called during startup before any remap reload can occur. +void +init_remap_volume_host_records() +{ + UrlRewrite *table = rewrite_table.load(std::memory_order_acquire); + + if (!table) { + return; + } + + table->acquire(); + + Dbg(dbg_ctl_url_rewrite, "Initializing volume_host_rec for all remap rules after cache init"); + + // Initialize for all mapping stores + init_store_volume_host_records(table->forward_mappings); + init_store_volume_host_records(table->reverse_mappings); + init_store_volume_host_records(table->permanent_redirects); + init_store_volume_host_records(table->temporary_redirects); + init_store_volume_host_records(table->forward_mappings_with_recv_port); + + table->release(); +} + int url_rewrite_CB(const char * /* name ATS_UNUSED */, RecDataT /* data_type ATS_UNUSED */, RecData data, void *cookie) { @@ -175,7 +233,7 @@ url_rewrite_CB(const char * /* name ATS_UNUSED */, RecDataT /* data_type ATS_UNU switch (my_token) { case REVERSE_CHANGED: - rewrite_table->SetReverseFlag(data.rec_int); + rewrite_table.load()->SetReverseFlag(data.rec_int); break; case TSNAME_CHANGED: diff --git a/src/proxy/http/HttpCacheSM.cc b/src/proxy/http/HttpCacheSM.cc index 7893fda4b1..1739f25491 100644 --- a/src/proxy/http/HttpCacheSM.cc +++ b/src/proxy/http/HttpCacheSM.cc @@ -323,12 +323,20 @@ HttpCacheSM::_schedule_read_retry() Action * HttpCacheSM::do_cache_open_read(const HttpCacheKey &key) { + Action *action_handle = nullptr; + open_read_tries++; ink_assert(pending_action == nullptr); // Initialising read-while-write-inprogress flag this->readwhilewrite_inprogress = false; - Action *action_handle = cacheProcessor.open_read(this, &key, this->read_request_hdr, &http_params); + + if (master_sm && master_sm->t_state.cache_info.volume_host_rec) { + action_handle = cacheProcessor.open_read(this, &key, this->read_request_hdr, &http_params, CACHE_FRAG_TYPE_HTTP, + master_sm->t_state.cache_info.volume_host_rec); + } else { + action_handle = cacheProcessor.open_read(this, &key, this->read_request_hdr, &http_params); + } if (action_handle != ACTION_RESULT_DONE) { pending_action = action_handle; @@ -422,9 +430,15 @@ HttpCacheSM::open_write(const HttpCacheKey *key, URL *url, HTTPHdr *request, Cac return ACTION_RESULT_DONE; } - // INKqa11166 CacheHTTPInfo *info = allow_multiple ? reinterpret_cast<CacheHTTPInfo *>(CACHE_ALLOW_MULTIPLE_WRITES) : old_info; - Action *action_handle = cacheProcessor.open_write(this, key, info, pin_in_cache); + Action *action_handle = nullptr; + + if (master_sm && master_sm->t_state.cache_info.volume_host_rec) { + action_handle = + cacheProcessor.open_write(this, key, info, pin_in_cache, CACHE_FRAG_TYPE_HTTP, master_sm->t_state.cache_info.volume_host_rec); + } else { + action_handle = cacheProcessor.open_write(this, key, info, pin_in_cache); + } if (action_handle != ACTION_RESULT_DONE) { pending_action = action_handle; diff --git a/src/proxy/http/HttpSM.cc b/src/proxy/http/HttpSM.cc index 4b977c1bc6..f454ddf24e 100644 --- a/src/proxy/http/HttpSM.cc +++ b/src/proxy/http/HttpSM.cc @@ -332,7 +332,7 @@ HttpSM::init(bool from_early_data) t_state.http_config_param = HttpConfig::acquire(); // Acquire a lease on the global remap / rewrite table (stupid global name ...) - m_remap = rewrite_table->acquire(); + m_remap = rewrite_table.load()->acquire(); // Simply point to the global config for the time being, no need to copy this // entire struct if nothing is going to change it. diff --git a/src/proxy/http/remap/RemapConfig.cc b/src/proxy/http/remap/RemapConfig.cc index 64d29d37c6..95d51af39c 100644 --- a/src/proxy/http/remap/RemapConfig.cc +++ b/src/proxy/http/remap/RemapConfig.cc @@ -36,6 +36,9 @@ #include "tscore/Filenames.h" #include "proxy/IPAllow.h" #include "proxy/http/remap/PluginFactory.h" +#include "iocore/cache/Cache.h" + +extern CacheHostRecord *createCacheHostRecord(const char *volume_str, char *errbuf, size_t errbufsize); using namespace std::literals; @@ -48,6 +51,7 @@ namespace DbgCtl dbg_ctl_url_rewrite{"url_rewrite"}; DbgCtl dbg_ctl_remap_plugin{"remap_plugin"}; DbgCtl dbg_ctl_url_rewrite_regex{"url_rewrite_regex"}; + } // end anonymous namespace /** @@ -829,6 +833,14 @@ remap_check_option(const char *const *argv, int argc, unsigned long findmode, in *argptr = &argv[i][9]; } ret_flags |= REMAP_OPTFLG_STRATEGY; + } else if (!strncasecmp(argv[i], "volume=", 7)) { + if ((findmode & REMAP_OPTFLG_VOLUME) != 0) { + idx = i; + } + if (argptr) { + *argptr = &argv[i][7]; + } + ret_flags |= REMAP_OPTFLG_VOLUME; } else { Warning("ignoring invalid remap option '%s'", argv[i]); } @@ -1197,12 +1209,74 @@ remap_parse_config_bti(const char *path, BUILD_TABLE_INFO *bti) if ((bti->remap_optflg & REMAP_OPTFLG_MAP_ID) != 0) { int idx = 0; int ret = remap_check_option(bti->argv, bti->argc, REMAP_OPTFLG_MAP_ID, &idx); + if (ret & REMAP_OPTFLG_MAP_ID) { - char *c = strchr(bti->argv[idx], static_cast<int>('=')); + char *c = strchr(bti->argv[idx], static_cast<int>('=')); + new_mapping->map_id = static_cast<unsigned int>(atoi(++c)); } } + // Parse @volume= option with comma-separated syntax (@volume=3,4) + for (int i = 0; i < bti->argc; i++) { + if (!strncasecmp(bti->argv[i], "volume=", 7)) { + const char *volume_str = &bti->argv[i][7]; + + if (!volume_str || !*volume_str) { + snprintf(errStrBuf, sizeof(errStrBuf), "Empty @volume= directive at line %d", cln + 1); + errStr = errStrBuf; + goto MAP_ERROR; + } + + { + swoc::TextView vol_list{volume_str}; + + if (vol_list.back() == ',') { + snprintf(errStrBuf, sizeof(errStrBuf), "Invalid @volume=%s at line %d (trailing comma)", volume_str, cln + 1); + errStr = errStrBuf; + goto MAP_ERROR; + } + while (!vol_list.empty()) { + swoc::TextView span; + swoc::TextView token{vol_list.take_prefix_at(',')}; + auto n = swoc::svtoi(token, &span); + + if (span.size() != token.size() || token.empty()) { + snprintf(errStrBuf, sizeof(errStrBuf), "Invalid @volume=%s at line %d (expected comma-separated numbers 1-255)", + volume_str, cln + 1); + errStr = errStrBuf; + goto MAP_ERROR; + } else if (n < 1 || n > 255) { + snprintf(errStrBuf, sizeof(errStrBuf), "Volume number %jd out of range (1-255) in @volume=%s at line %d", n, + volume_str, cln + 1); + errStr = errStrBuf; + goto MAP_ERROR; + } + } + } + + // Check if cache is ready (will be true during config reload, possibly false during initial startup) + if (CacheProcessor::IsCacheEnabled() == CacheInitState::INITIALIZED) { + char volume_errbuf[256]; + CacheHostRecord *rec = createCacheHostRecord(volume_str, volume_errbuf, sizeof(volume_errbuf)); + + if (!rec) { + snprintf(errStrBuf, sizeof(errStrBuf), "Failed to build volume record for @volume=%s at line %d: %s", volume_str, + cln + 1, volume_errbuf); + errStr = errStrBuf; + goto MAP_ERROR; + } + new_mapping->volume_host_rec.store(rec, std::memory_order_release); + Dbg(dbg_ctl_url_rewrite, "[BuildTable] Cache volume directive built: @volume=%s", volume_str); + } else { + // Store the volume string for lazy initialization after cache is ready + new_mapping->setVolume(volume_str); + Dbg(dbg_ctl_url_rewrite, "[BuildTable] Cache volume directive stored (deferred): @volume=%s", volume_str); + } + break; + } + } + map_from = bti->paramv[1]; length = UrlWhack(map_from, &origLength); diff --git a/src/proxy/http/remap/RemapProcessor.cc b/src/proxy/http/remap/RemapProcessor.cc index e14f8c1217..550af9360e 100644 --- a/src/proxy/http/remap/RemapProcessor.cc +++ b/src/proxy/http/remap/RemapProcessor.cc @@ -171,6 +171,9 @@ RemapProcessor::finish_remap(HttpTransact::State *s, UrlRewrite *table) return false; } + // Pass the volume_host_rec to the transaction state + s->cache_info.volume_host_rec = map->getVolumeHostRec(); + // Do fast ACL filtering (it is safe to check map here) table->PerformACLFiltering(s, map); diff --git a/src/proxy/http/remap/UrlMapping.cc b/src/proxy/http/remap/UrlMapping.cc index 0a3a85236e..887625c91a 100644 --- a/src/proxy/http/remap/UrlMapping.cc +++ b/src/proxy/http/remap/UrlMapping.cc @@ -27,6 +27,11 @@ #include "records/RecCore.h" #include "tscore/ink_cap.h" +// Avoid including private header files +struct CacheHostRecord; +extern void destroyCacheHostRecord(CacheHostRecord *rec); +extern CacheHostRecord *createCacheHostRecord(const char *volume_str, char *errbuf, size_t errbufsize); + namespace { DbgCtl dbg_ctl_url_rewrite{"url_rewrite"}; @@ -84,11 +89,39 @@ url_mapping::~url_mapping() delete afr; } + // Destroy any volume hosting records + destroyCacheHostRecord(volume_host_rec.load(std::memory_order_acquire)); + // Destroy the URLs fromURL.destroy(); toURL.destroy(); } +bool +url_mapping::initVolumeHostRec(char *errbuf, size_t errbufsize) +{ + if (_volume_str.empty() || volume_host_rec.load(std::memory_order_acquire) != nullptr) { + return true; + } + + CacheHostRecord *new_rec = createCacheHostRecord(_volume_str.c_str(), errbuf, errbufsize); + + if (!new_rec) { + return false; + } + + CacheHostRecord *expected = nullptr; + + if (volume_host_rec.compare_exchange_strong(expected, new_rec, std::memory_order_acq_rel)) { + Dbg(dbg_ctl_url_rewrite, "Initialized volume_host_rec for @volume=%s", _volume_str.c_str()); + return true; + } else { + // Another thread beat us to it, destroy our copy + destroyCacheHostRecord(new_rec); + return true; + } +} + void url_mapping::Print() const { diff --git a/src/records/RecordsConfig.cc b/src/records/RecordsConfig.cc index 215b6ffd6a..4ea75df726 100644 --- a/src/records/RecordsConfig.cc +++ b/src/records/RecordsConfig.cc @@ -82,6 +82,8 @@ static constexpr RecordElement RecordsConfig[] = , {RECT_CONFIG, "proxy.config.cache.persist_bad_disks", RECD_INT, "0", RECU_RESTART_TS, RR_NULL, RECC_INT, "[0-1]", RECA_NULL} , + {RECT_CONFIG, "proxy.config.cache.default_volumes", RECD_STRING, "", RECU_RESTART_TS, RR_NULL, RECC_NULL, nullptr, RECA_NULL} + , {RECT_CONFIG, "proxy.config.output.logfile.name", RECD_STRING, "traffic.out", RECU_RESTART_TS, RR_REQUIRED, RECC_NULL, nullptr, RECA_NULL} , diff --git a/src/traffic_server/traffic_server.cc b/src/traffic_server/traffic_server.cc index 811539f143..c9852947fc 100644 --- a/src/traffic_server/traffic_server.cc +++ b/src/traffic_server/traffic_server.cc @@ -831,6 +831,10 @@ CB_After_Cache_Init() start = ink_atomic_swap(&delay_listen_for_cache, -1); emit_fully_initialized_message(); + // Initialize volume_host_rec for any remap rules with @volume= directives + // that were deferred during startup because cache wasn't ready yet + init_remap_volume_host_records(); + if (1 == start) { // The delay_listen_for_cache value was 1, therefore the main function // delayed the call to start_HttpProxyServer until we got here. We must diff --git a/tests/gold_tests/autest-site/ats_replay.test.ext b/tests/gold_tests/autest-site/ats_replay.test.ext index 913257e7c3..976dc73b3d 100644 --- a/tests/gold_tests/autest-site/ats_replay.test.ext +++ b/tests/gold_tests/autest-site/ats_replay.test.ext @@ -53,6 +53,15 @@ def configure_ats(obj: 'TestRun', server: 'Process', ats_config: dict, dns: Opti if logging_yaml != None: ts.Disk.logging_yaml.AddLines(yaml.dump(logging_yaml).split('\n')) + # Configure volume.config if specified. + volume_config = ats_config.get('volume_config', []) + for vol in volume_config: + parts = [f"volume={vol['volume']}", f"scheme={vol['scheme']}", f"size={vol['size']}"] + for opt_key in ('ramcache', 'ram_cache_size', 'ram_cache_cutoff', 'avg_obj_size', 'fragment_size'): + if opt_key in vol: + parts.append(f"{opt_key}={vol[opt_key]}") + ts.Disk.volume_config.AddLine(' '.join(parts)) + remap_config = ats_config.get('remap_config', []) for remap_entry in remap_config: if isinstance(remap_entry, str): @@ -63,11 +72,14 @@ def configure_ats(obj: 'TestRun', server: 'Process', ats_config: dict, dns: Opti to_url = to_url.replace('{SERVER_HTTP_PORT}', str(server.Variables.http_port)) to_url = to_url.replace('{SERVER_HTTPS_PORT}', str(server.Variables.https_port)) plugins = remap_entry.get('plugins', []) + options = remap_entry.get('options', []) line = f'map {from_url} {to_url}' for plugin in plugins: line += f' @plugin={plugin["name"]}' for arg in plugin["args"]: line += f' @pparam={arg}' + for option in options: + line += f' {option}' ts.Disk.remap_config.AddLine(line) if dns: ts.Disk.records_config.update( diff --git a/tests/gold_tests/cache/cache_volume_defaults.replay.yaml b/tests/gold_tests/cache/cache_volume_defaults.replay.yaml new file mode 100644 index 0000000000..7f90c7efc9 --- /dev/null +++ b/tests/gold_tests/cache/cache_volume_defaults.replay.yaml @@ -0,0 +1,144 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +meta: + version: "1.0" + +# Test that proxy.config.cache.default_volumes and @volume= work correctly. +# Uses a single volume at 100% so it fits in the 256MB min_cfg storage +# (128MB stripe rounding requires at least 128MB per volume). +autest: + description: "Test proxy.config.cache.default_volumes configuration" + + dns: + name: "dns" + + server: + name: "server" + + client: + name: "client" + + ats: + name: "ts" + + records_config: + proxy.config.diags.debug.enabled: 1 + proxy.config.diags.debug.tags: "cache|cache_hosting|cache_init" + proxy.config.http.insert_response_via_str: 0 + proxy.config.cache.enable_read_while_writer: 0 + # Set default volumes to volume 1 + proxy.config.cache.default_volumes: "1" + + volume_config: + # Single volume — multi-param line proves parsing works + - volume: 1 + scheme: "http" + size: "100%" + + remap_config: + # Default volume selection (uses default_volumes -> volume 1) + - from: "http://default.example.com/"; + to: "http://backend.ex:{SERVER_HTTP_PORT}/"; + + # Explicit @volume=1 selection + - from: "http://volume1.example.com/"; + to: "http://backend.ex:{SERVER_HTTP_PORT}/"; + options: + - "@volume=1" + +# All transactions in a single session for sequential execution. +sessions: +- transactions: + + # Test 1: Request using default_volumes + - client-request: + method: "GET" + version: "1.1" + url: "/default_test" + headers: + fields: + - [Host, default.example.com] + - [X-Test-ID, "default-volumes-test"] + - [uuid, default-miss] + + server-response: + status: 200 + reason: OK + headers: + fields: + - [Content-Type, "text/plain"] + - [Cache-Control, "max-age=3600"] + - [X-Default-Volumes-Test, "success"] + content: + data: "Content using default_volumes" + + proxy-response: + status: 200 + headers: + fields: + - [X-Default-Volumes-Test, { value: "success", as: equal }] + + # Test 2: Request with explicit @volume=1 + - client-request: + method: "GET" + version: "1.1" + url: "/volume1_test" + headers: + fields: + - [Host, volume1.example.com] + - [X-Test-ID, "volume1-override-test"] + - [uuid, volume1-miss] + + server-response: + status: 200 + reason: OK + headers: + fields: + - [Content-Type, "text/plain"] + - [Cache-Control, "max-age=3600"] + - [X-Volume-Override, "volume1"] + content: + data: "Content explicitly on volume 1" + + proxy-response: + status: 200 + headers: + fields: + - [X-Volume-Override, { value: "volume1", as: equal }] + + # Test 3: Cache hit - verify content cached via default_volumes is served + - client-request: + delay: 100ms + method: "GET" + version: "1.1" + url: "/default_test" + headers: + fields: + - [Host, default.example.com] + - [X-Test-ID, "cache-hit-verify"] + - [uuid, default-hit] + + # Server should not be contacted for a cache hit + server-response: + status: 404 + reason: "Should not reach server" + + proxy-response: + status: 200 + content: + data: "Content using default_volumes" + verify: { as: equal } diff --git a/tests/gold_tests/cache/cache_volume_defaults.test.py b/tests/gold_tests/cache/cache_volume_defaults.test.py new file mode 100644 index 0000000000..8e6cde2f85 --- /dev/null +++ b/tests/gold_tests/cache/cache_volume_defaults.test.py @@ -0,0 +1,24 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +Test.Summary = ''' +Test proxy.config.cache.default_volumes configuration: +- Verify that default_volumes is used as fallback when no other volume selection applies +- Verify that @volume= directive takes priority over default_volumes +- Verify that hosting.config takes priority over default_volumes +''' + +Test.ATSReplayTest(replay_file="cache_volume_defaults.replay.yaml") diff --git a/tests/gold_tests/cache/cache_volume_features.replay.yaml b/tests/gold_tests/cache/cache_volume_features.replay.yaml new file mode 100644 index 0000000000..6c7c815427 --- /dev/null +++ b/tests/gold_tests/cache/cache_volume_features.replay.yaml @@ -0,0 +1,141 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +meta: + version: "1.0" + +# This test proves that multi-parameter volume.config lines are parsed +# correctly. The generated volume.config lines are: +# +# volume=1 scheme=http size=50% ram_cache_size=32M ram_cache_cutoff=8K +# volume=2 scheme=http size=50% +# +# All 5 key=value pairs on line 1 must be parsed from the single line. +# If the inner loop's "tmp = line_end" advancement were broken (as claimed +# in a PR review), only "volume=1" would be parsed, scheme and size would +# be missing, and the volume would be rejected -- disabling the cache. +autest: + description: "Test cache volume features: per-volume RAM cache and @volume= directive" + + dns: + name: "dns" + + server: + name: "server" + + client: + name: "client" + + ats: + name: "ts" + + records_config: + proxy.config.diags.debug.enabled: 1 + proxy.config.diags.debug.tags: "cache|cache_hosting|ram_cache" + proxy.config.cache.ram_cache.size: 128M + proxy.config.cache.ram_cache_cutoff: 4K + proxy.config.http.insert_response_via_str: 0 + proxy.config.cache.enable_read_while_writer: 0 + + volume_config: + # Volume 1 with all new parameters — 5 key=value pairs on one line + - volume: 1 + scheme: "http" + size: "50%" + ram_cache_size: "32M" + ram_cache_cutoff: "8K" + # Volume 2 — simple line, no extra params + - volume: 2 + scheme: "http" + size: "50%" + + log_validation: + traffic_out: + excludes: + # No volume.config lines should be discarded + - expression: "discarding.*entry at line" + description: "No volume.config entries should be discarded" + + remap_config: + - from: "http://volume1.example.com/"; + to: "http://backend.ex:{SERVER_HTTP_PORT}/"; + options: + - "@volume=1" + - from: "http://volume2.example.com/"; + to: "http://backend.ex:{SERVER_HTTP_PORT}/"; + options: + - "@volume=2" + +# All transactions in a single session for sequential execution. +sessions: +- transactions: + + # Test 1: Request through volume 1 with per-volume RAM cache settings + - client-request: + method: "GET" + version: "1.1" + url: "/test1" + headers: + fields: + - [Host, volume1.example.com] + - [X-Test-ID, "volume1-ramcache-test"] + - [uuid, 1] + + server-response: + status: 200 + reason: OK + headers: + fields: + - [Content-Type, "text/plain"] + - [Cache-Control, "max-age=3600"] + - [X-Volume-Test, "1"] + content: + data: "Content for volume 1 with custom RAM cache" + + proxy-response: + status: 200 + headers: + fields: + - [X-Volume-Test, { value: "1", as: equal }] + + # Test 2: Request through volume 2 (simple volume, no extra params) + - client-request: + method: "GET" + version: "1.1" + url: "/test2" + headers: + fields: + - [Host, volume2.example.com] + - [X-Test-ID, "volume2-test"] + - [uuid, 2] + + server-response: + status: 200 + reason: OK + headers: + fields: + - [Content-Type, "text/plain"] + - [Cache-Control, "max-age=3600"] + - [X-Volume-Test, "2"] + content: + data: "Content for volume 2" + + proxy-response: + status: 200 + headers: + fields: + - [X-Volume-Test, { value: "2", as: equal }] + diff --git a/tests/gold_tests/cache/cache_volume_features.test.py b/tests/gold_tests/cache/cache_volume_features.test.py new file mode 100644 index 0000000000..c52575ef56 --- /dev/null +++ b/tests/gold_tests/cache/cache_volume_features.test.py @@ -0,0 +1,25 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +Test.Summary = ''' +Comprehensive test suite for cache volume features: +- Per-volume RAM cache configuration (ram_cache_size, ram_cache_cutoff) +- @volume= directive in remap.config for volume selection +- Integration between both features +''' +# TODO: hosting.config + @volume= priority interaction test requires ats_replay.test.ext to support volume_config/hosting_config. + +Test.ATSReplayTest(replay_file="cache_volume_features.replay.yaml")
