[Wikidata-bugs] [Maniphest] [Commented On] T173696: Cache constraint check results

[aaron]

aaron added a comment.

If want to avoid flooding cache with rarely used long-tail combinations, maybe something like this could be done:

    $textHash = hash( 'sha256', $text );
    $cacheMap = $this->cache->getWithSetCallback(
		$this->cache->makeKey(
			'WikibaseQualityConstraints', // extension
			'regex', // action
			'WDQS-Java', // regex flavor
			hash( 'sha256', $regex )
		),
		WANObjectCache::TTL_DAY,
		function ( $curCacheMap ) use ( $text, $regex, $textHash ) {
			// Initialize the cache map if not set
			if ( $curCacheMap === false ) {
				return [];
			}
			// Refresh triggered by hotTTR...
			// Add regex-text check result to top of the LRU map if present
			if ( isset( $curCacheMap[$textHash] ) ) {
				return [ $textHash => $curCacheMap[$textHash] ] + $curCacheMap;
			}
			// Get the regex-text check result
			$value = (int)$this->matchesRegularExpressionWithSparql( $text, $regex );
			// Add regex-text check to the bottom 3/8s of the LRU map
			$index = intval( count( $curCacheMap ) * 5/8 );
			$newCacheMap = [];
			$pos = 0;
			foreach ( $curCacheMap as $k => $v ) {
				if ( $pos == $index ) {
					$newCacheMap[$textHash] = $value; // inject
					++$pos;
				}
				if ( $pos++ >= 100 ) {
					break; // prune to size
				}
				$newCacheMap[$k] = $v; // preserve
			}
				
			return $newCacheMap;
		},
		[ 
			// Once map is > 1 sec old, consider refreshing
			'ageNew' => 1,
			// Increase likely of refresh to certainty once 1 minute old;
			// the most common keys are more likely to trigger this
			'hotTTR' => 60,
			// avoid querying cache servers multiple times in a request
			'pcTTL' => $cache::PROC_TTL_LONG
		]
    );
    
    $value = isset( $cacheMap[$textHash] ) 
        ? $cacheMap[$textHash]
        : (int)$this->matchesRegularExpressionWithSparql( $text, $regex );

TASK DETAIL

https://phabricator.wikimedia.org/T173696

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: aaron
Cc: Krinkle, aaron, gerritbot, Ladsgroup, daniel, Aklapper, Jonas, Lucas_Werkmeister_WMDE, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Agabi10, Izno, Wikidata-bugs, aude, Mbch331

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs