Contact emails [email protected], [email protected]
Explainer https://github.com/webmachinelearning/prompt-api#sampling-parameters Specification https://webmachinelearning.github.io/prompt-api Summary Allows developers to configure language model output variety and creativity in the Prompt API using high-level samplingMode presets. Instead of exposing raw numerical parameters like topK and temperature that vary unpredictably across models, developers can choose semantic presets ranging from "most-predictable" for deterministic extraction to "most-creative" for open-ended brainstorming. The browser maps these presets to optimal sampling hyperparameters for the underlying model, providing developers with reliable response tuning while preserving cross-browser interoperability. enum LanguageModelSamplingMode { "most-predictable", // For strict consistency/factual extraction "predictable", // For highly focused outputs "slightly-predictable", // For moderately focused, consistent outputs "balanced", // The default state for standard prompting "slightly-creative", // For moderately varied, expressive outputs "creative", // For tasks favoring variety over strict facts "most-creative" // For maximum token diversity and brainstorming }; This feature is an enhancement of the Prompt API (https://chromestatus.com/feature/5134603979063296). Blink component Blink>AI>Prompt Web Feature ID languagemodel Motivation Web developers using the Prompt API often need to steer response variance to match their task, such as deterministic decoding for structured data extraction, classification, and tool calling, or higher output diversity for open-ended drafting and brainstorming. Raw numerical hyperparameters (`temperature` and `topK`) were excluded from the initial Prompt API launch due to cross-browser and cross-model interoperability concerns raised during TAG review (https://github.com/w3ctag/design-reviews/issues/1093), leaving web applications with no way to tune output variability beyond default session behavior. `samplingMode` bridges this gap with seven semantic presets (`"most-predictable"` through `"most-creative"`) that express developer intent without coupling web code to a single model's hyperparameter scale. User agents map each preset to model-specific sampling configurations and can apply mode-specific inference optimizations (such as greedy speculative decoding in `"most-predictable"` sessions). Live interactive demo: https://isaacahouma.github.io/prompt-api-sampling-presets/ Initial public proposal https://github.com/webmachinelearning/prompt-api/issues/203 TAG review https://github.com/w3ctag/design-reviews/issues/1093 This update directly addresses the TAG's Prompt API early design review feedback on “Interoperable model parameters”. We adopted TAG guidance by replacing raw hyperparameters with samplingMode presets, which enable user agents to map semantic developer intent to model-specific sampling configurations. Because this incremental refinement directly fulfills prior TAG advice rather than a new architectural surface, a separate review is not merited. TAG review status Issues addressed Origin Trial Name Prompt API Sampling Parameters Goals for experimentation topK and temperature parameters were excluded from the initial Prompt API launch, due to interoperability concerns. Developers have expressed value in tuning parameters, for testing and use-case specific optimization. Our primary goal during this extension is to gather real-world usage data on the newly expanded categorical presets to see which modes developers gravitate toward most. We will use this data and developer feedback to conduct more concerted medium-term mode evaluations. Specifically, we want to evaluate developer adoption of this expanded spectrum, and validate that the new granularity effectively covers the previously identified dead zones. Chromium Trial Name AIPromptAPIParams Origin Trial documentation link https://github.com/webmachinelearning/prompt-api?tab=readme-ov-file#sampling-parameters WebFeature UseCounter name kLanguageModel_Create_Temperature Risks Interoperability and Compatibility The original raw numerical parameters (topK and temperature) were excluded from the initial Prompt API launch due to cross-browser and cross-model interoperability concerns raised during TAG review (https://github.com/w3ctag/design-reviews/issues/1093). Replacing raw parameters with semantic samplingMode presets lets each browser map the developer's intent to appropriate model-specific sampling configurations. Based on Origin Trial feedback, the enum was refined from 5 to 7 presets to provide finer granularity across the predictable and creative spectrum while preserving cross-browser interoperability. Gecko: Negative (https://github.com/mozilla/standards-positions/issues/1213) View reflects parent specification WebKit: Oppose (https://github.com/WebKit/standards-positions/issues/495) View reflects parent specification Web developers: Positive (https://github.com/webmachinelearning/prompt-api/issues/203#issuecomment-4676828673) Relative to the current shipped baseline (no sampling control on the web, since raw temperature and topK were excluded per TAG guidance), a significant fraction of developers support adding samplingMode to steer output variance, though a subset would still prefer the raw knobs: Developer Demand & Surveys: When asked to compare presets directly against raw parameters, 75% of parameter-tuning Early Preview Program (EPP) respondents accepted presets (55% preferred presets outright, 20% were open to them), while 25% would still prefer raw temperature / topK floats. Interoperable presets also address concerns from EPP sign-ups, where 33% flagged “Cross-Engine Prompt Fragility” (needing per-model tuning) and 22% cited “API Configuration Gaps” (eg inability to tune). The number of presets expanded from 5 to 7 options, partly in response to OT developer feedback around gaps in the predictable end of the spectrum. Telemetry: Across the modest samplingMode OT and the larger legacy (temperature & topK) Chrome extension usage, over 95% of tuned sessions opt for predictable outputs, with 75% choosing most-predictable and >60% setting equivalent parameters (temperature = 0 / topK = 1). Adoption Examples: Origin Trial partners (production e-commerce platforms, merchant website builders, and shopping extensions) and open-source projects rely on most-predictable and predictable (often paired with responseConstraint) for deterministic tasks such as catalog attribute extraction, merchant copywriting tools, intent classification, analysis of administrative procedures survey data by a governmental agency, and extraction of audio impairments from clinical report images. Other signals: The API implementation can deliver mode-specific inference speedups like Multi-Token Prediction (MTP) speculative decoding in most-predictable sessions (75% of parameter overrides, per our telemetry). This benefits users and developers by providing quicker responses while consuming less compute and power resources. Other signals: Edge: Positive (https://github.com/webmachinelearning/prompt-api/issues/203) Microsoft confirmed the API shape is interoperable, even if backends offer less granularity. They would have liked stronger signals regarding granularity, especially on the creative side. WebView application risks Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications? None Debuggability Developers can inspect the active sampling preset on any session via the session.samplingMode attribute in the DevTools Console, and invalid LanguageModelSamplingMode values throw a descriptive TypeError. Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)? No The Prompt API currently supports Windows, Mac, Linux, and ChromeOS. Is this feature fully tested by web-platform-tests? Yes Yes; the API surface is tested in https://wpt.fyi/results/ai/language-model (language-model-availability-sampling-mode.tentative.https.window.html and language-model-create-sampling-mode.tentative.https.window.html). Automated testing of sampling parameter effects on probabilistic model response is not readily feasible; instead we conduct rounds of evaluations on configuration updates. Flag name on about://flags prompt-api-sampling-mode Finch feature name AIPromptAPIParams Rollout plan Will ship enabled for all users Requires code in //chrome? True Tracking bug https://issues.chromium.org/573612811 Launch bug https://launch.corp.google.com/launch/4515786 Measurement Overall API calls are tracked by UseCounters (LanguageModel_Create and LanguageModel_Availability), while samplingMode adoption and per-preset distribution across create() and availability() are recorded in UMA histograms: AI.LanguageModel.APIUsage.{availability,create}.OptionsType AI.LanguageModel.APIUsage.{availability,create}.SamplingMode https://source.chromium.org/chromium/chromium/src/+/main:tools/metrics/histograms/metadata/ai/histograms.xml?q=AI.LanguageModel.APIUsage.*SamplingMode Estimated milestones Shipping on desktop 160 Origin trial desktop first 148 Origin trial desktop last 153 Origin trial extension 1 end milestone 159 Anticipated spec changes Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (eg links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (eg, changing to naming or structure of the API in a non-backward-compatible way). None Link to entry on the Chrome Platform Status https://chromestatus.com/feature/6325545693478912?gate=6484998367608832 Links to previous Intent discussions Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/4KvH5XEBYtE Intent to Extend Experiment 1: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFk4SLZXUY8FQTQQrtLHkzLnkxkgJ7XXFBhDEAQSvy6XiYKgaQ%40mail.gmail.com This intent message was generated by Chrome Platform Status. -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6af20fd7.7772a632.11fdae.052f.GAE%40google.com.
