Logit Bias
Sometimes you want to force the model to include or avoid specific words. Logit bias lets you adjust the probability of individual tokens before sampling. It’s a powerful but advanced parameter.
Logit bias = add a number to the logit (score) of a specific token, making it more or less likely.
How It Works
The model computes a raw score (logit) for every possible next token. Logit bias adds a value to that score before applying softmax. Positive bias → more likely; negative bias → less likely.
Example: Avoiding the Word "however"
If the model overuses "however", you can reduce its probability. First, find the token ID for " however" (note the space).
logit_bias = {token_id: -100} # strongly discourageForcing a Specific Word
To force a word like "Python" at a certain point, give it a large positive bias (e.g., +100). However, forcing may break grammar.
Finding Token IDs
Different models use different tokenizers. OpenAI provides a tokenizer tool; for other models, you can use their tokenizer libraries.
Practical Use Cases
- Block offensive or banned words (set bias to -100).
- Encourage specific terms in domain‑specific tasks.
- Prevent the model from repeating a common error.
- Steer output without changing the prompt.
Caution
Logit bias is delicate. Too strong a bias can make output unnatural. Start with small adjustments (±5 to ±20). Also, it only works on exact token matches, not phrases.
Two Minute Drill
- Logit bias adjusts probability of specific tokens.
- Positive bias → more likely; negative bias → less likely.
- Useful for blocking words or encouraging domain terms.
- Requires token IDs; use small bias values to start.
Need more clarification?
Drop us an email at career@quipoinfotech.com