prompting.validators.reward.blacklist#

Module Contents#

Classes#

BlacklistRewardEvent

Blacklist

class prompting.validators.reward.blacklist.BlacklistRewardEvent#

Bases: prompting.validators.reward.reward.BaseRewardEvent

matched_ngram: str#
significance_score: float#
class prompting.validators.reward.blacklist.Blacklist(boundary=6, n_min=5, n_max=14, word_limit=2000, A=1.3, preprocess='[^(\\w|\\s)]', partial_ratio_boundary=95, half_life=20000, support=0.01, error=0.001, memory_lim=1000000, frequency_multiplier=100)#

Bases: prompting.validators.reward.reward.BaseRewardModel

Parameters:

  • boundary (float) –

  • n_min (int) –

  • n_max (int) –

  • word_limit (int) –

  • A (float) –

  • preprocess (str) –

  • partial_ratio_boundary (float) –

  • half_life (int) –

  • support (float) –

  • error (float) –

  • memory_lim (int) –

  • frequency_multiplier (float) –

property name: str#
Return type:

str

add(texts)#

Extract and add n-grams from a list of texts to counter

Parameters:

texts (list) – batch of completion texts

extract_ngrams(text)#

Extract n-grams from text string

Parameters:

text (str) – completion text

Returns:

List of n-gram tuples

Return type:

list

_add_ngrams(ngrams)#

Adds n-grams to counter, removing old n-grams periodically. Counting and pruning method based on Lossy counter. Reference: https://files.ifi.uzh.ch/dbtg/sdbs13/T01.3.pdf

Parameters:

ngrams (List[tuple]) – List of n-gram tuples

prune()#

Prune the counter when the count is smaller then bucket index.

reset()#

Reset counters to initial values.

calculate_significance()#

Calculate significance of all n-grams in counter. By construction, n-grams with count 1 will have significance 0.

Returns:

Dictionary of n-gram tuples and their significance scores

Return type:

dict

get_significance()#

Get significance scores, only recalculating if the counter has been updated.

Returns:

Dictionary of n-gram tuples and their significance scores

Return type:

dict

most_common(n=10)#

Get most common n-grams in queue

Parameters:

n (int) – Number of most common n-grams to return. Defaults to 10.

Returns:

Sorted dictionary of n-gram tuples and their counts

Return type:

dict

most_significant(n=10, force_update=True)#

Get most significant n-grams in queue based on significance scores

Parameters:

  • n (int, optional) – Number of most significant n-grams to return. Defaults to 10.

  • force_update (bool, optional) – Force recalculate the significance scores. Defaults to True.

Returns:

Sorted dictionary of n-gram tuples and their significance scores

Return type:

dict

set_counter_to_half()#

Set all the counters to half for a rolling window effect.

reward(prompt, completion, name)#

Reward function for blacklist reward model. Returns 1 if completion contains an n-gram with significance above the boundary, 0 otherwise.

Parameters:

  • prompt (str) – Prompt text

  • completion (str) – Completion text

  • name (str) – Name of the validation step

Returns:

Reward value {0,1}

Return type:

float

get_rewards(prompt, completions, name)#
Parameters:

  • prompt (str) –

  • completions (List[str]) –

  • name (str) –

Return type:

List[BlacklistRewardEvent]

normalize_rewards(rewards)#

This method normalizes the given rewards by updating the moving mean and variance statistics. The rewards are first standardized, and then scaled to the 0-1 range using a cumulative distribution function (CDF) to ensure they’re in a comparable range across different environments.

Args: rewards (torch.FloatTensor): The reward values to be normalized.

Returns: torch.FloatTensor: The normalized reward values.

Note: - This function uses Welford’s online algorithm to update the mean and variance. - It standardizes the reward values using the updated mean and variance. - It then scales the standardized values to the 0-1 range using the error function (erf) as a CDF.

Parameters:

rewards (torch.FloatTensor) –

Return type:

torch.FloatTensor