Editor

the base class for Knowledge Editing

BaseEditor

BaseEditor is the class for factual and generation editing. Given the edit descriptor and the edit target, you can use different editing methods to change the behavior of the model.

from_hparams() -> BaseEditor

Static method, editor can be initialized through this function

def from_hparams(cls, hparams: HyperParams):
    return cls(hparams)

• Paramters

  • hparams(Hyperparams): hyperparameters for editing method

• Return Type

  • editor(BaseEditor): The Editor class defined by hparams

edit()-> List[Dict]

Main function: do fact editing according to the selected editing method

def edit(self,
    prompts: Union[str, List[str]],
    target_new: Union[str, List[str]],
    ground_truth: Optional[Union[str, List[str]]],
    rephrase_prompts: Optional[Union[str, List[str]]] = None,
    locality_inputs:  Optional[Dict] = None,
    portability_inputs: Optional[Dict] = None,
    keep_original_weight=False,
    verbose=True,
    **kwargs
    )

• Paramters

  • prompts(Union[str, List[str]]): The prompt string of edit descriptor

  • target_new(Union[str, List[str]]): The prompt string of edit target

  • ground_truth(Optional[Union[str, List[str]]]): The original model output of the edit descriptor(you can set it None)

  • rephrase_prompts(Optional[Union[str, List[str]]]): The rephrase prompt string, semantically similar to prompts, in order to test Generalization

  • locality_inputs(Optional[Dict]): For each measurement dimension, you need to provide the corresponding prompt and its corresponding ground truth. Test Locality.

  • portability_inputs(Optional[Dict]): Similar to locality_inputs, in order to test Portability

  • keep_original_weight(bool): whether to edit sequentially

    • False: edit sequentially(because the original weight is not maintained after each edit)

    • True: not edit sequentially

  • verbose(bool): whether to print intermediate output

• Return Type

  • metrics(List[Dict]): the metric for model editing(see this link for more details)

  • edited_model(PreTrainedModel): model weights after editing

batch_edit()->List[Dict]

Main function: do fact editing according to the selected editing method

def batch_edit(self,
    prompts: List[str],
    target_new: List[str],
    ground_truth: Optional[List[str]] = None,
    rephrase_prompts: Optional[List[str]] = None,
    locality_prompts: Optional[List[str]] = None,
    locality_ground_truth: Optional[List[str]] = None,
    keep_original_weight=False,    
    verbose=True,
    **kwargs
    )

• Paramters

  • prompts(Union[str, List[str]]): The prompt string of edit descriptor

  • target_new(Union[str, List[str]]): The prompt string of edit target

  • ground_truth(Optional[Union[str, List[str]]]): The original model output of the edit descriptor(you can set it None)

  • rephrase_prompts(Optional[Union[str, List[str]]]): The rephrase prompt string, semantically similar to prompts, in order to test Generalization

  • locality_inputs(Optional[Dict]): For each measurement dimension, you need to provide the corresponding prompt and its corresponding ground truth. Test Locality.

  • portability_inputs(Optional[Dict]): Similar to locality_inputs, in order to test Portability

  • keep_original_weight(bool): whether to edit sequentially

    • False: edit sequentially(because the original weight is not maintained after each edit)

    • True: not edit sequentially

• Return Type

  • metrics(List[Dict]): the metric for model editing(see this link for more details)

  • edited_model(PreTrainedModel): model weights after editing

Example

hparams = MENDHyperParams.from_hparams('./hparams/MEND/gpt2-xl')
editor = BaseEditor.from_hparams(hparams)
metrics, edited_model, weight_copy= editor.edit(
        prompts='What university did Watts Humphrey attend?' if prompts is None else prompts,
        ground_truth='Illinois Institute of Technology' if ground_truth is None else ground_truth,
        target_new='University of Michigan' if target_new is None else target_new,
        keep_original_weight=True,
)