Hparams

the hyperparams class for Knowledge Editing

HyperParams

Simple wrapper to store hyperparameters for Python-based rewriting methods.

from_json() -> HyperParams

Static method, hyperparams can be initialized through json

def from_json(cls, fpath: str):

• Paramters

  • fpath(Str): config path for editing

• Return Type

  • hparams(Hyperparams): The hyperparams class for editing

from_hparams() -> HyperParams

Static method, hyperparams can be initialized through yaml

def from_hparams(cls, hparams_name_or_path: str):

• Paramters

  • hparams_name_or_path(Str): yaml config path for editing

• Return Type

  • hparams(Hyperparams): The hyperparams class for editing

Example

hparams = MEMITHyperParams.from_hparams('./hparams/MEMIT/llama-7b.yaml')

• content

alg_name: "MEMIT"
model_name: "./hugging_cache/llama-2-7b"
device: 0
layers: [4, 5, 6, 7, 8]
clamp_norm_factor: 4
layer_selection: "all"
fact_token: "subject_last"
v_num_grad_steps: 25
v_lr: 5e-1
v_loss_layer: 31
v_weight_decay: 1e-3
kl_factor: 0.0625
mom2_adjustment: true
mom2_update_weight: 15000
rewrite_module_tmp: "model.layers.{}.mlp.down_proj"
layer_module_tmp: "model.layers.{}"
mlp_module_tmp: "model.layers.{}.mlp"
attn_module_tmp: "model.layers.{}.self_attn"
ln_f_module: "model.norm"
lm_head_module: "lm_head"
mom2_dataset: "wikipedia"
mom2_n_samples: 100000
mom2_dtype: "float32"