scdynomics.FT_Encoder

class scdynomics.FT_Encoder(*args: Any, **kwargs: Any)

Bases: Module

Fine-Tuning Encoder that adapts a pretrained model for downstream tasks.

This wrapper module adapts the pretrained Masked_Pretrainer encoder for downstream fine-tuning and classification tasks. It manages the injection of Parameter-Efficient Fine-Tuning (PEFT) mechanisms like LoRA and Adapters.

__init__(base_model=None, unfreeze_embedder: bool = False, unfreeze_embedder_target: str = 'sgt', lora_type: str = None, shortcut_type: str = None, pre_block_num: int = None, lora_latent_dim: int = 4, lora_dropout: float = 0.0, lora_bias: bool = False, lora_activation: str = 'relu', pool_method: str = None, pool_attn_hidden: int = 32, pool_out_size: int = 1, **kwargs)

Parameters:

base_model: Masked_Pretrainer (default: None)

The pretrained foundation model to build upon.

unfreeze_embedder: bool (default: False)

Whether to unfreeze the initial embedding layers.

unfreeze_embedder_target: str (default: "sgt")

Which sub-embedding to unfreeze: "sgt" (gene-token / position embedding), "val" (value-bin embedding), or "all".

lora_type: str (default: None)

The PEFT architecture to apply (“lora”, “adapter”).

shortcut_type: str (default: None)

Type of shortcut connection for adapters.

pre_block_num: int (default: None)

Number of pre-blocks to freeze or adapt.

lora_latent_dim: int (default: 4)

The intermediate rank size for the LoRA projection matrices.

lora_dropout: float (default: 0.0)

Dropout probability applied to the LoRA module.

lora_bias: bool (default: False)

If True, trains the bias terms in the LoRA modules.

lora_activation: str (default: 'relu')

Activation function for LoRA.

pool_method: str (default: None)

Sequence pooling technique (“avg”, “max”, “attn”).

pool_attn_hidden: int (default: 32)

Hidden dimension size used if pool_method is “attn”.

pool_out_size: int (default: 1)

Target output size for the pooling layer.

kwargs: dict

Additional keyword arguments.

forward(x, explaining: bool = False, **kwargs) → torch.Tensor

get_init_embedding(x) → torch.Tensor

Get the initial embedding for the input tensor x by applying the vbinnig and init_embedder layers.

Parameters:

x: torch.Tensor

The input tensor to be embedded.

Returns: torch.Tensor

The initial embedding of the input tensor.

unfreeze_embedder(mode: bool = False, target: str = 'all')

Toggle requires_grad on the init_embedder sub-modules.

Parameters:

mode: bool (default: False)

False freezes (requires_grad=False), True unfreezes. Naming mirrors Freezer.unfreeze_base_model.

target: str (default: "all")

Which sub-embedding to toggle: "sgt" (gene-token / position embedding), "val" (value-bin embedding), or "all". Unknown targets raise ValueError.