Sam3

from nobg import Sam3
from nobg.sam3.modeling_sam3 import Sam3Config

Wraps transformers' Sam3Model (the Apache-2.0 reimplementation of SAM 3) and collapses its prompt-conditioned segmentation into a single alpha matte, so the output matches BiRefNet's. The instance-level outputs pass through untouched.

feyninc/multimatte — MultiMatte — is the published checkpoint, and AutoModel resolves this class from its nobg-sam3 tag.

Sam3Config

The three nobg-specific fields turn a promptable detector into a background remover; the rest are forwarded to the transformers SAM3 sub-configs.

Architecture fields, forwarded to the sub-configs:

__post_init__ raises if aggregate is not one of the three values, or if image_size is not divisible by vision_patch_size.

Choosing aggregate

Measured on facebook/sam3 against FeyNobg, on two photos:

Per-query mask logits are trained to be binarized at 0.5, so as soft alpha they come out much mushier. Under "max"/"mean", queries are scored the way SAM3's own instance post-processing scores them (pred_logits.sigmoid() * presence.sigmoid()), aggregation stays in logit space, and if nothing clears the threshold the single best-scoring query is kept — so the matte is never empty.

forward

forward(
    pixel_values,
    input_ids=None,
    attention_mask=None,
    input_boxes=None,
    input_boxes_labels=None,
    labels=None,
    score_threshold=None,
    **kwargs,
) -> dict

Returns:

forward raises a clear ValueError when input_ids is missing or pixel_values is not config.image_size square, rather than failing deep inside attention.

predict

predict(
    processor, image, prompt=None, boxes=None, *,
    score_threshold=None, batch_size=1, return_type="cutout", **processor_kwargs,
)

Positional order is the fixed argument first, then the varying ones by how often they are supplied. prompt applies to every image in the call; boxes is per-image, in the original image's pixel coordinates. See Text and box prompts.

process

process(
    image, prompt=None, boxes=None, *,
    tokenizer=None, score_threshold=None, batch_size=1, return_type="cutout", **processor_kwargs,
)

predict with the processor default_processor(tokenizer) describes.

default_processor

default_processor(tokenizer=None) -> Sam3Processor

The image side follows config.image_size and the prompt-free default follows config.default_prompt, but SAM3 also needs a text tokenizer, which no config.json describes. It is resolved in order:

Raises OSError if no candidate loads, and ValueError if the resolved tokenizer's vocabulary is larger than config.text_vocab_size — i.e. it can emit token ids the text embedding has no row for. The no-argument result is cached on the model.

from_origin

@classmethod
from_origin(origin, config=None, *, token=None, **overrides) -> Sam3

Reads an upstream transformers SAM3 checkpoint (nested config, upstream weight keys) or another nobg Sam3 — a repo id, a local directory or a live module — and produces a nobg Sam3, injecting every weight whose key and shape still match. It also records where the weights came from, so default_processor() can look for a tokenizer there.

model = Sam3.from_origin("facebook/sam3")                          # convert Meta's weights
model = Sam3.from_origin("feyninc/multimatte", aggregate="max")     # re-configure MultiMatte

This is also the only loader that takes config-field overrides: from_pretrained passes the checkpoint's own config and nothing else. Passing both an explicit config and overrides raises ValueError.

criterion

Defaults to nobg.loss.sam3_loss — SAM 3's own semantic-segmentation objective (weighted focal + dice) rather than BiRefNet's BCE+IoU+SSIM, so fine-tuning matches how the checkpoint was trained. Sam3 always passes a single-element scaled_preds list. See Losses.

ONNX hooks

• onnx_dynamo = False — the TorchScript tracer, because SAM3's decoder sizes tensors from data, which torch.export refuses to guard on.
• onnx_dummy_inputs(batch_size) adds input_ids and attention_mask (a synthetic 32-token prompt) to pixel_values. input_boxes is deliberately left out, since including it would make boxes mandatory for every call.

See ONNX.