Remove objects from videos with Netflix VOID

VOID runs on Netflix's code plus several sets of weights: the CogVideoX base model, its own two passes and SAM 2.

1. 1On your GPU machine, open a terminal in a fresh Python 3.12 environment.
2. 2Run the commands below. They clone VOID, install SAM 2 and SAM 3, and download the weights.
3. 3Give it time. The CogVideoX base model is the slow download.

git clone https://github.com/Netflix/void-model.git && cd void-model
pip install -r requirements.txt
git clone https://github.com/facebookresearch/sam2.git && (cd sam2 && pip install -e .)
git clone https://github.com/facebookresearch/sam3.git && (cd sam3 && pip install -e .)
hf download alibaba-pai/CogVideoX-Fun-V1.5-5b-InP --local-dir ./CogVideoX-Fun-V1.5-5b-InP
hf download netflix/void-model void_pass1.safetensors void_pass2.safetensors --local-dir .
wget https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_large.pt

You should see CogVideoX-Fun-V1.5-5b-InP, void_pass1.safetensors, void_pass2.safetensors and sam2_hiera_large.pt inside the void-model folder.

SAM 3 downloads its weights from Hugging Face, and the mask step asks Gemini what else the object affects.

1. 1On the SAM 3 model page, request access and wait for approval.
2. 2Create a Hugging Face access token. Run hf auth login and paste it.
3. 3Create a Gemini API key in Google AI Studio and export it, as below.
4. 4Run the sed line. It swaps the repo's retired Gemini model for Gemini 3.1 Pro.

hf auth login
hf auth whoami
export GEMINI_API_KEY=[YOUR GEMINI API KEY]
sed -i 's/gemini-3-pro-preview/gemini-3.1-pro-preview/' VLM-MASK-REASONER/stage2_vlm_analysis.py

You should see your Hugging Face username, printed by hf auth whoami.

Google shut down gemini-3-pro-preview on March 9, 2026, but the VOID repo still names it. Keep both keys out of scripts and shared files.

A few clicks tell SAM 2 which object to follow through the whole clip.

1. 1Put your clip in a clips folder. Save the config below as my_config.json in the void-model folder.
2. 2Run python VLM-MASK-REASONER/point_selector_gui.py, click Load Config and pick my_config.json.
3. 3Click a few points on the object. If part of it first shows up later, such as a hand, go to that frame and click it too.
4. 4Click Save All Points.

{
  "videos": [
    {
      "video_path": "clips/[YOUR CLIP].mp4",
      "output_dir": "data/my-clip",
      "instruction": "remove the [OBJECT]"
    }
  ]
}

You should see a message that the points were saved to my_config_points.json.

The mask marks the object and everything it touches, such as a cup it would have knocked over.

1. 1Run the pipeline below. It tracks the object, asks Gemini what the object affects, and writes the mask.
2. 2Play data/my-clip/quadmask_0.mp4. Black is the object, grey is what it affects, and white stays as it is.
3. 3If a shadow or a falling item is missing, run python VLM-MASK-REASONER/edit_quadmask.py and paint it in.

bash VLM-MASK-REASONER/run_pipeline.sh my_config_points.json --sam2-checkpoint sam2_hiera_large.pt

You should see Pipeline complete!, with input_video.mp4 and quadmask_0.mp4 in data/my-clip.

Pass 1 fills in the masked areas and replays what the object caused, so a held guitar falls when the person is gone.

1. 1Save a prompt.json that describes the scene after the removal, not the removal itself.
2. 2Run pass 1 with the command below.
3. 3Open the file ending in _tuple.mp4 in out/pass1 to compare before and after side by side.

echo '{"bg": "[THE SCENE WITHOUT THE OBJECT]"}' > data/my-clip/prompt.json
python inference/cogvideox_fun/predict_v2v.py \
  --config config/quadmask_cogvideox.py \
  --config.data.data_rootdir="data" \
  --config.experiment.run_seqs="my-clip" \
  --config.experiment.save_path="out/pass1" \
  --config.video_model.model_name="CogVideoX-Fun-V1.5-5b-InP" \
  --config.video_model.transformer_path="void_pass1.safetensors"

You should see out/pass1/my-clip.mp4, with the object and its effects gone.

Netflix's own samples show the idea: with a glass removed, the prompt is "A lime falls on the table."

Small video models can bend thin or fast things, and pass 2 uses the motion from pass 1 to hold shapes steady.

1. 1Watch hands, rackets and other thin or fast parts in the pass 1 clip.
2. 2If they bend or melt, run pass 2 with the command below.

python inference/cogvideox_fun/inference_with_pass1_warped_noise.py \
  --video_name my-clip \
  --data_rootdir data \
  --pass1_dir out/pass1 \
  --output_dir out/pass2 \
  --model_checkpoint void_pass2.safetensors \
  --model_name CogVideoX-Fun-V1.5-5b-InP

You should see a steadier version of your clip in out/pass2.