Medium

bloom-3b specs, VRAM requirements, and which GPUs can run it.

bloom-7b1 specs, VRAM requirements, and which GPUs can run it.

bloomz-3b specs, VRAM requirements, and which GPUs can run it.

bloomz-7b1 specs, VRAM requirements, and which GPUs can run it.

bloomz-7b1-mt specs, VRAM requirements, and which GPUs can run it.

CodeLlama-7b-Instruct-hf specs, VRAM requirements, and which GPUs can run it.

deep-ignorance-unfiltered specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-6.7b-base specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-6.7b-instruct specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-7b-base-v1.5 specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-7b-instruct-v1.5 specs, VRAM requirements, and which GPUs can run it.

DeepSeek-R1-0528-Qwen3-8B specs, VRAM requirements, and which GPUs can run it.

DeepSeek-R1-Distill-Qwen-7B specs, VRAM requirements, and which GPUs can run it.

falcon-11B specs, VRAM requirements, and which GPUs can run it.

falcon-7b-instruct specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-3B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-7B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

falcon-mamba-7b-instruct specs, VRAM requirements, and which GPUs can run it.

Falcon3-10B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-3B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon3-7B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Flex-reddit-2x7B-1T specs, VRAM requirements, and which GPUs can run it.

gemma-1.1-7b-it specs, VRAM requirements, and which GPUs can run it.

gemma-2-9b-it specs, VRAM requirements, and which GPUs can run it.

GLM-4.7-Flash-MLX-6bit specs, VRAM requirements, and which GPUs can run it.

GLM-4.7-Flash-MLX-8bit specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Pro-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Pro-Mistral-7B specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Theta-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Hermes-3-Llama-3.1-8B specs, VRAM requirements, and which GPUs can run it.

internlm2_5-7b specs, VRAM requirements, and which GPUs can run it.

internlm2-chat-7b-sft specs, VRAM requirements, and which GPUs can run it.

Jan-v3-4B-base-instruct specs, VRAM requirements, and which GPUs can run it.

LFM2-8B-A1B specs, VRAM requirements, and which GPUs can run it.

Llama 3.1 8B specs, VRAM requirements, and which GPUs can run it. The go-to small model for local inference.

Llama-2-7b-hf specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-8B-Instruct-FP8 specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Tulu-3-8B-SFT specs, VRAM requirements, and which GPUs can run it.

Llama-3.2-3B specs, VRAM requirements, and which GPUs can run it.

Llama-Guard-3-8B specs, VRAM requirements, and which GPUs can run it.

Llama-Guard-3-8B-INT8 specs, VRAM requirements, and which GPUs can run it.

LlamaGuard-7b specs, VRAM requirements, and which GPUs can run it.

llm-jp-3-3.7b-instruct specs, VRAM requirements, and which GPUs can run it.

LocoOperator-4B specs, VRAM requirements, and which GPUs can run it.

maira-2 specs, VRAM requirements, and which GPUs can run it.

MediPhi-Clinical specs, VRAM requirements, and which GPUs can run it.

MediPhi-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct-bnb-4bit specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct-FP8 specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-Guard-2-8B specs, VRAM requirements, and which GPUs can run it.

Mistral 7B specs, VRAM requirements, and which GPUs can run it. Efficient and fast for everyday tasks.

Mistral-7B-Instruct-v0.2 specs, VRAM requirements, and which GPUs can run it.

mistral-7b-v0.3-bnb-4bit specs, VRAM requirements, and which GPUs can run it.

Mistral-NeMo-Minitron-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Nanbeige4.1-3B specs, VRAM requirements, and which GPUs can run it.

Nanbeige4.1-3B-heretic specs, VRAM requirements, and which GPUs can run it.

Nemotron-H-4B-Base-8K specs, VRAM requirements, and which GPUs can run it.

Nemotron-H-4B-Instruct-128K specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-2-Mistral-7B-DPO specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-2-SOLAR-10.7B specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-llama-2-7b specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2 specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-Base specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-FP8 specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-Japanese specs, VRAM requirements, and which GPUs can run it.

OLMo-2-1124-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Olmo-3-1025-7B specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct-DPO specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct-SFT specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think-DPO specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think-SFT specs, VRAM requirements, and which GPUs can run it.

Olmo-3.1-7B-RL-Zero-Math specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-0724-hf specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-hf specs, VRAM requirements, and which GPUs can run it.

Olmo-Hybrid-Instruct-DPO-7B specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0125 specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0125-Instruct specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0924-Instruct specs, VRAM requirements, and which GPUs can run it.

Phi-3-mini-4k-instruct-gptq-4bit specs, VRAM requirements, and which GPUs can run it.

Phi-3-small-8k-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-mini-MoE-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-tiny-MoE-instruct specs, VRAM requirements, and which GPUs can run it.

polyglot-ko-5.8b specs, VRAM requirements, and which GPUs can run it.

pythia-12b specs, VRAM requirements, and which GPUs can run it.

pythia-6.9b specs, VRAM requirements, and which GPUs can run it.

Qwen2-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-3B specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-7B specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct-GPTQ-Int4 specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-VL-7B-Instruct-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3-14B-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-Instruct-2507-FP8 specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-SafeRL specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-Base specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-FP8 specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3.5-4B-Safety-Thinking specs, VRAM requirements, and which GPUs can run it.

Qwen3.5-9B-abliterated specs, VRAM requirements, and which GPUs can run it.

Qwen3Guard-Gen-4B specs, VRAM requirements, and which GPUs can run it.

Qwen3Guard-Gen-8B specs, VRAM requirements, and which GPUs can run it.

saiga_llama3_8b specs, VRAM requirements, and which GPUs can run it.

SOLAR-10.7B-v1.0 specs, VRAM requirements, and which GPUs can run it.

stablelm-base-alpha-7b-v2 specs, VRAM requirements, and which GPUs can run it.

Starling-LM-7B-beta specs, VRAM requirements, and which GPUs can run it.

steerling-8b specs, VRAM requirements, and which GPUs can run it.

tiny-aya-global specs, VRAM requirements, and which GPUs can run it.

wildguard specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-6B specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-6B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-32K specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-Chat-16K specs, VRAM requirements, and which GPUs can run it.

Yi-6B specs, VRAM requirements, and which GPUs can run it.

Yi-6B-200K specs, VRAM requirements, and which GPUs can run it.

Yi-6B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-9B specs, VRAM requirements, and which GPUs can run it.

Yi-9B-200K specs, VRAM requirements, and which GPUs can run it.

Yi-Coder-9B specs, VRAM requirements, and which GPUs can run it.

Yi-Coder-9B-Chat specs, VRAM requirements, and which GPUs can run it.

zephyr-7b-beta specs, VRAM requirements, and which GPUs can run it.