Medium

AI21-Jamba-Reasoning-3B specs, VRAM requirements, and which GPUs can run it.

Bielik-11B-v3.0-Instruct specs, VRAM requirements, and which GPUs can run it.

bloom-3b specs, VRAM requirements, and which GPUs can run it.

bloom-3b-intermediate specs, VRAM requirements, and which GPUs can run it.

bloom-7b1 specs, VRAM requirements, and which GPUs can run it.

bloom-7b1-intermediate specs, VRAM requirements, and which GPUs can run it.

bloomz-3b specs, VRAM requirements, and which GPUs can run it.

bloomz-7b1 specs, VRAM requirements, and which GPUs can run it.

bloomz-7b1-mt specs, VRAM requirements, and which GPUs can run it.

bloomz-7b1-p3 specs, VRAM requirements, and which GPUs can run it.

CodeLlama-7b-Instruct-hf specs, VRAM requirements, and which GPUs can run it.

CodeLlama-7b-Python-hf specs, VRAM requirements, and which GPUs can run it.

deep-ignorance-pretraining-stage-unfiltered specs, VRAM requirements, and which GPUs can run it.

deep-ignorance-unfiltered specs, VRAM requirements, and which GPUs can run it.

DeepHermes-3-Llama-3-8B-Preview specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-6.7b-base specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-6.7b-instruct specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-7b-base-v1.5 specs, VRAM requirements, and which GPUs can run it.

deepseek-coder-7b-instruct-v1.5 specs, VRAM requirements, and which GPUs can run it.

deepseek-math-7b-rl specs, VRAM requirements, and which GPUs can run it.

DeepSeek-R1-0528-Qwen3-8B specs, VRAM requirements, and which GPUs can run it.

DeepSeek-R1-Distill-Qwen-7B specs, VRAM requirements, and which GPUs can run it.

falcon-11B specs, VRAM requirements, and which GPUs can run it.

falcon-7b specs, VRAM requirements, and which GPUs can run it.

falcon-7b-instruct specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-3B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-7B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon-H1-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon-H1R-7B-FP8 specs, VRAM requirements, and which GPUs can run it.

falcon-mamba-7b-instruct specs, VRAM requirements, and which GPUs can run it.

falcon-rw-7b specs, VRAM requirements, and which GPUs can run it.

Falcon3-10B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-10B-Instruct-1.58bit specs, VRAM requirements, and which GPUs can run it.

Falcon3-3B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon3-7B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Falcon3-Mamba-7B-Base specs, VRAM requirements, and which GPUs can run it.

Falcon3-Mamba-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Flex-reddit-2x7B-1T specs, VRAM requirements, and which GPUs can run it.

Gemma 7B specs, VRAM requirements, and which GPUs can run it.

gemma-1.1-7b-it specs, VRAM requirements, and which GPUs can run it.

gemma-2-9b-it specs, VRAM requirements, and which GPUs can run it.

gemma-2b-AWQ specs, VRAM requirements, and which GPUs can run it.

gemma-4-E4B-it-OBLITERATED specs, VRAM requirements, and which GPUs can run it.

GLM-4.7-Flash-MLX-6bit specs, VRAM requirements, and which GPUs can run it.

GLM-4.7-Flash-MLX-8bit specs, VRAM requirements, and which GPUs can run it.

gpt-oss-20b-MXFP4-Q8 specs, VRAM requirements, and which GPUs can run it.

granite-3.3-8b-instruct specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Pro-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Pro-Mistral-7B specs, VRAM requirements, and which GPUs can run it.

Hermes-2-Theta-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Hermes-3-Llama-3.1-8B specs, VRAM requirements, and which GPUs can run it.

Hermes-3-Llama-3.2-3B specs, VRAM requirements, and which GPUs can run it.

internlm2_5-7b specs, VRAM requirements, and which GPUs can run it.

internlm2_5-7b-chat specs, VRAM requirements, and which GPUs can run it.

internlm2_5-7b-chat-1m specs, VRAM requirements, and which GPUs can run it.

internlm2-chat-7b-sft specs, VRAM requirements, and which GPUs can run it.

internlm2-math-7b specs, VRAM requirements, and which GPUs can run it.

internlm2-math-plus-7b specs, VRAM requirements, and which GPUs can run it.

Jan-v3-4B-base-instruct specs, VRAM requirements, and which GPUs can run it.

japanese-stablelm-base-gamma-7b specs, VRAM requirements, and which GPUs can run it.

japanese-stablelm-instruct-beta-7b specs, VRAM requirements, and which GPUs can run it.

japanese-stablelm-instruct-gamma-7b specs, VRAM requirements, and which GPUs can run it.

karma-electric-llama31-8b specs, VRAM requirements, and which GPUs can run it.

KD-Tinker specs, VRAM requirements, and which GPUs can run it.

LFM2-24B-A2B-MLX-4bit specs, VRAM requirements, and which GPUs can run it.

LFM2-24B-A2B-MLX-5bit specs, VRAM requirements, and which GPUs can run it.

LFM2-24B-A2B-MLX-6bit specs, VRAM requirements, and which GPUs can run it.

LFM2-24B-A2B-MLX-8bit specs, VRAM requirements, and which GPUs can run it.

LFM2-8B-A1B specs, VRAM requirements, and which GPUs can run it.

Llama 3.1 8B specs, VRAM requirements, and which GPUs can run it. The go-to small model for local inference.

Llama-2-7b-hf specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-8B-Instruct-FP8 specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Nemotron-Nano-4B-v1.1 specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Nemotron-Nano-8B-v1 specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Nemotron-Safety-Guard-8B-v3 specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Tulu-3-8B-SFT specs, VRAM requirements, and which GPUs can run it.

Llama-3.1-Tulu-3.1-8B specs, VRAM requirements, and which GPUs can run it.

Llama-3.2-3B specs, VRAM requirements, and which GPUs can run it.

Llama-3.2-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

llama-7b specs, VRAM requirements, and which GPUs can run it.

Llama-Guard-3-8B specs, VRAM requirements, and which GPUs can run it.

Llama-Guard-3-8B-INT8 specs, VRAM requirements, and which GPUs can run it.

LlamaGuard-7b specs, VRAM requirements, and which GPUs can run it.

llm-jp-3-3.7b-instruct specs, VRAM requirements, and which GPUs can run it.

LocoOperator-4B specs, VRAM requirements, and which GPUs can run it.

madlad400-8b-lm specs, VRAM requirements, and which GPUs can run it.

maira-2 specs, VRAM requirements, and which GPUs can run it.

MediPhi specs, VRAM requirements, and which GPUs can run it.

MediPhi-Clinical specs, VRAM requirements, and which GPUs can run it.

MediPhi-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3-8B specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct-bnb-4bit specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct-FP8 specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-3.1-8B-Instruct-FP8-dynamic specs, VRAM requirements, and which GPUs can run it.

Meta-Llama-Guard-2-8B specs, VRAM requirements, and which GPUs can run it.

Mistral 7B specs, VRAM requirements, and which GPUs can run it. Efficient and fast for everyday tasks.

Mistral-7B-Instruct-v0.2 specs, VRAM requirements, and which GPUs can run it.

Mistral-7B-v0.1 specs, VRAM requirements, and which GPUs can run it.

mistral-7b-v0.3-bnb-4bit specs, VRAM requirements, and which GPUs can run it.

Mistral-NeMo-Minitron-8B-Instruct specs, VRAM requirements, and which GPUs can run it.

Nanbeige4.1-3B specs, VRAM requirements, and which GPUs can run it.

Nanbeige4.1-3B-heretic specs, VRAM requirements, and which GPUs can run it.

Nemotron-H-4B-Base-8K specs, VRAM requirements, and which GPUs can run it.

Nemotron-H-4B-Instruct-128K specs, VRAM requirements, and which GPUs can run it.

Nemotron-H-8B-Base-8K specs, VRAM requirements, and which GPUs can run it.

NextCoder-7B specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-2-Mistral-7B-DPO specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-2-SOLAR-10.7B specs, VRAM requirements, and which GPUs can run it.

Nous-Hermes-llama-2-7b specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-3-Nano-4B-FP8 specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2 specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-Base specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-FP8 specs, VRAM requirements, and which GPUs can run it.

NVIDIA-Nemotron-Nano-9B-v2-Japanese specs, VRAM requirements, and which GPUs can run it.

OLMo-2-1124-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Olmo-3-1025-7B specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct-DPO specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Instruct-SFT specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-RL-Zero-Math specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think-DPO specs, VRAM requirements, and which GPUs can run it.

Olmo-3-7B-Think-SFT specs, VRAM requirements, and which GPUs can run it.

Olmo-3.1-7B-RL-Zero-Math specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-0424-hf specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-0724-hf specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-0724-Instruct-hf specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-hf specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

OLMo-7B-SFT-hf specs, VRAM requirements, and which GPUs can run it.

Olmo-Hybrid-Instruct-DPO-7B specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0125 specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0125-Instruct specs, VRAM requirements, and which GPUs can run it.

OLMoE-1B-7B-0924-Instruct specs, VRAM requirements, and which GPUs can run it.

Phi-3-mini-4k-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-3-mini-4k-instruct-gptq-4bit specs, VRAM requirements, and which GPUs can run it.

Phi-3-small-128k-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-3-small-8k-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-3.5-mini-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-mini-MoE-instruct specs, VRAM requirements, and which GPUs can run it.

Phi-tiny-MoE-instruct specs, VRAM requirements, and which GPUs can run it.

polyglot-ko-5.8b specs, VRAM requirements, and which GPUs can run it.

PowerMoE-3b specs, VRAM requirements, and which GPUs can run it.

pythia-12b specs, VRAM requirements, and which GPUs can run it.

pythia-6.9b specs, VRAM requirements, and which GPUs can run it.

Qianfan-OCR specs, VRAM requirements, and which GPUs can run it.

Qwen2-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-3B specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-3B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-7B specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-Coder-7B-Instruct-GPTQ-Int4 specs, VRAM requirements, and which GPUs can run it.

Qwen2.5-VL-7B-Instruct-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3-14B-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-Base specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-Instruct-2507 specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-Instruct-2507-FP8 specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-SafeRL specs, VRAM requirements, and which GPUs can run it.

Qwen3-4B-Thinking-2507 specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-AWQ specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-Base specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-FP8 specs, VRAM requirements, and which GPUs can run it.

Qwen3-8B-NVFP4 specs, VRAM requirements, and which GPUs can run it.

Qwen3-Coder-30B-A3B-Instruct-gptq-8bit specs, VRAM requirements, and which GPUs can run it.

Qwen3.5-35B-A3B-Text-qx64-hi-mlx specs, VRAM requirements, and which GPUs can run it.

Qwen3.5-4B-Safety-Thinking specs, VRAM requirements, and which GPUs can run it.

Qwen3.5-9B-abliterated specs, VRAM requirements, and which GPUs can run it.

Qwen3Guard-Gen-4B specs, VRAM requirements, and which GPUs can run it.

Qwen3Guard-Gen-8B specs, VRAM requirements, and which GPUs can run it.

recurrentgemma-9b specs, VRAM requirements, and which GPUs can run it.

recurrentgemma-9b-it specs, VRAM requirements, and which GPUs can run it.

saiga_llama3_8b specs, VRAM requirements, and which GPUs can run it.

SOLAR-10.7B-Instruct-v1.0 specs, VRAM requirements, and which GPUs can run it.

SOLAR-10.7B-v1.0 specs, VRAM requirements, and which GPUs can run it.

StableBeluga-7B specs, VRAM requirements, and which GPUs can run it.

stablecode-completion-alpha-3b-4k specs, VRAM requirements, and which GPUs can run it.

stablelm-2-12b specs, VRAM requirements, and which GPUs can run it.

stablelm-base-alpha-7b-v2 specs, VRAM requirements, and which GPUs can run it.

Starling-LM-7B-beta specs, VRAM requirements, and which GPUs can run it.

steerling-8b specs, VRAM requirements, and which GPUs can run it.

tiny-aya-global specs, VRAM requirements, and which GPUs can run it.

Turkish-Gemma-9b-T1 specs, VRAM requirements, and which GPUs can run it.

txgemma-9b-chat specs, VRAM requirements, and which GPUs can run it.

Unsloth Llama 3 8B Instruct specs, VRAM requirements, and which GPUs can run it.

UserLM-8b specs, VRAM requirements, and which GPUs can run it.

wildguard specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-6B specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-6B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-32K specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-1.5-9B-Chat-16K specs, VRAM requirements, and which GPUs can run it.

Yi-6B specs, VRAM requirements, and which GPUs can run it.

Yi-6B-200K specs, VRAM requirements, and which GPUs can run it.

Yi-6B-Chat specs, VRAM requirements, and which GPUs can run it.

Yi-6B-Chat-4bits specs, VRAM requirements, and which GPUs can run it.

Yi-9B specs, VRAM requirements, and which GPUs can run it.

Yi-9B-200K specs, VRAM requirements, and which GPUs can run it.

Yi-Coder-9B specs, VRAM requirements, and which GPUs can run it.

Yi-Coder-9B-Chat specs, VRAM requirements, and which GPUs can run it.

zephyr-7b-alpha specs, VRAM requirements, and which GPUs can run it.

zephyr-7b-beta specs, VRAM requirements, and which GPUs can run it.

zephyr-7b-gemma-sft-v0.1 specs, VRAM requirements, and which GPUs can run it.