Stable Diffusion Models: SD 1.5 vs SDXL vs SD 3.5 vs FLUX

09 November, 2026

AIDeep LearningDiffusion ModelGenerative ModelMachine LearningNeural Networks

SD 1.5, SDXL, SD 3.5, FLUX: people call all of these “Stable Diffusion models”, but under the hood they are quite different machines. In this article, I’m going to compare what actually changed from one generation to the next (the text encoder, the denoiser, and the latent space) and then tell you which one to pick for your use case.

If you have not read how Stable Diffusion works yet, start with Stable Diffusion Clearly Explained. I won’t repeat the basics (VAE, U-Net, CLIP, the denoising loop) here. Every number below links to its source; where I counted something myself, I show the code.

Quick Comparison

	SD 1.5	SDXL 1.0	SD 3.5 (Large / Medium)	FLUX.1 ([dev] / [schnell])
Released	Oct 20, 2022 [1]	Jul 26, 2023 [4]	Oct 22, 2024 (Large) / Oct 29, 2024 (Medium) [8]	Aug 1, 2024 [13]
Made by	Released by Runway; developed by R. Rombach & P. Esser [1][2]	Stability AI [6]	Stability AI [8]	Black Forest Labs [13]
Denoiser	U-Net + cross-attention [2]	3× larger U-Net [5]	MMDiT / MMDiT-X transformer [9][10]	Hybrid multimodal + parallel diffusion transformer [13]
Denoiser params	860M [5]	2.6B [5]	8.1B / 2.5B [8]	12B [14][15]
Training objective	Diffusion (noise prediction) [3]	Discrete-time diffusion, 1000 steps [5]	Rectified flow [11]	Rectified flow / flow matching [13][14]
Text encoders	CLIP ViT-L/14 [2]	CLIP ViT-L + OpenCLIP ViT-bigG [5]	CLIP ViT-L + OpenCLIP ViT-bigG + T5-XXL [9]	CLIP ViT-L + T5-v1.1-XXL [16]
VAE latent channels	4 [22]	4 [22]	16 (SD3 design) [11] *	16 [16]
Native resolution	512×512 [2]	1024×1024 [4]	~1 MP / 0.25–2 MP [8]	0.1–2.0 MP [13]
License	CreativeML OpenRAIL-M [2]	CreativeML Open RAIL++-M [4]	Stability AI Community License [12]	[dev]: FLUX.1 [dev] Non-Commercial; [schnell]: Apache-2.0 [14][15]
Commercial use of the model	Yes, with use-based restrictions [21]	Yes, with use-based restrictions [7]	Free under US$1M annual revenue; Enterprise license above [9]	[dev]: no (paid license needed); [schnell]: yes [17][15]

* The SD 3.5 VAE config sits in a gated repo, so the 16 comes from the SD3 paper, which SD 3.5 builds on, not from the SD 3.5 file itself.

Architecture lineage of Stable Diffusion models: SD 1.5, SDXL, SD 3.5 and FLUX.1, showing text encoders, denoiser and VAE for each — Same three-part recipe (text encoders → denoiser → VAE), four very different ingredients

The Three Knobs That Changed

All four models are latent generative models: a VAE squeezes the image into a smaller latent, a denoiser does the generation in that latent space, and text encoders tell the denoiser what to draw. So “what changed” boils down to three knobs.

Knob 1: The latent space got fatter

Every model here uses a VAE with a downsampling factor of 8. An image of size $H \times W \times 3$ becomes a latent of size

\frac{H}{8} \times \frac{W}{8} \times c

where $c$ is the number of latent channels. SD 1.5 and SDXL use $c = 4$ . SD3 and FLUX.1 use $c = 16$ . Take a 1024×1024 image as an example:

Pixels: 1024 × 1024 × 3 = 3,145,728 numbers
4-channel latent: 128 × 128 × 4 = 65,536 numbers, a 48× compression
16-channel latent: 128 × 128 × 16 = 262,144 numbers, only a 12× compression

Less compression means the VAE throws away less detail, so small text, fingers and fine textures survive the round trip. The SD3 authors measured exactly this: going from 4 to 16 channels improved every reconstruction metric they tested (for example, PSNR rose from 25.12 to 28.62 dB). The catch is that a richer latent is harder to predict, so the denoiser needs more capacity to benefit [11]. That is one reason the models after SDXL are so much bigger.

Knob 2: The denoiser went from U-Net to transformer

SD 1.5 and SDXL use a convolutional U-Net. The text enters through cross-attention: image features ask questions (queries) and the text embeddings only answer (keys and values). The text never gets updated by the image.

SD3 replaced this with the MMDiT (multimodal diffusion transformer). The latent is cut into 2×2 patches, so a 128×128 latent becomes 64 × 64 = 4,096 image tokens. These are concatenated with the text tokens, and both go through one joint attention operation. Each modality keeps its own weights, but information flows both ways [11]. FLUX.1 builds on the same idea and adds a second kind of block, explained below.

Knob 3: Diffusion became rectified flow

In the diffusion model used by SD 1.5 and SDXL, the noisy sample is $x_t = \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1-\bar{\alpha}_t}\, \varepsilon$ , a curved path from image to noise. SD3 and FLUX instead use rectified flow, which connects data and noise with a straight line [11]:

z_t = (1-t)\, x_0 + t\, \varepsilon, \qquad t \in [0, 1]

The network predicts the velocity $v_\Theta(z_t, t)$ along that line, and the target is simply $\varepsilon – x_0$ . A straight path is easier to follow in few steps: in the SD3 experiments, rectified flow variants lost less quality than the other formulations when the number of sampling steps was cut [11]. The paper also samples the middle timesteps more often during training (a logit-normal schedule), because those are the hardest to predict.

Bar charts comparing denoiser parameter counts and VAE latent channels of SD 1.5, SDXL, SD 3.5 Medium, SD 3.5 Large and FLUX.1 dev — Denoiser size and latent channels. SD 1.5 and SDXL are counted from the official configs; the others are the publishers’ figures

SD 1.5: The Baseline Everyone Fine-Tuned

Runway released the SD 1.5 checkpoints on October 20, 2022 [1]. Architecturally it is the same model as SD 1.4. The model card lists both as resumed from v1-2; v1-5 was trained for 595k steps at 512×512 on “laion-aesthetics v2 5+”, and 10% of the text conditioning was dropped to enable classifier-free guidance [2].

Text encoder: a frozen CLIP ViT-L/14. Its non-pooled output feeds the U-Net through cross-attention [2]. That is 768-dimensional embeddings for at most 77 tokens.
Denoiser: an 860M-parameter U-Net [5]. My own count from the official config gives 859.5M (code below).
VAE: 4 latent channels, so a 512×512 image becomes a 64×64×4 latent.

Why it still matters: SD 1.5 is small (about 1.07B parameters in total, including the text encoder and VAE) and the license allows commercial use. That combination made it the base of a huge ecosystem of fine-tunes, LoRAs and ControlNets. Its own model card is honest about the weak spots: it “cannot render legible text” and struggles with compositional prompts [2]. Many of these SD 1.x images also carry an invisible watermark, which I covered in Stable Diffusion: The Invisible Watermark in Generated Images.

SDXL: Same Idea, Scaled Up

Stability AI released SDXL 1.0 on July 26, 2023, under the CreativeML Open RAIL++-M license [4]. It is still a U-Net latent diffusion model, but nearly every part got bigger or smarter [5]:

Two text encoders. CLIP ViT-L and OpenCLIP ViT-bigG run side by side. Their penultimate outputs are concatenated into a 2048-dimensional context, and the pooled bigG embedding conditions the model too.
A 3× larger U-Net (2.6B). Most of the transformer blocks moved to the lower-resolution levels ([0, 2, 10] blocks instead of [1, 1, 1, 1]), and the deepest 8× level was removed.
Micro-conditioning. The original image size and the crop coordinates are fed in as extra conditions. This lets the model learn from small images without copying their blurriness, and it fixes the “cut-off head” crops typical of SD 1.5.
Multi-aspect training at about 1024² pixels, plus a retrained VAE. The VAE has the same architecture and still 4 channels, but better reconstructions.
An optional refiner, a second model specialised for the final denoising steps [6].

A nice puzzle about the numbers: the paper says “2.6B”, but Stability’s announcement says a “3.5B parameter base model” [4]. Both are right. When I count from the official configs, the U-Net has 2,567.5M parameters, the two text encoders 123.1M + 694.7M = 817.8M (the paper says 817M), and the VAE 83.7M. Altogether that is 3,469M ≈ 3.5B. So the paper counts the denoiser and the announcement counts the whole pipeline. Keep this in mind whenever you compare “model sizes” across vendors.

In the paper’s user study, raters preferred SDXL with the refiner 48.44% of the time, SDXL base 36.93%, SD 1.5 7.91% and SD 2.1 6.71% [5]. The same paper notes that SDXL’s COCO FID was actually worse than SD 1.5’s, which is a good reminder that FID does not track what humans like.

SD 3.5: The Switch to Transformers

SD 3.5 is the refined release of the SD3 architecture. Stability shipped it after admitting that SD3 Medium (June 2024) “didn’t fully meet our standards” [8]. There are three variants [8]:

SD 3.5 Large: 8.1B parameters, MMDiT, built for about 1 megapixel.
SD 3.5 Large Turbo: a distilled Large that generates in 4 steps.
SD 3.5 Medium (Oct 29, 2024): 2.5B parameters with the improved MMDiT-X architecture, for 0.25–2 megapixels. Stability quotes 9.9 GB of VRAM, excluding the text encoders.

What changed and why it matters:

Three text encoders. CLIP-L and OpenCLIP-bigG (77 tokens each) are joined by T5-XXL, a language model with a context of 77 or 256 tokens, depending on the training stage [9]. In the SD3 paper, the CLIP features (77 × 2048) are zero-padded to 4096 dimensions and stacked with the T5 features (77 × 4096) into a 154-token context [11].
T5 is optional at inference. Each encoder was dropped independently during training, so you can skip the 4.7B-parameter T5 to save memory. In the paper’s human evaluation, removing T5 left aesthetics unchanged (50% win rate) and cost a little prompt adherence (46%). It hurt typography the most (38%) [11]. In short: keep T5 if you need text in the image.
MMDiT with QK-normalization. Normalising queries and keys keeps attention logits from blowing up at high resolution, which made bf16 training stable [9][11].
MMDiT-X (Medium only). It adds extra self-attention in the early layers and uses progressive training from 256 to 1440 px [10]. Note that the card itself says “first 12” layers in one place and “first 13” in another.

License: the Stability AI Community License. It is free for research, non-commercial use, and commercial use by individuals or organisations with less than US$1M in annual revenue. Above that you need an Enterprise license [9][12]. Commercial users under the threshold are also asked to register with Stability [12].

FLUX.1: The Same Lineage, a New Company

FLUX.1 comes from Black Forest Labs, announced on August 1, 2024 [13]. It is not branded “Stable Diffusion”, but it is the natural next step: in its launch post BFL says its team created VQGAN, Latent Diffusion, Stable Diffusion XL and Rectified Flow Transformers (the SD3 paper) [13].

The variants and their licenses are the part people get wrong most often:

Variant	How you get it	License	Can I use it commercially?
FLUX.1 [pro]	API only [13]	BFL API terms	Via the paid API
FLUX.1 [dev]	Open weights, guidance-distilled from [pro] [13][14]	FLUX.1 [dev] Non-Commercial License [17]	Model: non-commercial only; a commercial license is sold separately. Outputs: yes, but not to train a competing model, and you must filter or review outputs
FLUX.1 [schnell]	Open weights, 1–4 steps [15]	Apache-2.0 [15]	Yes

What changed in the architecture: FLUX.1 is a 12B rectified flow transformer that uses “a hybrid architecture of multimodal and parallel diffusion transformer blocks”, rotary position embeddings and parallel attention layers [13][14]. BFL’s reference code spells out the shape [16]:

19 double-stream blocks are MMDiT-style, with separate weights for text and image tokens.
38 single-stream blocks follow them. Here text and image tokens are merged into one sequence and share weights, which saves parameters per block.
Hidden size 3072, 24 heads, and in_channels = 64: that is 16 latent channels × a 2×2 patch.
Text: T5-v1.1-XXL supplies the token sequence (up to 512 tokens by default in the reference code), and CLIP ViT-L supplies a single pooled 768-d vector for modulation. There is no OpenCLIP-bigG anymore.
VAE: 16 latent channels, the same idea as SD3.

Two distillation tricks explain the variants. Guidance distillation ([dev]) bakes classifier-free guidance into the network: the guidance strength becomes an input (guidance_embed = True in the config), so you run one forward pass per step instead of two. Latent adversarial diffusion distillation ([schnell]) trains the model to produce good images in only 1–4 steps [14][15].

What about FLUX.2?

BFL released FLUX.2 on November 25, 2025 [18]. The open FLUX.2 [dev] is a 32B model that couples a 24B Mistral vision-language model with a rectified flow transformer. It handles both generation and editing (up to 10 reference images, up to 4 MP), uses the FLUX Non-Commercial License, and has a new VAE with 32 latent channels (Apache-2.0) [18][20]. In January 2026 came FLUX.2 [klein]. The 4B variants are Apache-2.0 and run in about 13 GB of VRAM; the 9B variants use the FLUX Non-Commercial License [19]. The trend continues: fatter latents, bigger language models as text encoders.

Count the Parameters Yourself

You don’t need a GPU, or even the weights, to check these numbers. Diffusers can build a model on PyTorch’s meta device, which creates every layer’s shape without allocating memory. Only the small config.json files are downloaded:

import json, torch
from huggingface_hub import hf_hub_download
from diffusers import UNet2DConditionModel, AutoencoderKL

def cfg(repo, sub):
    return json.load(open(hf_hub_download(repo, f"{sub}/config.json")))

def n_params(model):
    return sum(p.numel() for p in model.parameters())

for repo in ["stable-diffusion-v1-5/stable-diffusion-v1-5",
             "stabilityai/stable-diffusion-xl-base-1.0"]:
    u, v = cfg(repo, "unet"), cfg(repo, "vae")
    with torch.device("meta"):          # shapes only, no memory, no weights
        unet = UNet2DConditionModel.from_config(u)
        vae = AutoencoderKL.from_config(v)
    print(repo)
    print(f"  UNet: {n_params(unet)/1e6:,.1f} M  latent {u['sample_size']} -> {u['sample_size']*8}px")
    print(f"  VAE latent_channels: {v['latent_channels']}  VAE: {n_params(vae)/1e6:,.1f} M")

Output (diffusers 0.42.1, CPU only):

stable-diffusion-v1-5/stable-diffusion-v1-5
  UNet: 859.5 M  latent 64 -> 512px
  VAE latent_channels: 4  VAE: 83.7 M
stabilityai/stable-diffusion-xl-base-1.0
  UNet: 2,567.5 M  latent 128 -> 1024px
  VAE latent_channels: 4  VAE: 83.7 M

The SD 3.5 and FLUX.1 repos are gated on Hugging Face, so their configs need a login. As a cross-check, I plugged the hyper-parameters from BFL’s public reference code into diffusers’ FluxTransformer2DModel, again on the meta device. It gives 11.90B for [dev] and 11.89B for [schnell], which matches the official “12B”. The difference between them is the guidance embedding.

Which Stable Diffusion Model Should You Use?

There is no single “best” model. It depends on your hardware, your license needs and whether you need text in the image. Here is how I would choose, based on the facts above:

If you want…	Pick	Why
To learn how diffusion works, or to experiment on a laptop GPU	SD 1.5	Smallest (~1B total), 512 px, tons of tutorials and fine-tunes
The biggest ecosystem of LoRAs/ControlNets at 1024 px	SDXL	Mature tooling; Stability said it should work on 8 GB VRAM GPUs [4]
Legible text and complex prompts, free for a small business	SD 3.5 Large (or Large Turbo for speed)	T5 + MMDiT + 16-ch latent; free under US$1M revenue [12]
A modern transformer model on mid-range GPUs	SD 3.5 Medium	2.5B; 9.9 GB VRAM excluding text encoders [8]
Top open-weight quality for personal or research use	FLUX.1 [dev]	12B guidance-distilled; non-commercial license on the model [17]
Fast generation in a commercial product, no license fees	FLUX.1 [schnell] (or FLUX.2 [klein] 4B)	Apache-2.0; 1–4 steps [15][19]
Fine-tuning research on small data	SD 1.5 or SDXL	Cheapest to train; permissive OpenRAIL licenses

One practical warning: the “open” in open-weight does not mean the same thing across these models. If you plan to ship a product, read the license of the exact variant you download. FLUX.1 [dev] and FLUX.1 [schnell] look like siblings, but their commercial terms are opposites.

Summary

From SD 1.5 to FLUX, three things changed: the text encoder (CLIP → CLIP + T5), the denoiser (U-Net → multimodal transformer) and the latent (4 → 16 channels), while the training objective moved from diffusion to rectified flow.
Choose by license and hardware first, quality second. SD 1.5 and SDXL are permissive, SD 3.5 is free under US$1M revenue, FLUX.1 [dev] is non-commercial and FLUX.1 [schnell] is Apache-2.0.

If you like this kind of side-by-side comparison, you may also enjoy StyleGAN vs StyleGAN2 vs StyleGAN2-ADA vs StyleGAN3.