Cross-checkpoint portability of SDXL LoRA fine-tunes

Research question

Subject-identity LoRA fine-tunes for SDXL are trained against one specific base checkpoint. Do the resulting weights transfer to other SDXL-family checkpoints without retraining, and what, if anything, breaks when the same LoRA is applied to a checkpoint of a different base architecture (SD1.5)?

Dataset

• 48 source images: self-portrait photography with mixed lighting, angle and quality. The images are not published; only the preparation pipeline and configuration are.
• Preprocessing (scripts/prepare_lora_dataset.py): images are resized to a 1024 px long edge, padded or backgrounded, and auto-captioned with WD14, which is vendored inside kohya_ss.
• WD14 was chosen over BLIP because its comma-separated tags compose cleanly with a single prepended trigger word (trigger_word, tag1, tag2, ...), the caption format kohya_ss LoRA training expects. BLIP’s sentence-style captions do not.
• Trigger word: Arynwood, using kohya’s repeats folder convention (10_Arynwood/), so each image is seen 10 times per epoch.

Training configuration

Run through scripts/run_lora_training.py, which wraps kohya_ss’s SDXL-specific sdxl_train_network.py.

Total run time was about 6 hours 45 minutes on the training host, producing one .safetensors checkpoint per epoch plus a final combined output (456 MB).

Why it transfers across SDXL checkpoints

SDXL-family checkpoints (Juggernaut XL, DreamShaper XL Turbo, RealVisXL, vanilla SDXL 1.0) share one fixed architecture: the same UNet layout, the same dual text encoders (OpenCLIP-G and CLIP-L feeding 2048-dim cross-attention conditioning), and the same VAE latent space. A LoRA’s weight-delta tensors are keyed to that shared architecture’s layer names and shapes, not to any one finetune’s learned weights. So a LoRA trained against Juggernaut XL loads without error on any other SDXL derivative.

Why it can’t transfer to SD1.5

SD1.5 is architecturally distinct: a single CLIP text encoder feeding 768-dim cross-attention, and a different UNet channel and block layout. None of an SDXL LoRA’s keys resolve against an SD1.5 checkpoint’s modules. Depending on how the inference backend handles unmatched keys, the loader either silently skips the LoRA (the output looks as if it was never selected) or raises a load error. The trigger word also carries no learned meaning in SD1.5’s separate text-encoder embedding space.

Practical implication

Loading is guaranteed to work across SDXL derivatives, but fidelity is not. The LoRA’s deltas were computed against Juggernaut XL’s specific weights, so applying them to a distilled or Turbo checkpoint (a different sampling schedule and fewer inference steps) is expected to give a weaker or drifted likeness compared with a checkpoint-specific retrain. The study currently documents the mechanism; a quantitative comparison is future work.

Limitations

• Output-path collision. The pipeline names its output by trigger word and project, not by base checkpoint, so retraining the same project on a different checkpoint overwrites the previous output. It’s a pipeline gap, not an architectural one, and is fixable by adding the base checkpoint to the filename.
• No quantitative fidelity metric has been computed yet. The portability claims above are architectural (based on tensor key and shape matching), not yet empirically scored.

Future work

1. A quantitative cross-checkpoint fidelity comparison (CLIP image similarity between generations and reference photos) across Juggernaut XL, DreamShaper XL Turbo and RealVisXL, with the same prompt and seed.
2. Fixing the output collision so per-checkpoint artifacts persist independently.
3. An SD1.5 training path (kohya’s train_network.py at 512 to 768 px) as a control, to test whether a separately trained SD1.5 LoRA reaches comparable fidelity.

Cite this work

Noble, L. (2026). Cross-Checkpoint Portability of Subject-Identity LoRA Fine-Tunes in SDXL (v0.2.0). Zenodo. https://doi.org/10.5281/zenodo.21282393

Licensed under CC BY 4.0. The training scripts, configuration and a pointer to the trained weights are in the GitHub repository; the weights are hosted on Zenodo. Author ORCID iD: 0009-0007-8723-2857.