Z-Image Turbo LoRA Training Guide | Create Custom LoRAs with AI Toolkit

Once you understand the basics of LoRA, the next step is creating your own. This guide walks you through training a LoRA for Z-Image Turbo using Ostris AI Toolkit, from dataset preparation to inference testing.

What You’ll Learn

• Required environment and setup for Z-Image Turbo LoRA training
• AI Toolkit installation steps
• Dataset preparation methods and best practices
• Training parameter configuration and execution
• Using trained LoRAs in ComfyUI
• Measured timings on Apple Silicon (MPS)

Prerequisites

• Understanding of LoRA basics
• Basic ComfyUI operation skills
• Basic terminal/command line familiarity

Key Concepts for Z-Image Turbo LoRA Training

Why Training Adapter Is Needed

Z-Image Turbo is a distilled model. While standard models require 20-50 steps to generate an image, Z-Image Turbo is optimized to generate in just 8 steps.

This distillation is efficient, but it creates a problem for LoRA training. Training a LoRA directly on a distilled model breaks the fast generation capability acquired through distillation. This is called “Turbo Drift.”

The Training Adapter solves this by temporarily reversing the distillation effect during training. At inference time, you remove the adapter and use only the LoRA.

Training Approaches

This guide uses the most common Turbo + Training Adapter v2 approach.

Environment Setup

Requirements

Required Models

AI Toolkit Setup

Download Training Adapter v2:

Setup Timings (Measured)

Dataset Preparation

Image Requirements

The quality of your LoRA depends entirely on your dataset.

Dataset rules:

• Resolution: 1024px+ recommended (512px works but lower quality)
• Diversity: Include different poses, angles, expressions, backgrounds
• Consistency: Keep the learning target (subject identity, etc.) consistent
• Backgrounds: For character LoRAs, vary the backgrounds
• Avoid: Blurry, low-res, watermarked, or multi-subject images

Composition Distribution (Character LoRA)

Creating Captions

Create a text file (.txt) with the same name for each image.

datasets/
└── my_dataset/
    ├── image1.jpg
    ├── image1.txt  ← "sks dog, a photo of a cute shiba inu dog"
    ├── image2.jpg
    ├── image2.txt
    └── ...

The trigger word (e.g., sks) should be a unique string that doesn’t conflict with existing vocabulary. Include it in all captions to activate the LoRA effect during inference.

Training Configuration

YAML Config

AI Toolkit manages training settings via YAML files. Template for Z-Image Turbo:

Key Parameters

Apple Silicon (MPS) Notes

When running AI Toolkit on Apple Silicon MPS, these config changes are needed:

• device: Change to mps:0
• optimizer: adamw (adamw8bit is CUDA-only)
• quantize: false (MPS doesn’t support quantization; 64GB unified memory allows unquantized training)
• num_workers: 0 (add to dataset config; MPS tensors don’t support multiprocess sharing)

Running Training

When training starts, the following steps execute in order:

1. Model loading: Load transformer, text encoder, and VAE
2. Training Adapter merge: Integrate the de-distillation adapter
3. LoRA network creation: Build training network at specified rank
4. Latent caching: Save VAE-encoded dataset images to disk
5. Training loop: Execute training for the specified number of steps

Monitoring Training

Monitor loss values in the logs. Normal training shows a gradual decrease in loss.

Measured Results: 100 Steps with 5 Training Images

We ran 100 steps of training on Apple Silicon M4 Pro (64GB) and performed the following verification.

Loss trend: Average loss for first 20 steps was 0.383, last 20 steps was 0.383. No significant loss decrease was observed at 100 steps. Per-step variance was large (0.21-0.60).

LoRA weight changes: LoRA B matrices (initialized at 0) moved to a mean norm of 0.16, confirming that gradient updates did occur.

Inference impact: Comparing images generated with the same prompt and seed, with and without LoRA, 98% of pixels showed differences. However, the mean difference was only 3.4/255 — the composition and subject were identical, with only subtle texture and color tone changes.

Conclusion: 5 images and 100 steps are sufficient to verify the pipeline works, but not enough to learn subject identity (e.g., steering toward the training data’s Shiba Inu). For practical LoRAs, we recommend at least 15+ images and 1,000+ steps.

Signs of overfitting:

• Extremely low loss values
• Sample images identical to training data
• No response to prompt changes

Solutions: Reduce steps, lower learning rate, add more data.

Measured Training Speed (Apple Silicon M4 Pro, 64GB)

With an NVIDIA RTX 4090, expect ~10-15 seconds per step, completing 3,000 steps in roughly 8-12 hours.

Using Trained LoRAs

ComfyUI Inference

After training completes, place the .safetensors file from the output directory into ComfyUI’s models/loras/.

ComfyUI workflow structure:

UNETLoader (Z-Image Turbo)
  ↓ MODEL
LoRA Loader (trained LoRA)
  ↓ MODEL              ↓ CLIP
  KSampler ← CLIPTextEncode (prompt with trigger word)
  ↓ LATENT
VAEDecode → SaveImage

Strength Adjustment

• LoRA strength: Start at 0.5-0.8 and adjust
• If too strong causes artifacts, reduce to ~0.5
• If too weak shows no effect, increase to 0.9-1.0

Stacking with Existing LoRAs

Multiple LoRAs can be combined:

Style LoRA (0.6) + Character LoRA (0.3) = Total 0.9

Keep total weight below 1.0 for best results.

Troubleshooting

Common Issues

Useful Resources

• Training a LoRA for Z-Image Turbo with AI Toolkit - HuggingFace Blog
• ostris/ai-toolkit - GitHub
• Training Adapter v2 - HuggingFace

Related Articles

What Is LoRA? A Beginner-Friendly Explanation of How It Works, How to Use It, and Where to Find It

z-image-turbo ComfyUI Workflow Distribution | Pre-Configured Optimal Settings

[Verified] Image Generation Prompt Best Practices