Linux for Local AI: What Actually Breaks and How to Fix It

The short version

Local AI on Linux is very workable. The failures people hit are rarely mysterious: the NVIDIA driver wasn't signed for Secure Boot, a kernel update outran the driver rebuild, an AMD card isn't on the ROCm support list, or a brand-new GPU needs a newer kernel than an LTS distro originally shipped. Each section below leads with what works, then the one step to know about. If you're still choosing hardware, the AI PC Builder surfaces the purchase-relevant notes from this guide automatically.

NVIDIA + Secure Boot: signed drivers or a two-minute key enrollment

Most modern PCs ship with UEFI Secure Boot enabled, and a Secure Boot kernel only loads signed modules. An unsigned NVIDIA module means the GPU stays invisible: nvidia-smi reports it "couldn't communicate with the NVIDIA driver" even though the packages installed fine.

The good news: on Ubuntu, the default ubuntu-drivers path installs prebuilt modules signed with Canonical's key — Secure Boot just works, no extra step. You only meet the signing requirement on the DKMS package path, Fedora's akmod packages, or NVIDIA's manual .run installer. There, the fix is a guided one-time MOK enrollment: set a password when prompted during install, reboot, and confirm the key in the blue "MOK Manager" screen. It takes about two minutes and only ever needs doing once per key.

Verify your setup:

mokutil --sb-state        # is Secure Boot on?
nvidia-smi                # does the driver see the GPU?
lsmod | grep nvidia       # is the module actually loaded?

Disabling Secure Boot in firmware also works, but try the signed path first — it keeps the security feature and is usually less effort than people expect. Sources: Ubuntu Server documentation on NVIDIA driver installation, NVIDIA's driver installation guide, and the RPM Fusion NVIDIA howto.

Driver stopped working after an update? Two cousins, two fixes

Nearly every "my NVIDIA GPU stopped working on Linux" report traces back to one of two update-drift problems. They look similar but have different error messages and different fixes — check which one you have before doing anything drastic.

Fork 1 — "Driver/library version mismatch": reboot first

If nvidia-smi prints "Failed to initialize NVML: Driver/library version mismatch", your system updated the NVIDIA driver (often via unattended upgrades) but the old kernel module is still loaded in memory. A plain reboot fixes most cases. Nothing is wrong with your hardware. If it persists after a reboot, the usual cause is mixed install sources — a .run installer layered over distro packages — and the fix is to purge and reinstall from a single source. On an always-on inference box, consider excluding the NVIDIA driver from unattended upgrades so updates happen when you can reboot.

nvidia-smi                          # reproduces the exact error
cat /proc/driver/nvidia/version     # version of the *loaded* module

Fork 2 — basic graphics after a kernel update: headers and rebuild

On DKMS-based installs (Debian, Ubuntu's DKMS path, the .run installer) and Fedora akmods, each kernel update triggers a driver-module rebuild. If matching kernel headers are missing, the rebuild silently fails and the next boot lands on basic graphics. Recovery: boot the previous kernel from the GRUB menu, install the headers, and rebuild.

uname -r                            # which kernel are you on?
dkms status                         # is the nvidia module built for it?
sudo apt install linux-headers-$(uname -r)
sudo dkms autoinstall               # Debian: dpkg-reconfigure nvidia-kernel-dkms

Prevention is one package: install the headers metapackage (linux-headers-generic on Ubuntu/Debian, kernel-devel on Fedora) so future kernels bring matching headers automatically. Better still, Ubuntu's default path now uses precompiled signed modules that update in lockstep with the kernel, designing this failure out — one more reason to prefer the distro installer over manual downloads. Sources: NVIDIA driver installation guide (Ubuntu), Debian wiki, and the moderator-answered NVIDIA forum thread on the NVML mismatch.

The nouveau "conflict" — historical for normal installs

Older guides make a big deal of blacklisting nouveau, the open-source NVIDIA driver that ships with Linux, before installing the real driver. That advice is mostly historical now: Debian and Ubuntu packages blacklist nouveau automatically, and Fedora's RPM Fusion packages add the blacklist kernel arguments for you. NVIDIA's own current install guides don't even list it as a step.

The one place it still bites is the manual .run installer from NVIDIA's website — which beginners shouldn't use anyway. If you do go manual, let the installer write the blacklist file, rebuild the initramfs, and reboot. And a fairness note: nouveau (with the NVK Vulkan driver) has become a genuinely usable graphics driver — it just has no CUDA, so it is not a local-AI compute path.

lsmod | grep -E 'nouveau|nvidia'    # which driver is actually loaded?

Suspend and loaded models: one config, fewer surprises

By default, the NVIDIA Linux driver saves only "essential" video memory across suspend/hibernate — NVIDIA's own documentation says this can lead to rendering corruption and application crashes after resume. For local AI that means a loaded model or CUDA context can misbehave after the machine wakes. Always-on servers never notice; desktops and laptops that sleep do.

The documented fix is a one-time config: set the module option NVreg_PreserveVideoMemoryAllocations=1 and enable the nvidia-suspend, nvidia-hibernate, and nvidia-resume systemd services. If your /tmp is tmpfs (RAM-backed), also point NVreg_TemporaryFilePath at disk-backed storage with room for your VRAM contents. Newer stacks are automating this: the 595-series driver adds NVreg_UseKernelSuspendNotifiers for the open kernel modules. Distro defaults vary — check rather than assume:

cat /proc/driver/nvidia/params | grep PreserveVideoMemoryAllocations
systemctl status nvidia-suspend.service nvidia-resume.service

Source: NVIDIA driver README, Configuring Power Management Support.

AMD: check the ROCm list, and Vulkan has your back

AMD works well for local AI on Linux — better than its reputation suggests. AMD's official compute stack, ROCm, now covers the current consumer lineup well: as of ROCm 7.2 the entire RX 9000 series and the RX 7700 and up (including the RX 7900 XTX) are on the official Linux support list. What's not listed: the budget RX 7600 class, consumer RX 6000 and older cards, and most integrated GPUs. On an unlisted card, ROCm-backed runtimes can quietly fall back to CPU inference — it's logged, but easy to miss.

If you're buying: check AMD's current ROCm support matrix for the exact card first — the list changes with ROCm releases.

If your card isn't listed, you're not stuck. llama.cpp has an official, mature Vulkan backend that runs on virtually every modern Radeon via the standard Mesa drivers; LM Studio ships a Vulkan runtime; and current Ollama includes Vulkan as an additional backend enabled by default (it is no longer ROCm-only on AMD). Honest expectations: chat/token-generation speed is close to ROCm, long-prompt processing is somewhat slower, and multi-GPU scaling is clearly weaker. There's also the runtime-documented HSA_OVERRIDE_GFX_VERSION override — it works on many near-miss cards but is not supported by AMD.

rocminfo                  # does ROCm see a GPU agent?
vulkaninfo --summary      # is the Vulkan (RADV) path available?
ollama ps                 # after loading a model: GPU or CPU placement?

Sources: AMD Radeon native-Linux compatibility, Ollama GPU documentation, and the llama.cpp build docs.

RTX 50-series (Blackwell): driver 570+ and the open kernel modules

Two permanent facts for GeForce RTX 50-series cards on Linux: support begins at driver 570 (so any current production series qualifies), and per NVIDIA these GPUs work only with the open-source GPU kernel modules — the legacy proprietary kernel modules will never support Blackwell. In practice the trap is package choice: on Ubuntu, pick the driver package ending in -open (for example nvidia-driver-580-open); on Arch, nvidia-open. Picking the non-open variant typically means an unrecognized GPU or a black screen until corrected.

Distro availability has caught up: Ubuntu 24.04's official archive now carries several 570+ series with -open variants, and Fedora and Arch track current drivers. The residual risk is unupdated installs and older releases (Ubuntu 22.04, Debian 12), where the default driver predates 570 — update first, then select the open package.

nvidia-smi                        # driver version and GPU visibility
modinfo nvidia | grep license     # open modules report "Dual MIT/GPL"

Sources: NVIDIA's open kernel module announcement and the NVIDIA Unix driver index.

LTS distros and launch-year hardware: know your kernel track

LTS distros freeze their default kernel at release — Ubuntu 24.04's GA kernel is 6.8 for its whole lifetime, and Debian 12 ships 6.1. Hardware launched after that freeze (new GPUs, new laptop platforms) may simply not be recognized. This is by design, and Ubuntu's documented answer is the HWE (hardware enablement) kernel track, which rolls forward through the LTS lifetime.

The practical picture in 2026: fresh Ubuntu Desktop installs from current point-release media already default to the HWE kernel — most desktop users clear the bar without knowing the term. The two common traps are Ubuntu Server, which defaults to the GA kernel (a very common choice for headless local-AI boxes), and systems installed from early 24.04 media that never opted in. The fix is one command:

uname -r                                    # 6.8 = GA kernel; newer = HWE
sudo apt install linux-generic-hwe-24.04    # move to the HWE track
apt policy linux-generic-hwe-24.04          # confirm the track is installed

On Debian, the equivalent moves are Debian 13 (kernel 6.12) or a backports kernel. Sources: Ubuntu kernel lifecycle and the HWE kernel documentation.

Intel Arc: a kernel-floor check before you buy

Intel Arc cards are interesting budget options for local AI, with one hard requirement: kernel age. The newer Battlemage cards (B580/B570) are supported only by the modern xe kernel driver and need kernel 6.12 or later out of the box, plus Mesa 24.3+ and current linux-firmware; the older Alchemist A-series needs kernel 6.2+. That rules out Ubuntu 24.04's original 6.8 GA kernel and Debian 12 — but updated Ubuntu 24.04 Desktop (HWE kernel), Debian 13, Fedora, and Arch all clear the bar.

For local AI specifically, note that Intel's compute stack (OpenCL / oneAPI Level Zero, which llama.cpp's SYCL backend sits on) is also kernel- and firmware-gated, and Battlemage compute can still be rough even on new kernels — verify the GPU is visible to compute before downloading large models:

uname -r                                  # 6.12+ for Battlemage, 6.2+ for Alchemist
lspci -k | grep -iA3 vga                  # Battlemage should show: Kernel driver in use: xe
clinfo | grep -i 'number of platforms'    # compute stack visible?

Sources: Intel Client GPU installation documentation and intel/compute-runtime issue #872.

Verify-your-setup checklist

Run these before downloading your first large model — two minutes here saves an evening of confusion. All commands are read-only.

# Any system
uname -r                     # kernel version
lspci -k | grep -iA3 vga     # GPU + which kernel driver claimed it

# NVIDIA
nvidia-smi                   # driver loaded, version, GPU visible
mokutil --sb-state           # Secure Boot state
dkms status                  # (DKMS installs) module built for this kernel?

# AMD
rocminfo                     # ROCm sees a GPU agent?
vulkaninfo --summary         # Vulkan path available?

# Intel Arc
clinfo | grep -i 'number of platforms'

# Runtime-level
ollama ps                    # after loading a model: GPU or CPU placement

When Windows is the simpler path

Honesty over advocacy: if your machine dual-boots or you're starting from scratch and just want to try local models today, Windows with Ollama or LM Studio is a perfectly good first step — NVIDIA's Windows drivers install without signing ceremonies, and LM Studio's Vulkan runtime covers AMD there too. Linux earns its keep when you want an always-on server, containerized stacks, or squeeze-every-megabyte VRAM efficiency. Many people prototype on Windows and graduate to a Linux box later; both are valid.

Freshness and sources

Every claim in this guide was verified against the linked primary sources on July 10, 2026. Driver versions, ROCm support lists, and kernel availability all change — where this guide names a version, treat it as a floor ("570 or newer"), and when in doubt, trust the linked vendor documentation over any secondhand summary, including ours. The same records that power this guide drive the setup notes in the AI PC Builder, so both surfaces stay in sync.