Orthos 2 is SUSE Labs' management tool of choice for sharing bare-metal machines with colleagues. The talk will provide an introduction to Ansible & Terraform, and show how internally at SUSE you can use Ansible to provision machines reproducibly after installation.

SUSE has been a birthbed for key Confidential Computing projects like the "COCONUT Secure VM Service Module" (SVSM), which has moved under the umbrella of the LF's Confidential Computing Consortium since. It became clear during early stages of development that some means of securely persisting sensitive core state, like that of an emulated TPM, would be needed. Assuming malicious hypervisors, existing FDE solutions operating at the block-layer (e.g. stacked "dm-crypt" + "dm-integrity"), which have been designed with the intent to protect data at rest, don't address the security needs of the SVSM at a satisfactory level. This led the presenter to create a new special-purpose filesystem format specifically tailored to the needs for protecting the most sensitive core TEE state: "CocoonFs". After revisiting the relevant abstract security definitions, the talk will give a quick overview on where common FDE solutions fall them short (on purpose), introduce the main "CocoonFs" format features, and discuss some of the trade-offs made.

A New tool to manage Virtual Machine. No X11 required: runs perfectly on headless servers and remote connections via SSH. Transhypervisor View: Manage VMs across multiple Libvirt servers in one unified dashboard. Smart Caching: caching reduces latency and API load. Built directly on the Libvirt API for maximum compatibility and stability with QEMU/KVM environments. Real-time updates via events ensure ultra-low bandwidth usage and immediate UI responsiveness. Full support for snapshots and disk overlays: branch your VM states or revert changes with confidence. Manage large scale fleets by selecting VMs using flexible regex patterns and group filters. Custom Migration support: workaround some migration limitation. Granular Server Preferences allow you to manage storage pools and network configurations remotely. Auto installation support: Autoyast and Agama for openSUSE and SLES.

Creating accurate changelog entries is often the final hurdle in the package submission process, where time pressure can lead to critical updates being overlooked. Since submission requests in the Open Build Service (OBS) contain the necessary technical context—including spec file diffs, file lists, and release notes—this task is an ideal candidate for automation using Large Language Models (LLMs). This talk explores the end-to-end workflow of fine-tuning a local LLM for this specialized task. We will cover how to extract data from OBS and the essential steps required to prepare it for training. Additionally, the talk discusses the fundamentals of fine-tuning, including model size selection and the optimization of tuning parameters.

The amount of rust code in the kernel grows steadily and we should start figuring out how to deal with it. There are a lot points to consider, rust version, kernel packaging, kABI, livepatching, etc.

The Linux scheduler is undergoing a significant transformation, with the transition from the long-standing Completely Fair Scheduler (CFS) to the Earliest Eligible Virtual Deadline First (EEVDF), and even more so with the extensible, BPF-based, sched_ext framework. This session will show the results of a benchmarking campaign aimed at comparing legacy (as fairly as we'll be able to) CFS, EEVDF, and a few of the emerging sched_ext based solutions across different SUSE and openSUSE distributions. By simulating multi-VM scenarios, with varying degrees of oversubscription and heterogeneous workloads, we will try to offer a quantitative estimation of the impact of the different scheduling algorithms on some virt-typical KPIs, such as throughput, latency and resource utilization.

In this presentation, we will discuss packaging guidelines and scenarios you might encounter as a package maintainer. We will explain how to integrate independent custom modules, deal with SELinux-aware applications, how to troubleshoot issues and when to delegate to the SELinux working group. Additionally, we will discuss some things that should not be done when integrating the package in an SELinux-enabled system.

SUSE has a strong "Upstream First" policy. While many of our colleagues joined the company as established contributors or even maintainers, others (like myself) joined as kernel newbies and are trying to make that leap. There is a gap between contributing the occasional fix for a product issue and actively reviewing patches, contributing a major feature, or maintaining a subsystem. This session is an open discussion that hopes to help others like me to bridge that gap. Some questions I have in mind are: * How do you balance reactive/proactive product maintenance (L3, bug fixes) with upstream work? * How do you context-switch effectively? * What advice do you have for engineers looking to take on more upstream responsibility? We invite experienced contributors/maintainers to come share their workflows, and newer contributors to bring their questions.

LTP receives a steady flow of patches from contributors worldwide, and every one of them needs to be reviewed against the project's coding standards before it can be merged. This is time-consuming work that falls on a small group of maintainers. LTP Agent automates that first pass. It watches Patchwork for new submissions, reviews each patch against the project's own rules, smoke-tests the code, and posts a verdict back — all within minutes of submission. Contributors get immediate feedback on common issues like missing cleanup, non-portable code, or incorrect API usage, so they can send a revised version before a maintainer even looks at it. The result is that maintainers spend less time on mechanical checks and more time on design and correctness. Contributors iterate faster. And the codebase stays consistent as it grows. On top of that, we are currently working on using LLMs for converting legacy code into the new LTP API.

My talk is supposed to focus on * How do we store and reference sources of - packages; - upstream code repositories; - code streams and products. * The various ways to rebuild our sources - Pure git based build; - Including remote resurces from servers; - SLSA provenance rebuild. * How this enables us to improve or support at all - proven soverignity of our products; - SBOM generation; - product certifications; - WIP/future SLSA 1.2 source tracking support. * If time permits - enablement of AI usage for our packaging thanks to the new build mechanics

Being started in Hack Week 19, the simpledrm graphics driver reached the upstream kernel in 2021. Since then it has been adopted by all major Linux distributions. Simpledrm is nowadays the umbrella term for the Linux kernel's boot-up graphics. It's a success story out of SUSE Labs that has quietly impacted the Linux graphics stack. I did a first presentation on the topic at SUSE Labs 2022. This is a follow-up talk that hopefully wraps up the topic for good. We'll go through the basics, some history, various other drivers, changes we had to do throughout the kernel and user space. We'll look at what worked and what didn't, and what is left to do.

CPU throttling is a tempting mechanism provided by the cgroup CPU controller. The temptation comes from the assumption that by restricting one container A, we ensure sufficient runtime for another container B (e.g. in case of runaways of A). However, this doesn't always hold up because of how Linux kernel is internally organized. Thankfully, there have been some recent changes to CPU scheduler that promise more predictable behavior. This talk will be graphs (and maybe formulas) comparing the old and the new behaviors. (The content loosely builds on top of [Throttling with CPU Controller](https://www.youtube.com/watch?v=-OibmKdl6SQ) from LabsConf 2022.)

In this talk, the cryptography packaging team will present our roadmap: openSSL support, support of forks, upstream plans, our own CI activites. Plans for the next FIPS-certification rounds. And an update on post-quantum resistant cryptography ( PQC ) available across our distributions.

Transparent Hugepages are the way the kernel has to provide userspace with larger granularity chunks of pages, which frequently improves performance. However, popular wisdom is often clouded with hearsay and bad advice. This talk will go over transparent hugepages, what they are and how to get them.

Sheepdog (http://ghttps://github.com/sheepdog/sheepdog) is a rather old (ish) project providing a distributed storage for KVM/Qemu. While originally only meant for Qemu (with direct integration etc) it still is a distributed storage system. And with the help of a sheepdog interface for ublk (block device in userspace) we can export a sheepdog volume as a linux block device, which in turn can be used as a backing store for the NVMe target. In this talk I will be presenting the building blocks required to build a distributed nvme cluster with sheepdog, and (hopefully) give a short demo of the features.

In this session, I will give an overview of the recent changes to the tool and future plans.

Last year, we presented a partially live Git Workflow. Since then, it's been deployed for SLFO and some devel projects with some pain-points and missing features. With TW transition being imminent, this presentation covers: the current state, efficient usage of these workflows, and current and future developments.

When doing hardware enablement for the SUSE kernel it is common to backport individual drivers to a more recent upstream version. For drivers in certain subsystems it can be more advantageous to backport the entire subsystem. This significantly increases the number of patches requiring backporting, which necessitates special tooling. This presentation will demonstrate the process and tools used to backport the DRM subsystem.

As the primary gateway for Secure Boot on Linux, the Shim bootloader is critical for establishing a system’s Chain of Trust. This session provides a technical review into the inner workings of Shim, moving beyond high-level concepts to analyze its source code and internal logic. We will trace the execution flow from the UEFI entry point to the handoff of the next-stage bootloader, focusing on how Shim manages MOK (Machine Owner Key) certificates and performs cryptographic verification of EFI binaries.

Over the past year, the kernel livepatching team’s primary goal has been to reduce the human intervention required for the livepatch creation. Several new approaches and tools have been implemented, ranging from automated patch analysis and monitoring dashboards to new filesystems, and are currently being integrated into the workflow. However, several roadblocks remain. This talk presents the evolution of our workflow, and seeks to foster a collaborative dialogue regarding our remaining challenges.

User space based Full Disk Encryption in openSUSE is old news. We can set up FDE installations using the systemd toolset. This allows us to use traditional passwords, the TPM2 to validate the status before mounting the encrypted device or a FIDO2 key to validate the presence of a token owned by authorized users. To increase the security, in Tumbleweed, we recommend combining both TPM2 and a password (TPM2+PIN). This guarantees the system is in a healthy state and the user has the correct authorization, but it still doesn't support a combination of TPM2 and FIDO2 keys. To fix this issue, we are proposing a new enrollment method in systemd: TPM2+FIDO2. This method will not use the hmac-secret extension but will rely on the more traditional system based on credentials and asserts.

The Rust standard library includes several data types that perform allocations, including Box (regular heap allocations), collections like Vec or HashMap, and reference-counted smart pointers (Rc/Arc). However, stable Rust has two shortcomings in this area: one may not specify which allocator is used, and a failing allocation causes an immediate panic, which is not acceptable for OS-level software. This talk provides an overview of the current status of the related unstable Rust std APIs, how the Rust subsystem in the Linux kernel approaches the same topic, and how we plan to implement something similar for the COCONUT SVSM project (https://github.com/coconut-svsm/svsm).

What tools does a Kernel Security Sentinel rely on to keep the core of the OS safe? This talk provides an overview of the tools ecosystem used by the KSS team. Capabilities of several custom tools will be presented, incl. short demos. Ideas to improve and what you use are indeed welcome.

Inside view into SUSE infrastructure for the Bugzilla service. How it works and what you can looking forward for.

openSUSE MicroOS is a snapshot-based, immutable operating system that features automatic updates and recovery. Health-checker is the system tool responsible for handling automatic recovery and rollbacks, and it comes installed by default. It was recently rewritten to support both systemd-boot and grub2-bls, utilizing systemd-bless-boot and Automatic Boot Assessment. In this talk, we will provide a brief explanation of the Boot Loader Specification (BLS), which is supported by both systemd-boot and grub2-bls. Next, we will explain Automatic Boot Assessment, describe how it is used by health-checker, and show how it can be used to check the system status at boot and act accordingly.

This session decodes how OVMF drives UEFI initialization in virtualized environments, providing a clear walkthrough of critical transitions across the boot phases and the key handoffs between OVMF and the hypervisor. Attendees will gain practical insights for debugging OVMF issues and validating firmware behavior across different virtualized workloads. The talk also covers SUSE’s tailored OVMF builds, highlighting how workload-specific firmware choices improve cryptographic key management and strengthen the trusted execution path from the earliest boot state.

There are several open questions on how to proceed with our git-fixes tracking infrastructure brought up over email, e.g. - tracking through bugzilla; - how aggressive should be the filtering; - the overall purpose of the infrastructure. We have not been able to find consensus over those and discuss them in person might help to align on those topics.

For a kernel branch maintainer some activities seem to be perfect fits for further automation, like better merge conflict handling, but for sure others have different pain points. I'd like to discuss what could be done to: - increase the automatic merge success rate; - decrease the number of potential errors when doing manual merges; - other enhancements of the maintainer workflow

As a QA Engineer, I often support performance developer for gathering metrics and researching performance defects during OS (pre-)validation. A programmable kernel debugger drgn is used during debugging issue, it helped lots to identify and verify the possible performance differences. I would like to share the insights and examples learned while co-working with developer. 1) The basic usage and script of drgn. 2) Example: CPU idle and cost issue, performance profiling and understand results

How are features for SUSE Linux Enterprise Server evaluated and processed? While SLES is supposed to be stable, SUSE is still adding quite some features for new releases, but also during the Maintenance phase of a product. What information is important for requesting a new package version or a completely new technology? What change may pose a risk and how do we make sure no regressions are introduced? In this talk all of these questions will be answered.

Himmelblau provides a unified approach to integrating Linux systems with modern identity providers. This session highlights recent work on Entra ID and Intune integration, alongside new support for OpenID Connect providers like Keycloak. We’ll demonstrate how the browser orchestration model enables consistent authentication and system behavior across different identity backends. A demo will showcase both Entra ID-based and Keycloak-based login flows, illustrating how Himmelblau enables identity flexibility without increasing complexity.

When creating source-based live patches, a key challenge lies in extracting relevant portions of the original codebase into a separate, compact project. Existing automated tools for C projects, such as klp- ccp and clang-extract, address this by invoking source-to-source compilation techniques that construct a “closure” of all required dependencies, thereby minimizing code size and ensuring the result can be compiled independently of external components. In this work, we present a work-in-progress approach that leverages the GCC compiler to eliminate all symbols that are not reachable from the target function within a given compilation unit. This method removes the need for source-to-source compilers and offers improved scalability, extending support beyond C to additional languages such as C++ and Fortran.

You got a bug report assigned and rush to look at it. Suddenly, you recognize a few words “pods”, “nodes”, “CNI”, “cluster”, “workers”, “kubectl”. You vision begins to blur and you deeply wish it were 2015. Your take a deep breath and start working on the report. If that sounds familiar, this talk might help you. We will look at the different options we have to debug an issue related to Kubernetes without deploying a whole Kubernetes cluster. A set of tools I have been using during the last year as a kernel engineer who keeps working on Kubernetes related bugs but does not know Kubernetes at all.

Managing a fleet of KVM hypervisors with shared storage introduces coordination challenges that cluster managers tend to solve with heavyweight infrastructure. VirtX takes a different approach: a lightweight Go service running on each host, using libvirt for VM lifecycle management, serf for decentralized host membership and events, and sanlock for distributed resource lease management directly on shared block storage. VirtX is an open source platform that is simple to reason about, offers its full functionality through a REST API, has no external coordination service dependencies beyond the shared storage itself, and makes its safety guarantees explicit through the storage layer rather than through distributed consensus. This talk covers the requirements, architecture trade-offs and the practical engineering decisions made along the way, upstream contributions to sanlock and pending upstream changes to libvirt necessary to realize a resource provisioning and de-provisioning lifecycle. In addition to manual installs, VMs can be provisioned using cloud-ready images and cloud-init support for first-boot configuration. We will also touch on experimental AI integration, and provide a demo of the current implementation, as well as highlighting missing functionality and potential improvements.

How can a compiler like GCC tell pointers apart? I use 'int *p' and 'int *q' in my program. Can they ever point to the same int? Questions like this one are extremely important for compiler optimization. One of the mechanisms that can answer them is "points-to analysis". In particular, if you're trying to analyze pointers that get passed around between functions, you need interprocedural points-to analysis (IPA PTA). However, you have probably never compiled with IPA PTA. GCC doesn't enable it by default because it doesn't scale well enough. My ambitious project is to change that! In this talk I'll explain what points-to analysis is. I'll talk about how my approach -- using Steensgaard's algorithm -- is different from what GCC currently does. Finally, I'll show some examples of what GCC can optimize with my modifications that it couldn't before.

Fast RISC-V hardware is still hard to get and QEMU is slow. Can we do better? RISC86 allows to boot a RISC-V Linux kernel on x86 hardware, providing direct access to hardware with minimal overhead. This talk starts with a deep dive into the RISC-V system architecture, how it compares to x86 and explains the hacks^W novel techniques used by RISC86 to achieve minimal overhead. Includes some info about the histories of RISC-V and x86 as well as some rants about weirdnesses in those architectures.

Filesystem notification framework (fsnotify) is a Linux kernel subsystem for notifications about changes in the filesystem. Currently this functionality is mostly exposed to userspace through fanotify interface and syscalls. In this talk we first give a quick overview of basic fanotify functionality and then go over a new functionality that fanotify has acquired since the last update I gave in 2020. Support for unpriviledged users using fanotify, support for notification events to implement hierarchical storage management, support for reporting of filesystem errors through fanotify, and support for notification about mount events. We also briefly overview new features under discussion.

In an increasingly complex hardware landscape, ensuring seamless interoperability between operating systems and diverse hardware platforms is critical for mitigating risk and reducing total cost of ownership. This session provides a deep dive into the SUSE YES Certification program, a hardware assurance standard with a legacy spanning over 38 years. We will explore the technical rigors of the certification process, detailing how SUSE Partner Engineering collaborates with Independent Hardware Vendors (IHVs) to resolve compatibility issues before they reach the end-user. A central focus will be the SUSE YES System Certification Kit (SCK), the specialized test suite used to validate everything from CPU, memory, storage and network stress testing to advanced power management, persistent memory, GPU accelerators, TPM and more across x86_64, Arm, IBM Power, and IBM Z architectures.

Discussion about SUSE internal kernel development infrastructure

I will provide an update on Orthos development in the past year and what's coming next.

In this session we discuss the topic of AI with the focus on our work and workplace; both internal and in upstream projects. Since earlier this year I've been experimenting with Gemini and how to integrate it into my workflow. I'll open up the discussion with results and thoughts on how well this has worked so far. Afterwards you're welcome to report on your findings, experience, thoughts, etc. This is an open discussion and everyone in Labs is welcome to participate.

This session dives into the core mechanisms of Linux kernel tracing. We explore dynamic instrumentation such as exception-based kprobe and code-patching ftrace, comparing internals and performance overhead. The talk also covers low-overhead, statically defined tracepoints and counter-based sampling via perf_event. Finally, we introduce BPF as a programmable execution layer that reuses and enhances tracing sources for flexible and efficient kernel observation.

A detailed overview on virtualising SAP workloads with SUSE KVM with a focus on virtualising SAP HANA with SUSE KVM.

During the SLE16 SAP validation cycle, our QE team found an interesting performance regression affecting a single cpu generation. In this talk, I'll retrace my steps, from running the workload inside the SAP infrastructure down to identifying the specific micro-architectural interaction that caused the issue. In parallel, I'll present methods to investigate real-world performance bug in constrained environment, with near black-box workloads. Beyond an interesting story, these methods are useful to anyone investigating bugs directly in the SAP infrastructure in the future.