1. Documentation
    2. history bug_report picture_as_pdf
×

OpenShift Virtualization release notes

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

About Red Hat OpenShift Virtualization

Red Hat OpenShift Virtualization enables you to bring traditional virtual machines (VMs) into OpenShift Container Platform where they run alongside containers, and are managed as native Kubernetes objects.

OpenShift Virtualization is represented by the OpenShift Virtualization icon.

You can use OpenShift Virtualization with either the OVN-Kubernetes or the OpenShiftSDN default Container Network Interface (CNI) network provider.

Learn more about what you can do with OpenShift Virtualization.

Learn more about OpenShift Virtualization architecture and deployments.

Prepare your cluster for OpenShift Virtualization.

OpenShift Virtualization supported cluster version

OpenShift Virtualization 4.13 is supported for use on OpenShift Container Platform Branch Build clusters. To use the latest z-stream release of OpenShift Virtualization, you must first upgrade to the latest version of OpenShift Container Platform.

Supported guest operating systems

To view the supported guest operating systems for OpenShift Virtualization, refer to Certified Guest Operating Systems in Red Hat OpenStack Platform, Red Hat Virtualization and OpenShift Virtualization.

New and changed features

  • OpenShift Virtualization is certified in Microsoft’s Windows Server Virtualization Validation Program (SVVP) to run Windows Server workloads.

    The SVVP Certification applies to:

    • Red Hat Enterprise Linux CoreOS workers. In the Microsoft SVVP Catalog, they are named Red Hat OpenShift Container Platform 4 on RHEL CoreOS 9.

    • Intel and AMD CPUs.

  • OpenShift Virtualization is now based on Red Hat Enterprise Linux (RHEL) 9.

    • There is a new RHEL 9 machine type for VMs: machineType: pc-q35-rhel9.2.0.

      All VM templates that are included with OpenShift Virtualization now use this machine type by default.

  • The "Logging, events, and monitoring" documentation is now called Support. The monitoring tools documentation has been moved to Monitoring.

Quick starts

  • Quick start tours are available for several OpenShift Virtualization features. To view the tours, click the Help icon ? in the menu bar on the header of the OpenShift Virtualization console and then select Quick Starts. You can filter the available tours by entering the virtualization keyword in the Filter field.

Installation

Networking

Storage

  • OpenShift Virtualization storage resources now migrate automatically to the beta API versions. Alpha API versions are no longer supported.

Web console

  • On the VirtualMachine details page, the Scheduling, Environment, Network interfaces, Disks, and Scripts tabs are displayed on the new Configuration tab.

  • You can now paste a string from your client’s clipboard into the guest when using the VNC console.

  • The VirtualMachine details → Details tab now provides a new SSH service type SSH over LoadBalancer to expose the SSH service over a load balancer.

  • The option to make a hot-plug volume a persistent volume is added to the Disks tab.

Deprecated and removed features

Deprecated features

Deprecated features are included in the current release and supported. However, they will be removed in a future release and are not recommended for new deployments.

Removed features

Removed features are not supported in the current release.

  • Red Hat Enterprise Linux 6 is no longer supported on OpenShift Virtualization.

  • Support for the legacy HPP custom resource, and the associated storage class, has been removed for all new deployments. In OpenShift Virtualization 4.13, the HPP Operator uses the Kubernetes Container Storage Interface (CSI) driver to configure local storage. A legacy HPP custom resource is supported only if it had been installed on a previous version of OpenShift Virtualization.

  • OpenShift Virtualization 4.11 removed support for nmstate, including the following objects:

    • NodeNetworkState

    • NodeNetworkConfigurationPolicy

    • NodeNetworkConfigurationEnactment

    To preserve and support your existing nmstate configuration, install the Kubernetes NMState Operator before updating to OpenShift Virtualization 4.11. For 4.13 for Extended Update Support (EUS) versions, install the Kubernetes NMState Operator after updating to 4.13. You can install the Operator from the OperatorHub in the OpenShift Container Platform web console, or by using the OpenShift CLI (oc).

  • The Node Maintenance Operator (NMO) is no longer shipped with OpenShift Virtualization. You can install the NMO from the OperatorHub in the OpenShift Container Platform web console, or by using the OpenShift CLI (oc).

    You must perform one of the following tasks before updating to OpenShift Virtualization 4.11 from OpenShift Virtualization 4.10.2 and later 4.10 releases. For Extended Update Support (EUS) versions, you must perform the following tasks before updating to OpenShift Virtualization 4.13 from 4.10.2 and later 4.10 releases:

    • Move all nodes out of maintenance mode.

    • Install the standalone NMO and replace the nodemaintenances.nodemaintenance.kubevirt.io custom resource (CR) with a nodemaintenances.nodemaintenance.medik8s.io CR.

Notable technical changes

Technology Preview features

Some features in this release are currently in Technology Preview. These experimental features are not intended for production use. Note the following scope of support on the Red Hat Customer Portal for these features:

Technology Preview Features Support Scope

  • You can now use Prometheus to monitor the following metrics:

    • kubevirt_vmi_cpu_system_usage_seconds returns the physical system CPU time consumed by the hypervisor.

    • kubevirt_vmi_cpu_user_usage_seconds returns the physical user CPU time consumed by the hypervisor.

    • kubevirt_vmi_cpu_usage_seconds returns the total CPU time used in seconds by calculating the sum of the vCPU and the hypervisor usage.

  • You can now run a checkup to verify if your OpenShift Container Platform cluster node can run a virtual machine with a Data Plane Development Kit (DPDK) workload with zero packet loss.

  • The Tekton Tasks Operator (TTO) now integrates OpenShift Virtualization with Red Hat OpenShift Pipelines. TTO includes cluster tasks and example pipelines that allow you to:

    • Create and manage virtual machines (VMs), persistent volume claims (PVCs), and data volumes.

    • Run commands in VMs.

    • Manipulate disk images with libguestfs tools.

    • Install Windows 10 into a new data volume from a Windows installation image (ISO file).

    • Customize a basic Windows 10 installation and then create a new image and template.

  • You can now use the guest agent ping probe to determine if the QEMU guest agent is running on a virtual machine.

  • You can now use Microsoft Windows 11 as a guest operating system. However, OpenShift Virtualization 4.13 does not support USB disks, which are required for a critical function of BitLocker recovery. To protect recovery keys, use other methods described in the BitLocker recovery guide.

  • You can create live migration policies with specific parameters, such as bandwidth usage, maximum number of parallel migrations, and timeout, and apply the policies to groups of virtual machines by using virtual machine and namespace labels.

Bug fixes

Known issues

  • You cannot run OpenShift Virtualization on a single-stack IPv6 cluster. (BZ#2193267)

  • When you use two pods with different SELinux contexts, VMs with the ocs-storagecluster-cephfs storage class fail to migrate and the VM status changes to Paused. This is because both pods try to access the shared ReadWriteMany CephFS volume at the same time. (BZ#2092271)

    • As a workaround, use the ocs-storagecluster-ceph-rbd storage class to live migrate VMs on a cluster that uses Red Hat Ceph Storage.

  • The TopoLVM provisioner name string has changed in OpenShift Virtualization 4.12. As a result, the automatic import of operating system images might fail with the following error message (BZ#2158521):

    DataVolume.storage spec is missing accessMode and volumeMode, cannot get access mode from StorageProfile.

    • As a workaround:

      1. Update the claimPropertySets array of the storage profile:

        $ oc patch storageprofile <storage_profile> --type=merge -p '{"spec": {"claimPropertySets": [{"accessModes": ["ReadWriteOnce"], "volumeMode": "Block"}, \
    {"accessModes": ["ReadWriteOnce"], "volumeMode": "Filesystem"}]}}'

      2. Delete the affected data volumes in the openshift-virtualization-os-images namespace. They are recreated with the access mode and volume mode from the updated storage profile.

  • When restoring a VM snapshot for storage whose binding mode is WaitForFirstConsumer, the restored PVCs remain in the Pending state and the restore operation does not progress.

    • As a workaround, start the restored VM, stop it, and then start it again. The VM will be scheduled, the PVCs will be in the Bound state, and the restore operation will complete. (BZ#2149654)

  • VMs created from common templates on a Single Node OpenShift (SNO) cluster display a VMCannotBeEvicted alert because the template’s default eviction strategy is LiveMigrate. You can ignore this alert or remove the alert by updating the VM’s eviction strategy. (BZ#2092412)

  • Uninstalling OpenShift Virtualization does not remove the feature.node.kubevirt.io node labels created by OpenShift Virtualization. You must remove the labels manually. (CNV-22036)

  • Some persistent volume claim (PVC) annotations created by the Containerized Data Importer (CDI) can cause the virtual machine snapshot restore operation to hang indefinitely. (BZ#2070366)

    • As a workaround, you can remove the annotations manually:

      1. Obtain the VirtualMachineSnapshotContent custom resource (CR) name from the status.virtualMachineSnapshotContentName value in the VirtualMachineSnapshot CR.

      2. Edit the VirtualMachineSnapshotContent CR and remove all lines that contain k8s.io/cloneRequest.

      3. If you did not specify a value for spec.dataVolumeTemplates in the VirtualMachine object, delete any DataVolume and PersistentVolumeClaim objects in this namespace where both of the following conditions are true:

        1. The object’s name begins with restore-.

        2. The object is not referenced by virtual machines.

          This step is optional if you specified a value for spec.dataVolumeTemplates.

      4. Repeat the restore operation with the updated VirtualMachineSnapshot CR.

  • Windows 11 virtual machines do not boot on clusters running in FIPS mode. Windows 11 requires a TPM (trusted platform module) device by default. However, the swtpm (software TPM emulator) package is incompatible with FIPS. (BZ#2089301)

  • If your OpenShift Container Platform cluster uses OVN-Kubernetes as the default Container Network Interface (CNI) provider, you cannot attach a Linux bridge or bonding device to a host’s default interface because of a change in the host network topology of OVN-Kubernetes. (BZ#1885605)

    • As a workaround, you can use a secondary network interface connected to your host, or switch to the OpenShift SDN default CNI provider.

  • In some instances, multiple virtual machines can mount the same PVC in read-write mode, which might result in data corruption. (BZ#1992753)

    • As a workaround, avoid using a single PVC in read-write mode with multiple VMs.

  • The Pod Disruption Budget (PDB) prevents pod disruptions for migratable virtual machine images. If the PDB detects pod disruption, then openshift-monitoring sends a PodDisruptionBudgetAtLimit alert every 60 minutes for virtual machine images that use the LiveMigrate eviction strategy. (BZ#2026733)

  • OpenShift Virtualization links a service account token in use by a pod to that specific pod. OpenShift Virtualization implements a service account volume by creating a disk image that contains a token. If you migrate a VM, then the service account volume becomes invalid. (BZ#2037611)

    • As a workaround, use user accounts rather than service accounts because user account tokens are not bound to a specific pod.

  • If you clone more than 100 VMs using the csi-clone cloning strategy, then the Ceph CSI might not purge the clones. Manually deleting the clones can also fail. (BZ#2055595)

    • As a workaround, you can restart the ceph-mgr to purge the VM clones.