The SDK for XenServer 6.2.0 Service Pack 1 is ideal for developers wishing to access programmatically XenServer’s new management features for GPU virtualization (including the new vGPU and GPU pass-through). The five available XenServer SDKs, one for each of C, C#, Java, PowerShell and Python, expose the new XenAPI commands for working with physical GPUs (pGPUs), GPU groups, virtual GPUs (vGPUs) and virtual GPU types.

The GPU technologies for XenDesktop and XenServer do of course also come with rich GUI configuration operations and provisioning via XenCenter, XenDesktop and MCS.

Over our last few development cycles we have invested in re-writing XenServer’s PowerShell API to provide developers and administrators with a PoSH alternative to using the XenServer command line (CLI) interface. In particular this interface is proving popular with those looking to automate bespoke vGPU and GPU pass-through configuration and benchmarking or auto-test frameworks. The PowerShell API is also a popular choice for XenDesktop and Windows administrators working with XenServer.

The following example is a tutorial of a typical vGPU configuration use case using the XenServer PowerShell SDK cmdlets. It may be useful to read my colleague Si’s blog on the XenServer object model provided for GPUs as background.

The example output provided is from a server with both a NVIDIA Grid K1 and K2 card installed.

We start by connecting to the server:

PS C:\> Connect-XenServer -Server server -UserName username -Password password

The first thing we probably want to check is which GPU groups exist (these have been created automatically once the graphics hardware is installed):

PS C:\> Get-XenGPUGroup | select name_label, GPU_types, allocation_algorithm

name_label                                             GPU_types   allocation_algorithm
----------                                             ---------   --------------------
Group of NVIDIA Corporation GK104GL [GRID K2] GPUs   {10de/11bf}            depth_first
Group of NVIDIA Corporation GK107GL [GRID K1] GPUs   {10de/0ff2}          breadth_first

The allocation_algorithm is the placement policy for assigning VMs to GPUs  in order to achieve either maximum density by placing as many VMs as possible on the same GPU (depth_first), or maximum performance by placing VMs on as many GPUs as possible (breadth_first). For example, the following command sets the allocation_algorithm to achieve maximum performance:

PS C:\> Get-XenGPUGroup | Set-XenGPUGroup -AllocationAlgorithm breadth_first
PS C:\> Get-XenGPUGroup | select name_label, GPU_types, allocation_algorithm

name_label                                             GPU_types    allocation_algorithm
----------                                             ---------    --------------------
Group of NVIDIA Corporation GK104GL [GRID K2] GPUs   {10de/11bf}           breadth_first
Group of NVIDIA Corporation GK107GL [GRID K1] GPUs   {10de/0ff2}           breadth_first

Now we can list some information about the vGPU types. The vGPU types are pre-sets which can be used to create different kinds of vGPUs.

PS C:\> Get-XenVGPUType | ft vendor_name, model_name, framebuffer_size, max_heads, `
>> max_resolution_x, max_resolution_y
>>

vendor_name           model_name   framebuffer_size    max_heads   max_resolution_x   max_resolution_y
-----------           ----------   ----------------    ---------   ----------------   ----------------
NVIDIA Corporation    GRID K100           268435456            2               1920               1200
NVIDIA Corporation    GRID K140Q         1006632960            2               2560               1600
NVIDIA Corporation    GRID K240Q         1006632960            2               2560               1600
NVIDIA Corporation    GRID K260Q         2013265920            4               2560               1600
NVIDIA Corporation    GRID K200           268435456            2               1920               1200
                      passthrough                 0            0                  0                  0

The vGPU type “passthrough” is supported for all PCI display devices, and can be used to create pass-through vGPUs as supported since XenServer 6.0.

We can see for which PCI display devices a vGPU type is supported as follows:

PS C:\> Get-XenVGPUType | Where {$_.model_name -eq "GRID K100"} `
>> | Get-XenVGPUTypeProperty -XenProperty SupportedOnPGPUs `
>> | Get-XenPGPU | Get-XenPCI -Ref {$_.PCI} | select device_name, pci_id
>>

device_name          pci_id
-----------          ------
GK107GL [GRID K1]    0000:0a:00.0
GK107GL [GRID K1]    0000:07:00.0
GK107GL [GRID K1]    0000:08:00.0
GK107GL [GRID K1]    0000:09:00.0

A vGPU type will show up as supported or enabled in a GPU group if it is supported or enabled respectively on at least one of the pGPUs in the group. We can query the GPU groups to find out which vGPU types are supported or enabled. For example:

PS C:\> Get-XenGPUGroup | Where {$_.name_label -match "GRID K2"} `
>> | Get-XenGPUGroupProperty -XenProperty EnabledVGPUTypes `
>> | Get-XenVGPUType | select model_name
>>

model_name
----------
GRID K200
passthrough
GRID K260Q
GRID K240Q

We may want to disallow a certain vGPU type on a pGPU. For example, the following commands disable and then re-enable the vGPU type Grid K240Q:

PS C:\> $vgpuType = Get-XenVgpuType | where {$_.model_name -eq "GRID K240Q"}
PS C:\> $vgpuType | Get-XenVGPUTypeProperty -XenProperty EnabledOnPGPUs `
>> | Get-XenPGPU | select uuid
>>

uuid
----
a913a90d-0be9-aea3-862f-3fbfe1823a3f
8a0d1316-0f09-1ee9-f95b-c778c759ee40

PS C:\> Remove-XenPGPUProperty -Uuid a913a90d-0be9-aea3-862f-3fbfe1823a3f `
>> -EnabledVGPUTypes $vgputype.opaque_ref
>>

PS C:\> $vgpuType | Get-XenVGPUTypeProperty -XenProperty EnabledOnPGPUs |get-xenpgpu | select uuid

uuid
----
8a0d1316-0f09-1ee9-f95b-c778c759ee40

PS C:\> Add-XenPGPU -Uuid a913a90d-0be9-aea3-862f-3fbfe1823a3f -EnabledVGPUTypes $vgputype.opaque_ref
PS C:\> $vgpuType | Get-XenVGPUTypeProperty -XenProperty EnabledOnPGPUs |get-xenpgpu | select uuid

uuid
----
a913a90d-0be9-aea3-862f-3fbfe1823a3f
8a0d1316-0f09-1ee9-f95b-c778c759ee40

We may want to find out how many vGPUs of a certain type can be started on the pGPUs in a group, in addition to the vGPUs which are already running:

PS C:\> $gpuGroups = Get-XenGPUGroup
PS C:\> Get-XenVGPUtype | % {Get-XenGPUGroupProperty $gpuGroups[0] `
>> -XenProperty RemainingCapacity -VgpuType $_}
>>
0
0
8
4
16
2

Now suppose we want to assign a vGPU of type Grid K260Q to a VM which has no vGPU yet. This can be done as follows:

PS C:\> $vm = Get-XenVm -Name "w7-test"
PS C:\> $vgpuTypes = Get-XenVGPUtype
PS C:\> Get-XenVMProperty -VM $vm -XenProperty VGPUs
PS C:\>
PS C:\> New-XenVGPU -VM $vm -GPUGroup $gpuGroups[0] -Device 0 -Type $vgpuTypes[3] -PassThru

uuid               : f1122947-3b11-3fd3-0630-779701b37265
VM                 : XenAPI.XenRef`1[XenAPI.VM]
GPU_group          : XenAPI.XenRef`1[XenAPI.GPU_group]
device             : 0
currently_attached : False
other_config       : {}
type               : XenAPI.XenRef`1[XenAPI.VGPU_type]
resident_on        : XenAPI.XenRef`1[XenAPI.PGPU]
opaque_ref         : OpaqueRef:15b2d1a9-8944-2f28-53df-6b8274d4d6fb
Changed            : True

At this stage we can boot the VM, install the in-guest NVIDIA drivers, and enable RDP. Then we may want to disable the VNC console as this has a significant performance overhead. To do this we need to shut down the VM, set the flag vgpu_vnc_enabled to false and then boot the VM.

PS C:\> Invoke-XenVM -VM $vm -XenAction Shutdown –Async –PassThru `
>> | Wait-XenTask –ShowProgress
>>
PS C:\> $p = Get-XenVMProperty -VM $vm -XenProperty Platform
PS C:\> $p["vgpu_vnc_enabled"]="false"
PS C:\> Set-XenVM -VM $vm -Platform $p
PS C:\> Invoke-XenVM -VM $vm -XenAction Start –Async –PassThru `
>> | Wait-XenTask –ShowProgress
>>

 

 

Before finishing we should remember to disconnect from the server:

PS C:\> Get-XenSession | Disconnect-XenServer

We have an active community of PowerShell users over on the XenServer SDK forum, where the community and development engineers answer questions and share expertise, so if you have any questions please do head over there.