Nvidia

Setting up NVIDIA Tesla P100

the P100 was choses for balance between performance and price (second hand ebay)

Aliexpress

PCIE 4.0 3.0 16X Riser Kabel 90/180 Graden Mount Video Grafisch

Gpu 10pin Naar 1x8pin Dual 8(6 + 2)Pin Power Adapter Kabel Voor Hp Dl380 Gen8 Gen9 Server

Cutting the Riser

a special Riser for the HP Proliant gen8 is a good idea because it has a PCIE 16x in the middle which allows for the GPU to be plugged in. (hard to find and expensive)

I choose the cheap option (evidently) and cut a hole in a riser to extend with a riser cable into this riser. Some more cutting had to be done because the GPU is too long.

For the gen9 there a cheaper riser card options, so no cutting needed…

Configure the BIOS

(changed IRQ for network card to 11, since conflict)

enter BIOS (F9) and on main bios screen / “Service options” menu item. Under this, enable “PCI Express 64-bit BAR

configure proxmox host

https://digitalspaceport.com/proxmox-lxc-gpu-passthru-setup-guide/

PROXMOX PCI-E GPU passthru

When running proxmox on this hardware, there is more config needed to enable passthru of this GPU to VM. Enable IOMMU

the idea

the idea is to get nvidia to work on the proxmox host (there will be kernel modules)

for the LXC machines there is no need for kernel drivers!! since already on the host

Download NVIDIA drivers

adapt lxc.conf

/etc/pve/lxc/105.conf

Append these values with the IDs you noted earlier to your file like so. Note the placement of the 195, 234 and 509. This is for a SINGLE gpu also, if you have multiple add additional

copy NVIDIA driver to LXC

Install the NVIDIA Container Toolkit

install ollama docker container for GPU usage

#in portainer you still need to start ollama this way: