1		PCI Express I/O Virtualization Howto
  2		Copyright (C) 2009 Intel Corporation
  3		    Yu Zhao <yu.zhao@intel.com>
  4
  5		Update: November 2012
  6			-- sysfs-based SRIOV enable-/disable-ment
  7		Donald Dutile <ddutile@redhat.com>
  8
  91. Overview
 10
 111.1 What is SR-IOV
 12
 13Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended
 14capability which makes one physical device appear as multiple virtual
 15devices. The physical device is referred to as Physical Function (PF)
 16while the virtual devices are referred to as Virtual Functions (VF).
 17Allocation of the VF can be dynamically controlled by the PF via
 18registers encapsulated in the capability. By default, this feature is
 19not enabled and the PF behaves as traditional PCIe device. Once it's
 20turned on, each VF's PCI configuration space can be accessed by its own
 21Bus, Device and Function Number (Routing ID). And each VF also has PCI
 22Memory Space, which is used to map its register set. VF device driver
 23operates on the register set so it can be functional and appear as a
 24real existing PCI device.
 25
 262. User Guide
 27
 282.1 How can I enable SR-IOV capability
 29
 30Multiple methods are available for SR-IOV enablement.
 31In the first method, the device driver (PF driver) will control the
 32enabling and disabling of the capability via API provided by SR-IOV core.
 33If the hardware has SR-IOV capability, loading its PF driver would
 34enable it and all VFs associated with the PF.  Some PF drivers require
 35a module parameter to be set to determine the number of VFs to enable.
 36In the second method, a write to the sysfs file sriov_numvfs will
 37enable and disable the VFs associated with a PCIe PF.  This method
 38enables per-PF, VF enable/disable values versus the first method,
 39which applies to all PFs of the same device.  Additionally, the
 40PCI SRIOV core support ensures that enable/disable operations are
 41valid to reduce duplication in multiple drivers for the same
 42checks, e.g., check numvfs == 0 if enabling VFs, ensure
 43numvfs <= totalvfs.
 44The second method is the recommended method for new/future VF devices.
 45
 462.2 How can I use the Virtual Functions
 47
 48The VF is treated as hot-plugged PCI devices in the kernel, so they
 49should be able to work in the same way as real PCI devices. The VF
 50requires device driver that is same as a normal PCI device's.
 51
 523. Developer Guide
 53
 543.1 SR-IOV API
 55
 56To enable SR-IOV capability:
 57(a) For the first method, in the driver:
 58	int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
 59	'nr_virtfn' is number of VFs to be enabled.
 60(b) For the second method, from sysfs:
 61	echo 'nr_virtfn' > \
 62        /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
 63
 64To disable SR-IOV capability:
 65(a) For the first method, in the driver:
 66	void pci_disable_sriov(struct pci_dev *dev);
 67(b) For the second method, from sysfs:
 68	echo  0 > \
 69        /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs
 70
 713.2 Usage example
 72
 73Following piece of code illustrates the usage of the SR-IOV API.
 74
 75static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id)
 76{
 77	pci_enable_sriov(dev, NR_VIRTFN);
 78
 79	...
 80
 81	return 0;
 82}
 83
 84static void dev_remove(struct pci_dev *dev)
 85{
 86	pci_disable_sriov(dev);
 87
 88	...
 89}
 90
 91static int dev_suspend(struct pci_dev *dev, pm_message_t state)
 92{
 93	...
 94
 95	return 0;
 96}
 97
 98static int dev_resume(struct pci_dev *dev)
 99{
100	...
101
102	return 0;
103}
104
105static void dev_shutdown(struct pci_dev *dev)
106{
107	...
108}
109
110static int dev_sriov_configure(struct pci_dev *dev, int numvfs)
111{
112	if (numvfs > 0) {
113		...
114		pci_enable_sriov(dev, numvfs);
115		...
116		return numvfs;
117	}
118	if (numvfs == 0) {
119		....
120		pci_disable_sriov(dev);
121		...
122		return 0;
123	}
124}
125
126static struct pci_driver dev_driver = {
127	.name =		"SR-IOV Physical Function driver",
128	.id_table =	dev_id_table,
129	.probe =	dev_probe,
130	.remove =	dev_remove,
131	.suspend =	dev_suspend,
132	.resume =	dev_resume,
133	.shutdown =	dev_shutdown,
134	.sriov_configure = dev_sriov_configure,
135};