Loading...
1.. SPDX-License-Identifier: GPL-2.0
2.. _xfrm_device:
3
4===============================================
5XFRM device - offloading the IPsec computations
6===============================================
7
8Shannon Nelson <shannon.nelson@oracle.com>
9Leon Romanovsky <leonro@nvidia.com>
10
11
12Overview
13========
14
15IPsec is a useful feature for securing network traffic, but the
16computational cost is high: a 10Gbps link can easily be brought down
17to under 1Gbps, depending on the traffic and link configuration.
18Luckily, there are NICs that offer a hardware based IPsec offload which
19can radically increase throughput and decrease CPU utilization. The XFRM
20Device interface allows NIC drivers to offer to the stack access to the
21hardware offload.
22
23Right now, there are two types of hardware offload that kernel supports.
24 * IPsec crypto offload:
25 * NIC performs encrypt/decrypt
26 * Kernel does everything else
27 * IPsec packet offload:
28 * NIC performs encrypt/decrypt
29 * NIC does encapsulation
30 * Kernel and NIC have SA and policy in-sync
31 * NIC handles the SA and policies states
32 * The Kernel talks to the keymanager
33
34Userland access to the offload is typically through a system such as
35libreswan or KAME/raccoon, but the iproute2 'ip xfrm' command set can
36be handy when experimenting. An example command might look something
37like this for crypto offload:
38
39 ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
40 reqid 0x07 replay-window 32 \
41 aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
42 sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
43 offload dev eth4 dir in
44
45and for packet offload
46
47 ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
48 reqid 0x07 replay-window 32 \
49 aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
50 sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
51 offload packet dev eth4 dir in
52
53 ip x p add src 14.0.0.70 dst 14.0.0.52 offload packet dev eth4 dir in
54 tmpl src 14.0.0.70 dst 14.0.0.52 proto esp reqid 10000 mode transport
55
56Yes, that's ugly, but that's what shell scripts and/or libreswan are for.
57
58
59
60Callbacks to implement
61======================
62
63::
64
65 /* from include/linux/netdevice.h */
66 struct xfrmdev_ops {
67 /* Crypto and Packet offload callbacks */
68 int (*xdo_dev_state_add) (struct xfrm_state *x, struct netlink_ext_ack *extack);
69 void (*xdo_dev_state_delete) (struct xfrm_state *x);
70 void (*xdo_dev_state_free) (struct xfrm_state *x);
71 bool (*xdo_dev_offload_ok) (struct sk_buff *skb,
72 struct xfrm_state *x);
73 void (*xdo_dev_state_advance_esn) (struct xfrm_state *x);
74 void (*xdo_dev_state_update_stats) (struct xfrm_state *x);
75
76 /* Solely packet offload callbacks */
77 int (*xdo_dev_policy_add) (struct xfrm_policy *x, struct netlink_ext_ack *extack);
78 void (*xdo_dev_policy_delete) (struct xfrm_policy *x);
79 void (*xdo_dev_policy_free) (struct xfrm_policy *x);
80 };
81
82The NIC driver offering ipsec offload will need to implement callbacks
83relevant to supported offload to make the offload available to the network
84stack's XFRM subsystem. Additionally, the feature bits NETIF_F_HW_ESP and
85NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload.
86
87
88
89Flow
90====
91
92At probe time and before the call to register_netdev(), the driver should
93set up local data structures and XFRM callbacks, and set the feature bits.
94The XFRM code's listener will finish the setup on NETDEV_REGISTER.
95
96::
97
98 adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
99 adapter->netdev->features |= NETIF_F_HW_ESP;
100 adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;
101
102When new SAs are set up with a request for "offload" feature, the
103driver's xdo_dev_state_add() will be given the new SA to be offloaded
104and an indication of whether it is for Rx or Tx. The driver should
105
106 - verify the algorithm is supported for offloads
107 - store the SA information (key, salt, target-ip, protocol, etc)
108 - enable the HW offload of the SA
109 - return status value:
110
111 =========== ===================================
112 0 success
113 -EOPNETSUPP offload not supported, try SW IPsec,
114 not applicable for packet offload mode
115 other fail the request
116 =========== ===================================
117
118The driver can also set an offload_handle in the SA, an opaque void pointer
119that can be used to convey context into the fast-path offload requests::
120
121 xs->xso.offload_handle = context;
122
123
124When the network stack is preparing an IPsec packet for an SA that has
125been setup for offload, it first calls into xdo_dev_offload_ok() with
126the skb and the intended offload state to ask the driver if the offload
127will serviceable. This can check the packet information to be sure the
128offload can be supported (e.g. IPv4 or IPv6, no IPv4 options, etc) and
129return true of false to signify its support.
130
131Crypto offload mode:
132When ready to send, the driver needs to inspect the Tx packet for the
133offload information, including the opaque context, and set up the packet
134send accordingly::
135
136 xs = xfrm_input_state(skb);
137 context = xs->xso.offload_handle;
138 set up HW for send
139
140The stack has already inserted the appropriate IPsec headers in the
141packet data, the offload just needs to do the encryption and fix up the
142header values.
143
144
145When a packet is received and the HW has indicated that it offloaded a
146decryption, the driver needs to add a reference to the decoded SA into
147the packet's skb. At this point the data should be decrypted but the
148IPsec headers are still in the packet data; they are removed later up
149the stack in xfrm_input().
150
151 find and hold the SA that was used to the Rx skb::
152
153 get spi, protocol, and destination IP from packet headers
154 xs = find xs from (spi, protocol, dest_IP)
155 xfrm_state_hold(xs);
156
157 store the state information into the skb::
158
159 sp = secpath_set(skb);
160 if (!sp) return;
161 sp->xvec[sp->len++] = xs;
162 sp->olen++;
163
164 indicate the success and/or error status of the offload::
165
166 xo = xfrm_offload(skb);
167 xo->flags = CRYPTO_DONE;
168 xo->status = crypto_status;
169
170 hand the packet to napi_gro_receive() as usual
171
172In ESN mode, xdo_dev_state_advance_esn() is called from xfrm_replay_advance_esn().
173Driver will check packet seq number and update HW ESN state machine if needed.
174
175Packet offload mode:
176HW adds and deletes XFRM headers. So in RX path, XFRM stack is bypassed if HW
177reported success. In TX path, the packet lefts kernel without extra header
178and not encrypted, the HW is responsible to perform it.
179
180When the SA is removed by the user, the driver's xdo_dev_state_delete()
181and xdo_dev_policy_delete() are asked to disable the offload. Later,
182xdo_dev_state_free() and xdo_dev_policy_free() are called from a garbage
183collection routine after all reference counts to the state and policy
184have been removed and any remaining resources can be cleared for the
185offload state. How these are used by the driver will depend on specific
186hardware needs.
187
188As a netdev is set to DOWN the XFRM stack's netdev listener will call
189xdo_dev_state_delete(), xdo_dev_policy_delete(), xdo_dev_state_free() and
190xdo_dev_policy_free() on any remaining offloaded states.
191
192Outcome of HW handling packets, the XFRM core can't count hard, soft limits.
193The HW/driver are responsible to perform it and provide accurate data when
194xdo_dev_state_update_stats() is called. In case of one of these limits
195occuried, the driver needs to call to xfrm_state_check_expire() to make sure
196that XFRM performs rekeying sequence.
1.. SPDX-License-Identifier: GPL-2.0
2
3===============================================
4XFRM device - offloading the IPsec computations
5===============================================
6
7Shannon Nelson <shannon.nelson@oracle.com>
8
9
10Overview
11========
12
13IPsec is a useful feature for securing network traffic, but the
14computational cost is high: a 10Gbps link can easily be brought down
15to under 1Gbps, depending on the traffic and link configuration.
16Luckily, there are NICs that offer a hardware based IPsec offload which
17can radically increase throughput and decrease CPU utilization. The XFRM
18Device interface allows NIC drivers to offer to the stack access to the
19hardware offload.
20
21Userland access to the offload is typically through a system such as
22libreswan or KAME/raccoon, but the iproute2 'ip xfrm' command set can
23be handy when experimenting. An example command might look something
24like this::
25
26 ip x s add proto esp dst 14.0.0.70 src 14.0.0.52 spi 0x07 mode transport \
27 reqid 0x07 replay-window 32 \
28 aead 'rfc4106(gcm(aes))' 0x44434241343332312423222114131211f4f3f2f1 128 \
29 sel src 14.0.0.52/24 dst 14.0.0.70/24 proto tcp \
30 offload dev eth4 dir in
31
32Yes, that's ugly, but that's what shell scripts and/or libreswan are for.
33
34
35
36Callbacks to implement
37======================
38
39::
40
41 /* from include/linux/netdevice.h */
42 struct xfrmdev_ops {
43 int (*xdo_dev_state_add) (struct xfrm_state *x);
44 void (*xdo_dev_state_delete) (struct xfrm_state *x);
45 void (*xdo_dev_state_free) (struct xfrm_state *x);
46 bool (*xdo_dev_offload_ok) (struct sk_buff *skb,
47 struct xfrm_state *x);
48 void (*xdo_dev_state_advance_esn) (struct xfrm_state *x);
49 };
50
51The NIC driver offering ipsec offload will need to implement these
52callbacks to make the offload available to the network stack's
53XFRM subsytem. Additionally, the feature bits NETIF_F_HW_ESP and
54NETIF_F_HW_ESP_TX_CSUM will signal the availability of the offload.
55
56
57
58Flow
59====
60
61At probe time and before the call to register_netdev(), the driver should
62set up local data structures and XFRM callbacks, and set the feature bits.
63The XFRM code's listener will finish the setup on NETDEV_REGISTER.
64
65::
66
67 adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
68 adapter->netdev->features |= NETIF_F_HW_ESP;
69 adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;
70
71When new SAs are set up with a request for "offload" feature, the
72driver's xdo_dev_state_add() will be given the new SA to be offloaded
73and an indication of whether it is for Rx or Tx. The driver should
74
75 - verify the algorithm is supported for offloads
76 - store the SA information (key, salt, target-ip, protocol, etc)
77 - enable the HW offload of the SA
78 - return status value:
79
80 =========== ===================================
81 0 success
82 -EOPNETSUPP offload not supported, try SW IPsec
83 other fail the request
84 =========== ===================================
85
86The driver can also set an offload_handle in the SA, an opaque void pointer
87that can be used to convey context into the fast-path offload requests::
88
89 xs->xso.offload_handle = context;
90
91
92When the network stack is preparing an IPsec packet for an SA that has
93been setup for offload, it first calls into xdo_dev_offload_ok() with
94the skb and the intended offload state to ask the driver if the offload
95will serviceable. This can check the packet information to be sure the
96offload can be supported (e.g. IPv4 or IPv6, no IPv4 options, etc) and
97return true of false to signify its support.
98
99When ready to send, the driver needs to inspect the Tx packet for the
100offload information, including the opaque context, and set up the packet
101send accordingly::
102
103 xs = xfrm_input_state(skb);
104 context = xs->xso.offload_handle;
105 set up HW for send
106
107The stack has already inserted the appropriate IPsec headers in the
108packet data, the offload just needs to do the encryption and fix up the
109header values.
110
111
112When a packet is received and the HW has indicated that it offloaded a
113decryption, the driver needs to add a reference to the decoded SA into
114the packet's skb. At this point the data should be decrypted but the
115IPsec headers are still in the packet data; they are removed later up
116the stack in xfrm_input().
117
118 find and hold the SA that was used to the Rx skb::
119
120 get spi, protocol, and destination IP from packet headers
121 xs = find xs from (spi, protocol, dest_IP)
122 xfrm_state_hold(xs);
123
124 store the state information into the skb::
125
126 sp = secpath_set(skb);
127 if (!sp) return;
128 sp->xvec[sp->len++] = xs;
129 sp->olen++;
130
131 indicate the success and/or error status of the offload::
132
133 xo = xfrm_offload(skb);
134 xo->flags = CRYPTO_DONE;
135 xo->status = crypto_status;
136
137 hand the packet to napi_gro_receive() as usual
138
139In ESN mode, xdo_dev_state_advance_esn() is called from xfrm_replay_advance_esn().
140Driver will check packet seq number and update HW ESN state machine if needed.
141
142When the SA is removed by the user, the driver's xdo_dev_state_delete()
143is asked to disable the offload. Later, xdo_dev_state_free() is called
144from a garbage collection routine after all reference counts to the state
145have been removed and any remaining resources can be cleared for the
146offload state. How these are used by the driver will depend on specific
147hardware needs.
148
149As a netdev is set to DOWN the XFRM stack's netdev listener will call
150xdo_dev_state_delete() and xdo_dev_state_free() on any remaining offloaded
151states.