Loading...
1=================================
2HOWTO interact with BPF subsystem
3=================================
4
5This document provides information for the BPF subsystem about various
6workflows related to reporting bugs, submitting patches, and queueing
7patches for stable kernels.
8
9For general information about submitting patches, please refer to
10Documentation/process/submitting-patches.rst. This document only describes
11additional specifics related to BPF.
12
13.. contents::
14 :local:
15 :depth: 2
16
17Reporting bugs
18==============
19
20Q: How do I report bugs for BPF kernel code?
21--------------------------------------------
22A: Since all BPF kernel development as well as bpftool and iproute2 BPF
23loader development happens through the bpf kernel mailing list,
24please report any found issues around BPF to the following mailing
25list:
26
27 bpf@vger.kernel.org
28
29This may also include issues related to XDP, BPF tracing, etc.
30
31Given netdev has a high volume of traffic, please also add the BPF
32maintainers to Cc (from kernel ``MAINTAINERS`` file):
33
34* Alexei Starovoitov <ast@kernel.org>
35* Daniel Borkmann <daniel@iogearbox.net>
36
37In case a buggy commit has already been identified, make sure to keep
38the actual commit authors in Cc as well for the report. They can
39typically be identified through the kernel's git tree.
40
41**Please do NOT report BPF issues to bugzilla.kernel.org since it
42is a guarantee that the reported issue will be overlooked.**
43
44Submitting patches
45==================
46
47Q: How do I run BPF CI on my changes before sending them out for review?
48------------------------------------------------------------------------
49A: BPF CI is GitHub based and hosted at https://github.com/kernel-patches/bpf.
50While GitHub also provides a CLI that can be used to accomplish the same
51results, here we focus on the UI based workflow.
52
53The following steps lay out how to start a CI run for your patches:
54
55- Create a fork of the aforementioned repository in your own account (one time
56 action)
57
58- Clone the fork locally, check out a new branch tracking either the bpf-next
59 or bpf branch, and apply your to-be-tested patches on top of it
60
61- Push the local branch to your fork and create a pull request against
62 kernel-patches/bpf's bpf-next_base or bpf_base branch, respectively
63
64Shortly after the pull request has been created, the CI workflow will run. Note
65that capacity is shared with patches submitted upstream being checked and so
66depending on utilization the run can take a while to finish.
67
68Note furthermore that both base branches (bpf-next_base and bpf_base) will be
69updated as patches are pushed to the respective upstream branches they track. As
70such, your patch set will automatically (be attempted to) be rebased as well.
71This behavior can result in a CI run being aborted and restarted with the new
72base line.
73
74Q: To which mailing list do I need to submit my BPF patches?
75------------------------------------------------------------
76A: Please submit your BPF patches to the bpf kernel mailing list:
77
78 bpf@vger.kernel.org
79
80In case your patch has changes in various different subsystems (e.g.
81networking, tracing, security, etc), make sure to Cc the related kernel mailing
82lists and maintainers from there as well, so they are able to review
83the changes and provide their Acked-by's to the patches.
84
85Q: Where can I find patches currently under discussion for BPF subsystem?
86-------------------------------------------------------------------------
87A: All patches that are Cc'ed to netdev are queued for review under netdev
88patchwork project:
89
90 https://patchwork.kernel.org/project/netdevbpf/list/
91
92Those patches which target BPF, are assigned to a 'bpf' delegate for
93further processing from BPF maintainers. The current queue with
94patches under review can be found at:
95
96 https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173
97
98Once the patches have been reviewed by the BPF community as a whole
99and approved by the BPF maintainers, their status in patchwork will be
100changed to 'Accepted' and the submitter will be notified by mail. This
101means that the patches look good from a BPF perspective and have been
102applied to one of the two BPF kernel trees.
103
104In case feedback from the community requires a respin of the patches,
105their status in patchwork will be set to 'Changes Requested', and purged
106from the current review queue. Likewise for cases where patches would
107get rejected or are not applicable to the BPF trees (but assigned to
108the 'bpf' delegate).
109
110Q: How do the changes make their way into Linux?
111------------------------------------------------
112A: There are two BPF kernel trees (git repositories). Once patches have
113been accepted by the BPF maintainers, they will be applied to one
114of the two BPF trees:
115
116 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/
117 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/
118
119The bpf tree itself is for fixes only, whereas bpf-next for features,
120cleanups or other kind of improvements ("next-like" content). This is
121analogous to net and net-next trees for networking. Both bpf and
122bpf-next will only have a master branch in order to simplify against
123which branch patches should get rebased to.
124
125Accumulated BPF patches in the bpf tree will regularly get pulled
126into the net kernel tree. Likewise, accumulated BPF patches accepted
127into the bpf-next tree will make their way into net-next tree. net and
128net-next are both run by David S. Miller. From there, they will go
129into the kernel mainline tree run by Linus Torvalds. To read up on the
130process of net and net-next being merged into the mainline tree, see
131the documentation on netdev subsystem at
132Documentation/process/maintainer-netdev.rst.
133
134
135
136Occasionally, to prevent merge conflicts, we might send pull requests
137to other trees (e.g. tracing) with a small subset of the patches, but
138net and net-next are always the main trees targeted for integration.
139
140The pull requests will contain a high-level summary of the accumulated
141patches and can be searched on netdev kernel mailing list through the
142following subject lines (``yyyy-mm-dd`` is the date of the pull
143request)::
144
145 pull-request: bpf yyyy-mm-dd
146 pull-request: bpf-next yyyy-mm-dd
147
148Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be applied to?
149---------------------------------------------------------------------------------
150
151A: The process is the very same as described in the netdev subsystem
152documentation at Documentation/process/maintainer-netdev.rst,
153so please read up on it. The subject line must indicate whether the
154patch is a fix or rather "next-like" content in order to let the
155maintainers know whether it is targeted at bpf or bpf-next.
156
157For fixes eventually landing in bpf -> net tree, the subject must
158look like::
159
160 git format-patch --subject-prefix='PATCH bpf' start..finish
161
162For features/improvements/etc that should eventually land in
163bpf-next -> net-next, the subject must look like::
164
165 git format-patch --subject-prefix='PATCH bpf-next' start..finish
166
167If unsure whether the patch or patch series should go into bpf
168or net directly, or bpf-next or net-next directly, it is not a
169problem either if the subject line says net or net-next as target.
170It is eventually up to the maintainers to do the delegation of
171the patches.
172
173If it is clear that patches should go into bpf or bpf-next tree,
174please make sure to rebase the patches against those trees in
175order to reduce potential conflicts.
176
177In case the patch or patch series has to be reworked and sent out
178again in a second or later revision, it is also required to add a
179version number (``v2``, ``v3``, ...) into the subject prefix::
180
181 git format-patch --subject-prefix='PATCH bpf-next v2' start..finish
182
183When changes have been requested to the patch series, always send the
184whole patch series again with the feedback incorporated (never send
185individual diffs on top of the old series).
186
187Q: What does it mean when a patch gets applied to bpf or bpf-next tree?
188-----------------------------------------------------------------------
189A: It means that the patch looks good for mainline inclusion from
190a BPF point of view.
191
192Be aware that this is not a final verdict that the patch will
193automatically get accepted into net or net-next trees eventually:
194
195On the bpf kernel mailing list reviews can come in at any point
196in time. If discussions around a patch conclude that they cannot
197get included as-is, we will either apply a follow-up fix or drop
198them from the trees entirely. Therefore, we also reserve to rebase
199the trees when deemed necessary. After all, the purpose of the tree
200is to:
201
202i) accumulate and stage BPF patches for integration into trees
203 like net and net-next, and
204
205ii) run extensive BPF test suite and
206 workloads on the patches before they make their way any further.
207
208Once the BPF pull request was accepted by David S. Miller, then
209the patches end up in net or net-next tree, respectively, and
210make their way from there further into mainline. Again, see the
211documentation for netdev subsystem at
212Documentation/process/maintainer-netdev.rst for additional information
213e.g. on how often they are merged to mainline.
214
215Q: How long do I need to wait for feedback on my BPF patches?
216-------------------------------------------------------------
217A: We try to keep the latency low. The usual time to feedback will
218be around 2 or 3 business days. It may vary depending on the
219complexity of changes and current patch load.
220
221Q: How often do you send pull requests to major kernel trees like net or net-next?
222----------------------------------------------------------------------------------
223
224A: Pull requests will be sent out rather often in order to not
225accumulate too many patches in bpf or bpf-next.
226
227As a rule of thumb, expect pull requests for each tree regularly
228at the end of the week. In some cases pull requests could additionally
229come also in the middle of the week depending on the current patch
230load or urgency.
231
232Q: Are patches applied to bpf-next when the merge window is open?
233-----------------------------------------------------------------
234A: For the time when the merge window is open, bpf-next will not be
235processed. This is roughly analogous to net-next patch processing,
236so feel free to read up on the netdev docs at
237Documentation/process/maintainer-netdev.rst about further details.
238
239During those two weeks of merge window, we might ask you to resend
240your patch series once bpf-next is open again. Once Linus released
241a ``v*-rc1`` after the merge window, we continue processing of bpf-next.
242
243For non-subscribers to kernel mailing lists, there is also a status
244page run by David S. Miller on net-next that provides guidance:
245
246 http://vger.kernel.org/~davem/net-next.html
247
248Q: Verifier changes and test cases
249----------------------------------
250Q: I made a BPF verifier change, do I need to add test cases for
251BPF kernel selftests_?
252
253A: If the patch has changes to the behavior of the verifier, then yes,
254it is absolutely necessary to add test cases to the BPF kernel
255selftests_ suite. If they are not present and we think they are
256needed, then we might ask for them before accepting any changes.
257
258In particular, test_verifier.c is tracking a high number of BPF test
259cases, including a lot of corner cases that LLVM BPF back end may
260generate out of the restricted C code. Thus, adding test cases is
261absolutely crucial to make sure future changes do not accidentally
262affect prior use-cases. Thus, treat those test cases as: verifier
263behavior that is not tracked in test_verifier.c could potentially
264be subject to change.
265
266Q: samples/bpf preference vs selftests?
267---------------------------------------
268Q: When should I add code to ``samples/bpf/`` and when to BPF kernel
269selftests_?
270
271A: In general, we prefer additions to BPF kernel selftests_ rather than
272``samples/bpf/``. The rationale is very simple: kernel selftests are
273regularly run by various bots to test for kernel regressions.
274
275The more test cases we add to BPF selftests, the better the coverage
276and the less likely it is that those could accidentally break. It is
277not that BPF kernel selftests cannot demo how a specific feature can
278be used.
279
280That said, ``samples/bpf/`` may be a good place for people to get started,
281so it might be advisable that simple demos of features could go into
282``samples/bpf/``, but advanced functional and corner-case testing rather
283into kernel selftests.
284
285If your sample looks like a test case, then go for BPF kernel selftests
286instead!
287
288Q: When should I add code to the bpftool?
289-----------------------------------------
290A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide
291a central user space tool for debugging and introspection of BPF programs
292and maps that are active in the kernel. If UAPI changes related to BPF
293enable for dumping additional information of programs or maps, then
294bpftool should be extended as well to support dumping them.
295
296Q: When should I add code to iproute2's BPF loader?
297---------------------------------------------------
298A: For UAPI changes related to the XDP or tc layer (e.g. ``cls_bpf``),
299the convention is that those control-path related changes are added to
300iproute2's BPF loader as well from user space side. This is not only
301useful to have UAPI changes properly designed to be usable, but also
302to make those changes available to a wider user base of major
303downstream distributions.
304
305Q: Do you accept patches as well for iproute2's BPF loader?
306-----------------------------------------------------------
307A: Patches for the iproute2's BPF loader have to be sent to:
308
309 netdev@vger.kernel.org
310
311While those patches are not processed by the BPF kernel maintainers,
312please keep them in Cc as well, so they can be reviewed.
313
314The official git repository for iproute2 is run by Stephen Hemminger
315and can be found at:
316
317 https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/
318
319The patches need to have a subject prefix of '``[PATCH iproute2
320master]``' or '``[PATCH iproute2 net-next]``'. '``master``' or
321'``net-next``' describes the target branch where the patch should be
322applied to. Meaning, if kernel changes went into the net-next kernel
323tree, then the related iproute2 changes need to go into the iproute2
324net-next branch, otherwise they can be targeted at master branch. The
325iproute2 net-next branch will get merged into the master branch after
326the current iproute2 version from master has been released.
327
328Like BPF, the patches end up in patchwork under the netdev project and
329are delegated to 'shemminger' for further processing:
330
331 http://patchwork.ozlabs.org/project/netdev/list/?delegate=389
332
333Q: What is the minimum requirement before I submit my BPF patches?
334------------------------------------------------------------------
335A: When submitting patches, always take the time and properly test your
336patches *prior* to submission. Never rush them! If maintainers find
337that your patches have not been properly tested, it is a good way to
338get them grumpy. Testing patch submissions is a hard requirement!
339
340Note, fixes that go to bpf tree *must* have a ``Fixes:`` tag included.
341The same applies to fixes that target bpf-next, where the affected
342commit is in net-next (or in some cases bpf-next). The ``Fixes:`` tag is
343crucial in order to identify follow-up commits and tremendously helps
344for people having to do backporting, so it is a must have!
345
346We also don't accept patches with an empty commit message. Take your
347time and properly write up a high quality commit message, it is
348essential!
349
350Think about it this way: other developers looking at your code a month
351from now need to understand *why* a certain change has been done that
352way, and whether there have been flaws in the analysis or assumptions
353that the original author did. Thus providing a proper rationale and
354describing the use-case for the changes is a must.
355
356Patch submissions with >1 patch must have a cover letter which includes
357a high level description of the series. This high level summary will
358then be placed into the merge commit by the BPF maintainers such that
359it is also accessible from the git log for future reference.
360
361Q: Features changing BPF JIT and/or LLVM
362----------------------------------------
363Q: What do I need to consider when adding a new instruction or feature
364that would require BPF JIT and/or LLVM integration as well?
365
366A: We try hard to keep all BPF JITs up to date such that the same user
367experience can be guaranteed when running BPF programs on different
368architectures without having the program punt to the less efficient
369interpreter in case the in-kernel BPF JIT is enabled.
370
371If you are unable to implement or test the required JIT changes for
372certain architectures, please work together with the related BPF JIT
373developers in order to get the feature implemented in a timely manner.
374Please refer to the git log (``arch/*/net/``) to locate the necessary
375people for helping out.
376
377Also always make sure to add BPF test cases (e.g. test_bpf.c and
378test_verifier.c) for new instructions, so that they can receive
379broad test coverage and help run-time testing the various BPF JITs.
380
381In case of new BPF instructions, once the changes have been accepted
382into the Linux kernel, please implement support into LLVM's BPF back
383end. See LLVM_ section below for further information.
384
385Stable submission
386=================
387
388Q: I need a specific BPF commit in stable kernels. What should I do?
389--------------------------------------------------------------------
390A: In case you need a specific fix in stable kernels, first check whether
391the commit has already been applied in the related ``linux-*.y`` branches:
392
393 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/
394
395If not the case, then drop an email to the BPF maintainers with the
396netdev kernel mailing list in Cc and ask for the fix to be queued up:
397
398 netdev@vger.kernel.org
399
400The process in general is the same as on netdev itself, see also the
401the documentation on networking subsystem at
402Documentation/process/maintainer-netdev.rst.
403
404Q: Do you also backport to kernels not currently maintained as stable?
405----------------------------------------------------------------------
406A: No. If you need a specific BPF commit in kernels that are currently not
407maintained by the stable maintainers, then you are on your own.
408
409The current stable and longterm stable kernels are all listed here:
410
411 https://www.kernel.org/
412
413Q: The BPF patch I am about to submit needs to go to stable as well
414-------------------------------------------------------------------
415What should I do?
416
417A: The same rules apply as with netdev patch submissions in general, see
418the netdev docs at Documentation/process/maintainer-netdev.rst.
419
420Never add "``Cc: stable@vger.kernel.org``" to the patch description, but
421ask the BPF maintainers to queue the patches instead. This can be done
422with a note, for example, under the ``---`` part of the patch which does
423not go into the git log. Alternatively, this can be done as a simple
424request by mail instead.
425
426Q: Queue stable patches
427-----------------------
428Q: Where do I find currently queued BPF patches that will be submitted
429to stable?
430
431A: Once patches that fix critical bugs got applied into the bpf tree, they
432are queued up for stable submission under:
433
434 http://patchwork.ozlabs.org/bundle/bpf/stable/?state=*
435
436They will be on hold there at minimum until the related commit made its
437way into the mainline kernel tree.
438
439After having been under broader exposure, the queued patches will be
440submitted by the BPF maintainers to the stable maintainers.
441
442Testing patches
443===============
444
445Q: How to run BPF selftests
446---------------------------
447A: After you have booted into the newly compiled kernel, navigate to
448the BPF selftests_ suite in order to test BPF functionality (current
449working directory points to the root of the cloned git tree)::
450
451 $ cd tools/testing/selftests/bpf/
452 $ make
453
454To run the verifier tests::
455
456 $ sudo ./test_verifier
457
458The verifier tests print out all the current checks being
459performed. The summary at the end of running all tests will dump
460information of test successes and failures::
461
462 Summary: 418 PASSED, 0 FAILED
463
464In order to run through all BPF selftests, the following command is
465needed::
466
467 $ sudo make run_tests
468
469See :doc:`kernel selftest documentation </dev-tools/kselftest>`
470for details.
471
472To maximize the number of tests passing, the .config of the kernel
473under test should match the config file fragment in
474tools/testing/selftests/bpf as closely as possible.
475
476Finally to ensure support for latest BPF Type Format features -
477discussed in Documentation/bpf/btf.rst - pahole version 1.16
478is required for kernels built with CONFIG_DEBUG_INFO_BTF=y.
479pahole is delivered in the dwarves package or can be built
480from source at
481
482https://github.com/acmel/dwarves
483
484pahole starts to use libbpf definitions and APIs since v1.13 after the
485commit 21507cd3e97b ("pahole: add libbpf as submodule under lib/bpf").
486It works well with the git repository because the libbpf submodule will
487use "git submodule update --init --recursive" to update.
488
489Unfortunately, the default github release source code does not contain
490libbpf submodule source code and this will cause build issues, the tarball
491from https://git.kernel.org/pub/scm/devel/pahole/pahole.git/ is same with
492github, you can get the source tarball with corresponding libbpf submodule
493codes from
494
495https://fedorapeople.org/~acme/dwarves
496
497Some distros have pahole version 1.16 packaged already, e.g.
498Fedora, Gentoo.
499
500Q: Which BPF kernel selftests version should I run my kernel against?
501---------------------------------------------------------------------
502A: If you run a kernel ``xyz``, then always run the BPF kernel selftests
503from that kernel ``xyz`` as well. Do not expect that the BPF selftest
504from the latest mainline tree will pass all the time.
505
506In particular, test_bpf.c and test_verifier.c have a large number of
507test cases and are constantly updated with new BPF test sequences, or
508existing ones are adapted to verifier changes e.g. due to verifier
509becoming smarter and being able to better track certain things.
510
511LLVM
512====
513
514Q: Where do I find LLVM with BPF support?
515-----------------------------------------
516A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1.
517
518All major distributions these days ship LLVM with BPF back end enabled,
519so for the majority of use-cases it is not required to compile LLVM by
520hand anymore, just install the distribution provided package.
521
522LLVM's static compiler lists the supported targets through
523``llc --version``, make sure BPF targets are listed. Example::
524
525 $ llc --version
526 LLVM (http://llvm.org/):
527 LLVM version 10.0.0
528 Optimized build.
529 Default target: x86_64-unknown-linux-gnu
530 Host CPU: skylake
531
532 Registered Targets:
533 aarch64 - AArch64 (little endian)
534 bpf - BPF (host endian)
535 bpfeb - BPF (big endian)
536 bpfel - BPF (little endian)
537 x86 - 32-bit X86: Pentium-Pro and above
538 x86-64 - 64-bit X86: EM64T and AMD64
539
540For developers in order to utilize the latest features added to LLVM's
541BPF back end, it is advisable to run the latest LLVM releases. Support
542for new BPF kernel features such as additions to the BPF instruction
543set are often developed together.
544
545All LLVM releases can be found at: http://releases.llvm.org/
546
547Q: Got it, so how do I build LLVM manually anyway?
548--------------------------------------------------
549A: We recommend that developers who want the fastest incremental builds
550use the Ninja build system, you can find it in your system's package
551manager, usually the package is ninja or ninja-build.
552
553You need ninja, cmake and gcc-c++ as build requisites for LLVM. Once you
554have that set up, proceed with building the latest LLVM and clang version
555from the git repositories::
556
557 $ git clone https://github.com/llvm/llvm-project.git
558 $ mkdir -p llvm-project/llvm/build
559 $ cd llvm-project/llvm/build
560 $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
561 -DLLVM_ENABLE_PROJECTS="clang" \
562 -DCMAKE_BUILD_TYPE=Release \
563 -DLLVM_BUILD_RUNTIME=OFF
564 $ ninja
565
566The built binaries can then be found in the build/bin/ directory, where
567you can point the PATH variable to.
568
569Set ``-DLLVM_TARGETS_TO_BUILD`` equal to the target you wish to build, you
570will find a full list of targets within the llvm-project/llvm/lib/Target
571directory.
572
573Q: Reporting LLVM BPF issues
574----------------------------
575Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code
576generation back end or about LLVM generated code that the verifier
577refuses to accept?
578
579A: Yes, please do!
580
581LLVM's BPF back end is a key piece of the whole BPF
582infrastructure and it ties deeply into verification of programs from the
583kernel side. Therefore, any issues on either side need to be investigated
584and fixed whenever necessary.
585
586Therefore, please make sure to bring them up at netdev kernel mailing
587list and Cc BPF maintainers for LLVM and kernel bits:
588
589* Yonghong Song <yhs@fb.com>
590* Alexei Starovoitov <ast@kernel.org>
591* Daniel Borkmann <daniel@iogearbox.net>
592
593LLVM also has an issue tracker where BPF related bugs can be found:
594
595 https://bugs.llvm.org/buglist.cgi?quicksearch=bpf
596
597However, it is better to reach out through mailing lists with having
598maintainers in Cc.
599
600Q: New BPF instruction for kernel and LLVM
601------------------------------------------
602Q: I have added a new BPF instruction to the kernel, how can I integrate
603it into LLVM?
604
605A: LLVM has a ``-mcpu`` selector for the BPF back end in order to allow
606the selection of BPF instruction set extensions. By default the
607``generic`` processor target is used, which is the base instruction set
608(v1) of BPF.
609
610LLVM has an option to select ``-mcpu=probe`` where it will probe the host
611kernel for supported BPF instruction set extensions and selects the
612optimal set automatically.
613
614For cross-compilation, a specific version can be select manually as well ::
615
616 $ llc -march bpf -mcpu=help
617 Available CPUs for this target:
618
619 generic - Select the generic processor.
620 probe - Select the probe processor.
621 v1 - Select the v1 processor.
622 v2 - Select the v2 processor.
623 [...]
624
625Newly added BPF instructions to the Linux kernel need to follow the same
626scheme, bump the instruction set version and implement probing for the
627extensions such that ``-mcpu=probe`` users can benefit from the
628optimization transparently when upgrading their kernels.
629
630If you are unable to implement support for the newly added BPF instruction
631please reach out to BPF developers for help.
632
633By the way, the BPF kernel selftests run with ``-mcpu=probe`` for better
634test coverage.
635
636Q: clang flag for target bpf?
637-----------------------------
638Q: In some cases clang flag ``--target=bpf`` is used but in other cases the
639default clang target, which matches the underlying architecture, is used.
640What is the difference and when I should use which?
641
642A: Although LLVM IR generation and optimization try to stay architecture
643independent, ``--target=<arch>`` still has some impact on generated code:
644
645- BPF program may recursively include header file(s) with file scope
646 inline assembly codes. The default target can handle this well,
647 while ``bpf`` target may fail if bpf backend assembler does not
648 understand these assembly codes, which is true in most cases.
649
650- When compiled without ``-g``, additional elf sections, e.g.,
651 .eh_frame and .rela.eh_frame, may be present in the object file
652 with default target, but not with ``bpf`` target.
653
654- The default target may turn a C switch statement into a switch table
655 lookup and jump operation. Since the switch table is placed
656 in the global readonly section, the bpf program will fail to load.
657 The bpf target does not support switch table optimization.
658 The clang option ``-fno-jump-tables`` can be used to disable
659 switch table generation.
660
661- For clang ``--target=bpf``, it is guaranteed that pointer or long /
662 unsigned long types will always have a width of 64 bit, no matter
663 whether underlying clang binary or default target (or kernel) is
664 32 bit. However, when native clang target is used, then it will
665 compile these types based on the underlying architecture's conventions,
666 meaning in case of 32 bit architecture, pointer or long / unsigned
667 long types e.g. in BPF context structure will have width of 32 bit
668 while the BPF LLVM back end still operates in 64 bit. The native
669 target is mostly needed in tracing for the case of walking ``pt_regs``
670 or other kernel structures where CPU's register width matters.
671 Otherwise, ``clang --target=bpf`` is generally recommended.
672
673You should use default target when:
674
675- Your program includes a header file, e.g., ptrace.h, which eventually
676 pulls in some header files containing file scope host assembly codes.
677
678- You can add ``-fno-jump-tables`` to work around the switch table issue.
679
680Otherwise, you can use ``bpf`` target. Additionally, you *must* use bpf target
681when:
682
683- Your program uses data structures with pointer or long / unsigned long
684 types that interface with BPF helpers or context data structures. Access
685 into these structures is verified by the BPF verifier and may result
686 in verification failures if the native architecture is not aligned with
687 the BPF architecture, e.g. 64-bit. An example of this is
688 BPF_PROG_TYPE_SK_MSG require ``--target=bpf``
689
690
691.. Links
692.. _selftests:
693 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/
694
695Happy BPF hacking!
1=================================
2HOWTO interact with BPF subsystem
3=================================
4
5This document provides information for the BPF subsystem about various
6workflows related to reporting bugs, submitting patches, and queueing
7patches for stable kernels.
8
9For general information about submitting patches, please refer to
10`Documentation/process/`_. This document only describes additional specifics
11related to BPF.
12
13.. contents::
14 :local:
15 :depth: 2
16
17Reporting bugs
18==============
19
20Q: How do I report bugs for BPF kernel code?
21--------------------------------------------
22A: Since all BPF kernel development as well as bpftool and iproute2 BPF
23loader development happens through the bpf kernel mailing list,
24please report any found issues around BPF to the following mailing
25list:
26
27 bpf@vger.kernel.org
28
29This may also include issues related to XDP, BPF tracing, etc.
30
31Given netdev has a high volume of traffic, please also add the BPF
32maintainers to Cc (from kernel ``MAINTAINERS`` file):
33
34* Alexei Starovoitov <ast@kernel.org>
35* Daniel Borkmann <daniel@iogearbox.net>
36
37In case a buggy commit has already been identified, make sure to keep
38the actual commit authors in Cc as well for the report. They can
39typically be identified through the kernel's git tree.
40
41**Please do NOT report BPF issues to bugzilla.kernel.org since it
42is a guarantee that the reported issue will be overlooked.**
43
44Submitting patches
45==================
46
47Q: To which mailing list do I need to submit my BPF patches?
48------------------------------------------------------------
49A: Please submit your BPF patches to the bpf kernel mailing list:
50
51 bpf@vger.kernel.org
52
53In case your patch has changes in various different subsystems (e.g.
54networking, tracing, security, etc), make sure to Cc the related kernel mailing
55lists and maintainers from there as well, so they are able to review
56the changes and provide their Acked-by's to the patches.
57
58Q: Where can I find patches currently under discussion for BPF subsystem?
59-------------------------------------------------------------------------
60A: All patches that are Cc'ed to netdev are queued for review under netdev
61patchwork project:
62
63 https://patchwork.kernel.org/project/netdevbpf/list/
64
65Those patches which target BPF, are assigned to a 'bpf' delegate for
66further processing from BPF maintainers. The current queue with
67patches under review can be found at:
68
69 https://patchwork.kernel.org/project/netdevbpf/list/?delegate=121173
70
71Once the patches have been reviewed by the BPF community as a whole
72and approved by the BPF maintainers, their status in patchwork will be
73changed to 'Accepted' and the submitter will be notified by mail. This
74means that the patches look good from a BPF perspective and have been
75applied to one of the two BPF kernel trees.
76
77In case feedback from the community requires a respin of the patches,
78their status in patchwork will be set to 'Changes Requested', and purged
79from the current review queue. Likewise for cases where patches would
80get rejected or are not applicable to the BPF trees (but assigned to
81the 'bpf' delegate).
82
83Q: How do the changes make their way into Linux?
84------------------------------------------------
85A: There are two BPF kernel trees (git repositories). Once patches have
86been accepted by the BPF maintainers, they will be applied to one
87of the two BPF trees:
88
89 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/
90 * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/
91
92The bpf tree itself is for fixes only, whereas bpf-next for features,
93cleanups or other kind of improvements ("next-like" content). This is
94analogous to net and net-next trees for networking. Both bpf and
95bpf-next will only have a master branch in order to simplify against
96which branch patches should get rebased to.
97
98Accumulated BPF patches in the bpf tree will regularly get pulled
99into the net kernel tree. Likewise, accumulated BPF patches accepted
100into the bpf-next tree will make their way into net-next tree. net and
101net-next are both run by David S. Miller. From there, they will go
102into the kernel mainline tree run by Linus Torvalds. To read up on the
103process of net and net-next being merged into the mainline tree, see
104the :ref:`netdev-FAQ`
105
106
107
108Occasionally, to prevent merge conflicts, we might send pull requests
109to other trees (e.g. tracing) with a small subset of the patches, but
110net and net-next are always the main trees targeted for integration.
111
112The pull requests will contain a high-level summary of the accumulated
113patches and can be searched on netdev kernel mailing list through the
114following subject lines (``yyyy-mm-dd`` is the date of the pull
115request)::
116
117 pull-request: bpf yyyy-mm-dd
118 pull-request: bpf-next yyyy-mm-dd
119
120Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be applied to?
121---------------------------------------------------------------------------------
122
123A: The process is the very same as described in the :ref:`netdev-FAQ`,
124so please read up on it. The subject line must indicate whether the
125patch is a fix or rather "next-like" content in order to let the
126maintainers know whether it is targeted at bpf or bpf-next.
127
128For fixes eventually landing in bpf -> net tree, the subject must
129look like::
130
131 git format-patch --subject-prefix='PATCH bpf' start..finish
132
133For features/improvements/etc that should eventually land in
134bpf-next -> net-next, the subject must look like::
135
136 git format-patch --subject-prefix='PATCH bpf-next' start..finish
137
138If unsure whether the patch or patch series should go into bpf
139or net directly, or bpf-next or net-next directly, it is not a
140problem either if the subject line says net or net-next as target.
141It is eventually up to the maintainers to do the delegation of
142the patches.
143
144If it is clear that patches should go into bpf or bpf-next tree,
145please make sure to rebase the patches against those trees in
146order to reduce potential conflicts.
147
148In case the patch or patch series has to be reworked and sent out
149again in a second or later revision, it is also required to add a
150version number (``v2``, ``v3``, ...) into the subject prefix::
151
152 git format-patch --subject-prefix='PATCH bpf-next v2' start..finish
153
154When changes have been requested to the patch series, always send the
155whole patch series again with the feedback incorporated (never send
156individual diffs on top of the old series).
157
158Q: What does it mean when a patch gets applied to bpf or bpf-next tree?
159-----------------------------------------------------------------------
160A: It means that the patch looks good for mainline inclusion from
161a BPF point of view.
162
163Be aware that this is not a final verdict that the patch will
164automatically get accepted into net or net-next trees eventually:
165
166On the bpf kernel mailing list reviews can come in at any point
167in time. If discussions around a patch conclude that they cannot
168get included as-is, we will either apply a follow-up fix or drop
169them from the trees entirely. Therefore, we also reserve to rebase
170the trees when deemed necessary. After all, the purpose of the tree
171is to:
172
173i) accumulate and stage BPF patches for integration into trees
174 like net and net-next, and
175
176ii) run extensive BPF test suite and
177 workloads on the patches before they make their way any further.
178
179Once the BPF pull request was accepted by David S. Miller, then
180the patches end up in net or net-next tree, respectively, and
181make their way from there further into mainline. Again, see the
182:ref:`netdev-FAQ` for additional information e.g. on how often they are
183merged to mainline.
184
185Q: How long do I need to wait for feedback on my BPF patches?
186-------------------------------------------------------------
187A: We try to keep the latency low. The usual time to feedback will
188be around 2 or 3 business days. It may vary depending on the
189complexity of changes and current patch load.
190
191Q: How often do you send pull requests to major kernel trees like net or net-next?
192----------------------------------------------------------------------------------
193
194A: Pull requests will be sent out rather often in order to not
195accumulate too many patches in bpf or bpf-next.
196
197As a rule of thumb, expect pull requests for each tree regularly
198at the end of the week. In some cases pull requests could additionally
199come also in the middle of the week depending on the current patch
200load or urgency.
201
202Q: Are patches applied to bpf-next when the merge window is open?
203-----------------------------------------------------------------
204A: For the time when the merge window is open, bpf-next will not be
205processed. This is roughly analogous to net-next patch processing,
206so feel free to read up on the :ref:`netdev-FAQ` about further details.
207
208During those two weeks of merge window, we might ask you to resend
209your patch series once bpf-next is open again. Once Linus released
210a ``v*-rc1`` after the merge window, we continue processing of bpf-next.
211
212For non-subscribers to kernel mailing lists, there is also a status
213page run by David S. Miller on net-next that provides guidance:
214
215 http://vger.kernel.org/~davem/net-next.html
216
217Q: Verifier changes and test cases
218----------------------------------
219Q: I made a BPF verifier change, do I need to add test cases for
220BPF kernel selftests_?
221
222A: If the patch has changes to the behavior of the verifier, then yes,
223it is absolutely necessary to add test cases to the BPF kernel
224selftests_ suite. If they are not present and we think they are
225needed, then we might ask for them before accepting any changes.
226
227In particular, test_verifier.c is tracking a high number of BPF test
228cases, including a lot of corner cases that LLVM BPF back end may
229generate out of the restricted C code. Thus, adding test cases is
230absolutely crucial to make sure future changes do not accidentally
231affect prior use-cases. Thus, treat those test cases as: verifier
232behavior that is not tracked in test_verifier.c could potentially
233be subject to change.
234
235Q: samples/bpf preference vs selftests?
236---------------------------------------
237Q: When should I add code to ``samples/bpf/`` and when to BPF kernel
238selftests_?
239
240A: In general, we prefer additions to BPF kernel selftests_ rather than
241``samples/bpf/``. The rationale is very simple: kernel selftests are
242regularly run by various bots to test for kernel regressions.
243
244The more test cases we add to BPF selftests, the better the coverage
245and the less likely it is that those could accidentally break. It is
246not that BPF kernel selftests cannot demo how a specific feature can
247be used.
248
249That said, ``samples/bpf/`` may be a good place for people to get started,
250so it might be advisable that simple demos of features could go into
251``samples/bpf/``, but advanced functional and corner-case testing rather
252into kernel selftests.
253
254If your sample looks like a test case, then go for BPF kernel selftests
255instead!
256
257Q: When should I add code to the bpftool?
258-----------------------------------------
259A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide
260a central user space tool for debugging and introspection of BPF programs
261and maps that are active in the kernel. If UAPI changes related to BPF
262enable for dumping additional information of programs or maps, then
263bpftool should be extended as well to support dumping them.
264
265Q: When should I add code to iproute2's BPF loader?
266---------------------------------------------------
267A: For UAPI changes related to the XDP or tc layer (e.g. ``cls_bpf``),
268the convention is that those control-path related changes are added to
269iproute2's BPF loader as well from user space side. This is not only
270useful to have UAPI changes properly designed to be usable, but also
271to make those changes available to a wider user base of major
272downstream distributions.
273
274Q: Do you accept patches as well for iproute2's BPF loader?
275-----------------------------------------------------------
276A: Patches for the iproute2's BPF loader have to be sent to:
277
278 netdev@vger.kernel.org
279
280While those patches are not processed by the BPF kernel maintainers,
281please keep them in Cc as well, so they can be reviewed.
282
283The official git repository for iproute2 is run by Stephen Hemminger
284and can be found at:
285
286 https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/
287
288The patches need to have a subject prefix of '``[PATCH iproute2
289master]``' or '``[PATCH iproute2 net-next]``'. '``master``' or
290'``net-next``' describes the target branch where the patch should be
291applied to. Meaning, if kernel changes went into the net-next kernel
292tree, then the related iproute2 changes need to go into the iproute2
293net-next branch, otherwise they can be targeted at master branch. The
294iproute2 net-next branch will get merged into the master branch after
295the current iproute2 version from master has been released.
296
297Like BPF, the patches end up in patchwork under the netdev project and
298are delegated to 'shemminger' for further processing:
299
300 http://patchwork.ozlabs.org/project/netdev/list/?delegate=389
301
302Q: What is the minimum requirement before I submit my BPF patches?
303------------------------------------------------------------------
304A: When submitting patches, always take the time and properly test your
305patches *prior* to submission. Never rush them! If maintainers find
306that your patches have not been properly tested, it is a good way to
307get them grumpy. Testing patch submissions is a hard requirement!
308
309Note, fixes that go to bpf tree *must* have a ``Fixes:`` tag included.
310The same applies to fixes that target bpf-next, where the affected
311commit is in net-next (or in some cases bpf-next). The ``Fixes:`` tag is
312crucial in order to identify follow-up commits and tremendously helps
313for people having to do backporting, so it is a must have!
314
315We also don't accept patches with an empty commit message. Take your
316time and properly write up a high quality commit message, it is
317essential!
318
319Think about it this way: other developers looking at your code a month
320from now need to understand *why* a certain change has been done that
321way, and whether there have been flaws in the analysis or assumptions
322that the original author did. Thus providing a proper rationale and
323describing the use-case for the changes is a must.
324
325Patch submissions with >1 patch must have a cover letter which includes
326a high level description of the series. This high level summary will
327then be placed into the merge commit by the BPF maintainers such that
328it is also accessible from the git log for future reference.
329
330Q: Features changing BPF JIT and/or LLVM
331----------------------------------------
332Q: What do I need to consider when adding a new instruction or feature
333that would require BPF JIT and/or LLVM integration as well?
334
335A: We try hard to keep all BPF JITs up to date such that the same user
336experience can be guaranteed when running BPF programs on different
337architectures without having the program punt to the less efficient
338interpreter in case the in-kernel BPF JIT is enabled.
339
340If you are unable to implement or test the required JIT changes for
341certain architectures, please work together with the related BPF JIT
342developers in order to get the feature implemented in a timely manner.
343Please refer to the git log (``arch/*/net/``) to locate the necessary
344people for helping out.
345
346Also always make sure to add BPF test cases (e.g. test_bpf.c and
347test_verifier.c) for new instructions, so that they can receive
348broad test coverage and help run-time testing the various BPF JITs.
349
350In case of new BPF instructions, once the changes have been accepted
351into the Linux kernel, please implement support into LLVM's BPF back
352end. See LLVM_ section below for further information.
353
354Stable submission
355=================
356
357Q: I need a specific BPF commit in stable kernels. What should I do?
358--------------------------------------------------------------------
359A: In case you need a specific fix in stable kernels, first check whether
360the commit has already been applied in the related ``linux-*.y`` branches:
361
362 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/
363
364If not the case, then drop an email to the BPF maintainers with the
365netdev kernel mailing list in Cc and ask for the fix to be queued up:
366
367 netdev@vger.kernel.org
368
369The process in general is the same as on netdev itself, see also the
370:ref:`netdev-FAQ`.
371
372Q: Do you also backport to kernels not currently maintained as stable?
373----------------------------------------------------------------------
374A: No. If you need a specific BPF commit in kernels that are currently not
375maintained by the stable maintainers, then you are on your own.
376
377The current stable and longterm stable kernels are all listed here:
378
379 https://www.kernel.org/
380
381Q: The BPF patch I am about to submit needs to go to stable as well
382-------------------------------------------------------------------
383What should I do?
384
385A: The same rules apply as with netdev patch submissions in general, see
386the :ref:`netdev-FAQ`.
387
388Never add "``Cc: stable@vger.kernel.org``" to the patch description, but
389ask the BPF maintainers to queue the patches instead. This can be done
390with a note, for example, under the ``---`` part of the patch which does
391not go into the git log. Alternatively, this can be done as a simple
392request by mail instead.
393
394Q: Queue stable patches
395-----------------------
396Q: Where do I find currently queued BPF patches that will be submitted
397to stable?
398
399A: Once patches that fix critical bugs got applied into the bpf tree, they
400are queued up for stable submission under:
401
402 http://patchwork.ozlabs.org/bundle/bpf/stable/?state=*
403
404They will be on hold there at minimum until the related commit made its
405way into the mainline kernel tree.
406
407After having been under broader exposure, the queued patches will be
408submitted by the BPF maintainers to the stable maintainers.
409
410Testing patches
411===============
412
413Q: How to run BPF selftests
414---------------------------
415A: After you have booted into the newly compiled kernel, navigate to
416the BPF selftests_ suite in order to test BPF functionality (current
417working directory points to the root of the cloned git tree)::
418
419 $ cd tools/testing/selftests/bpf/
420 $ make
421
422To run the verifier tests::
423
424 $ sudo ./test_verifier
425
426The verifier tests print out all the current checks being
427performed. The summary at the end of running all tests will dump
428information of test successes and failures::
429
430 Summary: 418 PASSED, 0 FAILED
431
432In order to run through all BPF selftests, the following command is
433needed::
434
435 $ sudo make run_tests
436
437See the kernels selftest `Documentation/dev-tools/kselftest.rst`_
438document for further documentation.
439
440To maximize the number of tests passing, the .config of the kernel
441under test should match the config file fragment in
442tools/testing/selftests/bpf as closely as possible.
443
444Finally to ensure support for latest BPF Type Format features -
445discussed in `Documentation/bpf/btf.rst`_ - pahole version 1.16
446is required for kernels built with CONFIG_DEBUG_INFO_BTF=y.
447pahole is delivered in the dwarves package or can be built
448from source at
449
450https://github.com/acmel/dwarves
451
452pahole starts to use libbpf definitions and APIs since v1.13 after the
453commit 21507cd3e97b ("pahole: add libbpf as submodule under lib/bpf").
454It works well with the git repository because the libbpf submodule will
455use "git submodule update --init --recursive" to update.
456
457Unfortunately, the default github release source code does not contain
458libbpf submodule source code and this will cause build issues, the tarball
459from https://git.kernel.org/pub/scm/devel/pahole/pahole.git/ is same with
460github, you can get the source tarball with corresponding libbpf submodule
461codes from
462
463https://fedorapeople.org/~acme/dwarves
464
465Some distros have pahole version 1.16 packaged already, e.g.
466Fedora, Gentoo.
467
468Q: Which BPF kernel selftests version should I run my kernel against?
469---------------------------------------------------------------------
470A: If you run a kernel ``xyz``, then always run the BPF kernel selftests
471from that kernel ``xyz`` as well. Do not expect that the BPF selftest
472from the latest mainline tree will pass all the time.
473
474In particular, test_bpf.c and test_verifier.c have a large number of
475test cases and are constantly updated with new BPF test sequences, or
476existing ones are adapted to verifier changes e.g. due to verifier
477becoming smarter and being able to better track certain things.
478
479LLVM
480====
481
482Q: Where do I find LLVM with BPF support?
483-----------------------------------------
484A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1.
485
486All major distributions these days ship LLVM with BPF back end enabled,
487so for the majority of use-cases it is not required to compile LLVM by
488hand anymore, just install the distribution provided package.
489
490LLVM's static compiler lists the supported targets through
491``llc --version``, make sure BPF targets are listed. Example::
492
493 $ llc --version
494 LLVM (http://llvm.org/):
495 LLVM version 10.0.0
496 Optimized build.
497 Default target: x86_64-unknown-linux-gnu
498 Host CPU: skylake
499
500 Registered Targets:
501 aarch64 - AArch64 (little endian)
502 bpf - BPF (host endian)
503 bpfeb - BPF (big endian)
504 bpfel - BPF (little endian)
505 x86 - 32-bit X86: Pentium-Pro and above
506 x86-64 - 64-bit X86: EM64T and AMD64
507
508For developers in order to utilize the latest features added to LLVM's
509BPF back end, it is advisable to run the latest LLVM releases. Support
510for new BPF kernel features such as additions to the BPF instruction
511set are often developed together.
512
513All LLVM releases can be found at: http://releases.llvm.org/
514
515Q: Got it, so how do I build LLVM manually anyway?
516--------------------------------------------------
517A: We recommend that developers who want the fastest incremental builds
518use the Ninja build system, you can find it in your system's package
519manager, usually the package is ninja or ninja-build.
520
521You need ninja, cmake and gcc-c++ as build requisites for LLVM. Once you
522have that set up, proceed with building the latest LLVM and clang version
523from the git repositories::
524
525 $ git clone https://github.com/llvm/llvm-project.git
526 $ mkdir -p llvm-project/llvm/build
527 $ cd llvm-project/llvm/build
528 $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
529 -DLLVM_ENABLE_PROJECTS="clang" \
530 -DCMAKE_BUILD_TYPE=Release \
531 -DLLVM_BUILD_RUNTIME=OFF
532 $ ninja
533
534The built binaries can then be found in the build/bin/ directory, where
535you can point the PATH variable to.
536
537Set ``-DLLVM_TARGETS_TO_BUILD`` equal to the target you wish to build, you
538will find a full list of targets within the llvm-project/llvm/lib/Target
539directory.
540
541Q: Reporting LLVM BPF issues
542----------------------------
543Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code
544generation back end or about LLVM generated code that the verifier
545refuses to accept?
546
547A: Yes, please do!
548
549LLVM's BPF back end is a key piece of the whole BPF
550infrastructure and it ties deeply into verification of programs from the
551kernel side. Therefore, any issues on either side need to be investigated
552and fixed whenever necessary.
553
554Therefore, please make sure to bring them up at netdev kernel mailing
555list and Cc BPF maintainers for LLVM and kernel bits:
556
557* Yonghong Song <yhs@fb.com>
558* Alexei Starovoitov <ast@kernel.org>
559* Daniel Borkmann <daniel@iogearbox.net>
560
561LLVM also has an issue tracker where BPF related bugs can be found:
562
563 https://bugs.llvm.org/buglist.cgi?quicksearch=bpf
564
565However, it is better to reach out through mailing lists with having
566maintainers in Cc.
567
568Q: New BPF instruction for kernel and LLVM
569------------------------------------------
570Q: I have added a new BPF instruction to the kernel, how can I integrate
571it into LLVM?
572
573A: LLVM has a ``-mcpu`` selector for the BPF back end in order to allow
574the selection of BPF instruction set extensions. By default the
575``generic`` processor target is used, which is the base instruction set
576(v1) of BPF.
577
578LLVM has an option to select ``-mcpu=probe`` where it will probe the host
579kernel for supported BPF instruction set extensions and selects the
580optimal set automatically.
581
582For cross-compilation, a specific version can be select manually as well ::
583
584 $ llc -march bpf -mcpu=help
585 Available CPUs for this target:
586
587 generic - Select the generic processor.
588 probe - Select the probe processor.
589 v1 - Select the v1 processor.
590 v2 - Select the v2 processor.
591 [...]
592
593Newly added BPF instructions to the Linux kernel need to follow the same
594scheme, bump the instruction set version and implement probing for the
595extensions such that ``-mcpu=probe`` users can benefit from the
596optimization transparently when upgrading their kernels.
597
598If you are unable to implement support for the newly added BPF instruction
599please reach out to BPF developers for help.
600
601By the way, the BPF kernel selftests run with ``-mcpu=probe`` for better
602test coverage.
603
604Q: clang flag for target bpf?
605-----------------------------
606Q: In some cases clang flag ``-target bpf`` is used but in other cases the
607default clang target, which matches the underlying architecture, is used.
608What is the difference and when I should use which?
609
610A: Although LLVM IR generation and optimization try to stay architecture
611independent, ``-target <arch>`` still has some impact on generated code:
612
613- BPF program may recursively include header file(s) with file scope
614 inline assembly codes. The default target can handle this well,
615 while ``bpf`` target may fail if bpf backend assembler does not
616 understand these assembly codes, which is true in most cases.
617
618- When compiled without ``-g``, additional elf sections, e.g.,
619 .eh_frame and .rela.eh_frame, may be present in the object file
620 with default target, but not with ``bpf`` target.
621
622- The default target may turn a C switch statement into a switch table
623 lookup and jump operation. Since the switch table is placed
624 in the global readonly section, the bpf program will fail to load.
625 The bpf target does not support switch table optimization.
626 The clang option ``-fno-jump-tables`` can be used to disable
627 switch table generation.
628
629- For clang ``-target bpf``, it is guaranteed that pointer or long /
630 unsigned long types will always have a width of 64 bit, no matter
631 whether underlying clang binary or default target (or kernel) is
632 32 bit. However, when native clang target is used, then it will
633 compile these types based on the underlying architecture's conventions,
634 meaning in case of 32 bit architecture, pointer or long / unsigned
635 long types e.g. in BPF context structure will have width of 32 bit
636 while the BPF LLVM back end still operates in 64 bit. The native
637 target is mostly needed in tracing for the case of walking ``pt_regs``
638 or other kernel structures where CPU's register width matters.
639 Otherwise, ``clang -target bpf`` is generally recommended.
640
641You should use default target when:
642
643- Your program includes a header file, e.g., ptrace.h, which eventually
644 pulls in some header files containing file scope host assembly codes.
645
646- You can add ``-fno-jump-tables`` to work around the switch table issue.
647
648Otherwise, you can use ``bpf`` target. Additionally, you *must* use bpf target
649when:
650
651- Your program uses data structures with pointer or long / unsigned long
652 types that interface with BPF helpers or context data structures. Access
653 into these structures is verified by the BPF verifier and may result
654 in verification failures if the native architecture is not aligned with
655 the BPF architecture, e.g. 64-bit. An example of this is
656 BPF_PROG_TYPE_SK_MSG require ``-target bpf``
657
658
659.. Links
660.. _Documentation/process/: https://www.kernel.org/doc/html/latest/process/
661.. _netdev-FAQ: ../networking/netdev-FAQ.rst
662.. _selftests:
663 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/bpf/
664.. _Documentation/dev-tools/kselftest.rst:
665 https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html
666.. _Documentation/bpf/btf.rst: btf.rst
667
668Happy BPF hacking!