1========
  2hwpoison
  3========
  4
  5What is hwpoison?
  6=================
  7
  8Upcoming Intel CPUs have support for recovering from some memory errors
  9(``MCA recovery``). This requires the OS to declare a page "poisoned",
 10kill the processes associated with it and avoid using it in the future.
 11
 12This patchkit implements the necessary infrastructure in the VM.
 13
 14To quote the overview comment::
 15
 16	High level machine check handler. Handles pages reported by the
 17	hardware as being corrupted usually due to a 2bit ECC memory or cache
 18	failure.
 19
 20	This focusses on pages detected as corrupted in the background.
 21	When the current CPU tries to consume corruption the currently
 22	running process can just be killed directly instead. This implies
 23	that if the error cannot be handled for some reason it's safe to
 24	just ignore it because no corruption has been consumed yet. Instead
 25	when that happens another machine check will happen.
 26
 27	Handles page cache pages in various states. The tricky part
 28	here is that we can access any page asynchronous to other VM
 29	users, because memory failures could happen anytime and anywhere,
 30	possibly violating some of their assumptions. This is why this code
 31	has to be extremely careful. Generally it tries to use normal locking
 32	rules, as in get the standard locks, even if that means the
 33	error handling takes potentially a long time.
 34
 35	Some of the operations here are somewhat inefficient and have non
 36	linear algorithmic complexity, because the data structures have not
 37	been optimized for this case. This is in particular the case
 38	for the mapping from a vma to a process. Since this case is expected
 39	to be rare we hope we can get away with this.
 40
 41The code consists of a the high level handler in mm/memory-failure.c,
 42a new page poison bit and various checks in the VM to handle poisoned
 43pages.
 44
 45The main target right now is KVM guests, but it works for all kinds
 46of applications. KVM support requires a recent qemu-kvm release.
 47
 48For the KVM use there was need for a new signal type so that
 49KVM can inject the machine check into the guest with the proper
 50address. This in theory allows other applications to handle
 51memory failures too. The expectation is that most applications
 52won't do that, but some very specialized ones might.
 53
 54Failure recovery modes
 55======================
 56
 57There are two (actually three) modes memory failure recovery can be in:
 58
 59vm.memory_failure_recovery sysctl set to zero:
 60	All memory failures cause a panic. Do not attempt recovery.
 61
 62early kill
 63	(can be controlled globally and per process)
 64	Send SIGBUS to the application as soon as the error is detected
 65	This allows applications who can process memory errors in a gentle
 66	way (e.g. drop affected object)
 67	This is the mode used by KVM qemu.
 68
 69late kill
 70	Send SIGBUS when the application runs into the corrupted page.
 71	This is best for memory error unaware applications and default
 72	Note some pages are always handled as late kill.
 73
 74User control
 75============
 76
 77vm.memory_failure_recovery
 78	See sysctl.txt
 79
 80vm.memory_failure_early_kill
 81	Enable early kill mode globally
 82
 83PR_MCE_KILL
 84	Set early/late kill mode/revert to system default
 85
 86	arg1: PR_MCE_KILL_CLEAR:
 87		Revert to system default
 88	arg1: PR_MCE_KILL_SET:
 89		arg2 defines thread specific mode
 90
 91		PR_MCE_KILL_EARLY:
 92			Early kill
 93		PR_MCE_KILL_LATE:
 94			Late kill
 95		PR_MCE_KILL_DEFAULT
 96			Use system global default
 97
 98	Note that if you want to have a dedicated thread which handles
 99	the SIGBUS(BUS_MCEERR_AO) on behalf of the process, you should
100	call prctl(PR_MCE_KILL_EARLY) on the designated thread. Otherwise,
101	the SIGBUS is sent to the main thread.
102
103PR_MCE_KILL_GET
104	return current mode
105
106Testing
107=======
108
109* madvise(MADV_HWPOISON, ....) (as root) - Poison a page in the
110  process for testing
111
112* hwpoison-inject module through debugfs ``/sys/kernel/debug/hwpoison/``
113
114  corrupt-pfn
115	Inject hwpoison fault at PFN echoed into this file. This does
116	some early filtering to avoid corrupted unintended pages in test suites.
117
118  unpoison-pfn
119	Software-unpoison page at PFN echoed into this file. This way
120	a page can be reused again.  This only works for Linux
121	injected failures, not for real memory failures. Once any hardware
122	memory failure happens, this feature is disabled.
123
124  Note these injection interfaces are not stable and might change between
125  kernel versions
126
127  corrupt-filter-dev-major, corrupt-filter-dev-minor
128	Only handle memory failures to pages associated with the file
129	system defined by block device major/minor.  -1U is the
130	wildcard value.  This should be only used for testing with
131	artificial injection.
132
133  corrupt-filter-memcg
134	Limit injection to pages owned by memgroup. Specified by inode
135	number of the memcg.
136
137	Example::
138
139		mkdir /sys/fs/cgroup/mem/hwpoison
140
141	        usemem -m 100 -s 1000 &
142		echo `jobs -p` > /sys/fs/cgroup/mem/hwpoison/tasks
143
144		memcg_ino=$(ls -id /sys/fs/cgroup/mem/hwpoison | cut -f1 -d' ')
145		echo $memcg_ino > /debug/hwpoison/corrupt-filter-memcg
146
147		page-types -p `pidof init`   --hwpoison  # shall do nothing
148		page-types -p `pidof usemem` --hwpoison  # poison its pages
149
150  corrupt-filter-flags-mask, corrupt-filter-flags-value
151	When specified, only poison pages if ((page_flags & mask) ==
152	value).  This allows stress testing of many kinds of
153	pages. The page_flags are the same as in /proc/kpageflags. The
154	flag bits are defined in include/linux/kernel-page-flags.h and
155	documented in Documentation/admin-guide/mm/pagemap.rst
156
157* Architecture specific MCE injector
158
159  x86 has mce-inject, mce-test
160
161  Some portable hwpoison test programs in mce-test, see below.
162
163References
164==========
165
166http://halobates.de/mce-lc09-2.pdf
167	Overview presentation from LinuxCon 09
168
169git://git.kernel.org/pub/scm/utils/cpu/mce/mce-test.git
170	Test suite (hwpoison specific portable tests in tsrc)
171
172git://git.kernel.org/pub/scm/utils/cpu/mce/mce-inject.git
173	x86 specific injector
174
175
176Limitations
177===========
178- Not all page types are supported and never will. Most kernel internal
179  objects cannot be recovered, only LRU pages for now.
180
181---
182Andi Kleen, Oct 2009