1What:		/sys/class/rtrs-client
  2Date:		Feb 2020
  3KernelVersion:	5.7
  4Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
  5Description:	When a user of RTRS API creates a new session, a directory entry with
  6		the name of that session is created under /sys/class/rtrs-client/<session-name>/
  7
  8What:		/sys/class/rtrs-client/<session-name>/add_path
  9Date:		Feb 2020
 10KernelVersion:	5.7
 11Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 12Description:	RW, adds a new path (connection) to an existing session. Expected format is the
 13		following::
 14
 15		    <[source addr,]destination addr>
 16		    *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]
 17
 18What:		/sys/class/rtrs-client/<session-name>/max_reconnect_attempts
 19Date:		Feb 2020
 20KernelVersion:	5.7
 21Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 22Description:	Maximum number reconnect attempts the client should make before giving up
 23		after connection breaks unexpectedly.
 24
 25What:		/sys/class/rtrs-client/<session-name>/mp_policy
 26Date:		Feb 2020
 27KernelVersion:	5.7
 28Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 29Description:	Multipath policy specifies which path should be selected on each IO:
 30
 31		round-robin (0):
 32		    select path in per CPU round-robin manner.
 33
 34		min-inflight (1):
 35		    select path with minimum inflights.
 36
 37		min-latency (2):
 38		    select path with minimum latency.
 39
 40What:		/sys/class/rtrs-client/<session-name>/paths/
 41Date:		Feb 2020
 42KernelVersion:	5.7
 43Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 44Description:	Each path belonging to a given session is listed here by its source and
 45		destination address. When a new path is added to a session by writing to
 46		the "add_path" entry, a directory <src@dst> is created.
 47
 48What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
 49Date:		Feb 2020
 50KernelVersion:	5.7
 51Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 52Description:	RO, Contains "connected" if the session is connected to the peer and fully
 53		functional.  Otherwise the file contains "disconnected"
 54
 55What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
 56Date:		Feb 2020
 57KernelVersion:	5.7
 58Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 59Description:	Write "1" to the file in order to reconnect the path.
 60		Operation is blocking and returns 0 if reconnect was successful.
 61
 62What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
 63Date:		Feb 2020
 64KernelVersion:	5.7
 65Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 66Description:	Write "1" to the file in order to disconnect the path.
 67		Operation blocks until RTRS path is disconnected.
 68
 69What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
 70Date:		Feb 2020
 71KernelVersion:	5.7
 72Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 73Description:	Write "1" to the file in order to disconnected and remove the path
 74		from the session.  Operation blocks until the path is disconnected
 75		and removed from the session.
 76
 77What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
 78Date:		Feb 2020
 79KernelVersion:	5.7
 80Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 81Description:	RO, Contains the name of HCA the connection established on.
 82
 83What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
 84Date:		Feb 2020
 85KernelVersion:	5.7
 86Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 87Description:	RO, Contains the port number of active port traffic is going through.
 88
 89What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
 90Date:		Feb 2020
 91KernelVersion:	5.7
 92Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 93Description:	RO, Contains the source address of the path
 94
 95What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
 96Date:		Feb 2020
 97KernelVersion:	5.7
 98Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
 99Description:	RO, Contains the destination address of the path
100
101What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency
102Date:		Feb 2020
103KernelVersion:	5.7
104Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
105Description:	RO, Contains the latency time calculated by the heart-beat messages.
106		Whenever the client sends heart-beat message, it checks the time gap
107		between sending the heart-beat message and receiving the ACK.
108		This value can be changed regularly.
109
110What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
111Date:		Feb 2020
112KernelVersion:	5.7
113Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
114Description:	RW, Read will return usage help, write 0 will clear all the statistics.
115
116What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
117Date:		Feb 2020
118KernelVersion:	5.7
119Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
120Description:	RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
121		not the case, the processing of an I/O response could be processed on a
122		different CPU than where it was originally submitted.  This file shows
123		how many interrupts where generated on a non expected CPU.
124
125		"from:"
126		    is the CPU on which the IRQ was expected, but not generated.
127		"to:"
128		    is the CPU on which the IRQ was generated, but not expected.
129
130What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
131Date:		Feb 2020
132KernelVersion:	5.7
133Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
134Description:	Contains 2 unsigned int values, the first one records number of successful
135		reconnects in the path lifetime, the second one records number of failed
136		reconnects in the path lifetime.
137
138What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
139Date:		Feb 2020
140KernelVersion:	5.7
141Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
142Description:	Contains statistics regarding rdma operations and inflight operations.
143		The output consists of 6 values::
144
145		    <read-count> <read-total-size> <write-count> \
146		    <write-total-size> <inflights> <failovered>