Require drivers to report the total number of Infiniband ports. This
is necessary to report the correct number of ports on devices with
dynamic port types.
For example, dual-port Mellanox cards configured for (eth, ib) would
be rejected by the subnet manager, because they report using "port 2,
out of 1".
Signed-off-by: Christian Iversen <ci@iversenit.dk>
The malloc_dma() function allocates memory with specified physical
alignment, and is typically (though not exclusively) used to allocate
memory for DMA.
Rename to malloc_phys() to more closely match the functionality, and
to create name space for functions that specifically allocate and map
DMA-capable buffers.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When the area to be mapped straddles the 2GB boundary, the expression
(high+size) will overflow on the first loop iteration. Fix by using
(end-size), which cannot underflow.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Arbel seems to crash the system as soon as the first send WQE
completes on an RC queue pair. (NOPs complete successfully, so this
is a problem specific to the work queue rather than the completion
queue.) The cause of this problem has remained unknown for over a
year.
Check in the non-functioning code to avoid bit-rot, and in the hope
that someone will find the fix.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Mapping a single page at a time causes a several-second delay at
device initialisation time. Reduce this by mapping multiple pages at
a time, using the largest block sizes possible given the alignment
constraints.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use individual page mappings rather than a single whole-region
mapping, to avoid the waste of memory that occurs due to the
constraint that each mapped block must be aligned on its own size.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Backport some changes from the Hermon driver to the Arbel driver.
Specifically:
o Rename reserved_lkey to lkey
o Add arbel_rate() to calculate transmission rates
o Structure code to allow for addition of RC queue pairs
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Reduce the amount of ICM space required by choosing to order the
various allocations in approximately descending order of alignment
requirements.
This saves approximately 512kB of host memory.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The current method for ICM allocation exactly matches the addresses
chosen by the old Etherboot driver, but does not match the
specification. Some ICM tables (notably the queue pair context table)
therefore end up incorrectly aligned.
Fix by performing allocations as per the specification.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Improve the utility of debugging messages by including the relevant
port number, queue number (QPN, CQN, EQN), work queue entry (WQE)
number, and physical addresses wherever applicable.
Add arbel_dump_cqctx() for dumping a completion queue context and
arbel_dump_qpctx() for dumping a queue pair context.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
This is a backport of commit 0b1222f ("[hermon] Randomise the
high-order bits of queue pair numbers") to the Arbel driver.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
This is a backport of commit cd5a213 ("[hermon] Allow software GMA to
receive packets destined for QP1") to the Arbel driver.
This patch includes a correction to a bug in the autogenerated
hardware description header file.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Only port state change events are currently mapped to our event queue,
since those are the only events we are prepared to handle. This
ignores a potentially useful source of diagnostic information in the
case of unexpected failures.
Fix by mapping all events to the event queue; a build with debugging
enabled will therefore at least dump the raw content of the unexpected
events.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
No event is generated upon reaching INIT, so we must poll separately
for link state changes while we remain DOWN.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
ib_smc_update() potentially updates the Infiniband port state, and so
should almost always be followed by a call to ib_link_state_changed().
The one exception is the call made to ib_smc_update() before the
device is registered.
Fix by removing explicit calls to ib_link_state_changed() from drivers
using ib_smc_update(), including a call to ib_link_state_changed()
within ib_smc_update(), and creating a separate ib_smc_init() for use
prior to device registration.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Access to the gpxe.org and etherboot.org domains and associated
resources has been revoked by the registrant of the domain. Work
around this problem by renaming project from gPXE to iPXE, and
updating URLs to match.
Also update README, LOG and COPYRIGHTS to remove obsolete information.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Queue pairs are now assumed to be created in the INIT state, with a
call to ib_modify_qp() required to bring the queue pair to the RTS
state.
ib_modify_qp() no longer takes a modification list; callers should
modify the relevant queue pair parameters (e.g. qkey) directly and
then call ib_modify_qp() to synchronise the changes to the hardware.
The packet sequence number is now a property of the queue pair, rather
than of the device.
Each queue pair may have an associated address vector. For RC queue
pairs, this is the address vector that will be programmed in to the
hardware as the remote address. For UD queue pairs, it will be used
as the default address vector if none is supplied to ib_post_send().
This brings us in to line with Linux definitions, and also simplifies
adding x86_64 support since both platforms have 2-byte shorts, 4-byte
ints and 8-byte long longs.
Not all Infiniband cards have embedded subnet management agents.
Split out the code that communicates with such an embedded SMA into a
separate ib_smc.c file, and have drivers call ib_smc_update()
explicitly when they suspect that the answers given by the embedded
SMA may have changed.
Receive completion handlers now get passed an address vector
containing the information extracted from the packet headers
(including the GRH, if present), and only the payload remains in the
I/O buffer.
This breaks the symmetry between transmit and receive completions, so
remove the ib_completer_t type and use an ib_completion_queue_operations
structure instead.
Rename the "destination QPN" and "destination LID" fields in struct
ib_address_vector to reflect its new dual usage.
Since the ib_completion structure now contains only an IB status code,
("syndrome") replace it with a generic gPXE integer status code.
Avoid leaking I/O buffers in ib_destroy_qp() by completing any
outstanding work queue entries with a generic error code. This
requires the completion handlers to be available to ib_destroy_qp(),
which is done by making them static configuration parameters of the CQ
(set by ib_create_cq()) rather than being provided on each call to
ib_poll_cq().
This mimics the functionality of netdev_{tx,rx}_flush(). The netdev
flush functions would previously have been catching any I/O buffers
leaked by the IPoIB data queue (though not by the IPoIB metadata
queue).
We were accidentally allocating only half the required amount of
memory (given the alignment method) for the firmware buffer, leading
to conflicts between the firmware buffer and gPXE code/data segments.
Arbel and Hermon cards both have multiple ports. Add the
infrastructure required to register each port as a separate IB
device. Don't yet register more than one port, since registration
will currently fail unless a valid link is detected.
Use ib_*_{set,get}_{drv,owner}data wrappers to access driver- and
owner-private data on Infiniband structures.
Christian Iversen
Michael Brown
Michael Brown
Thomas Miletich
Alexey Zaytsev