Some BIOSes in AWS EC2 (observed with a c6i.metal instance in
eu-west-2) will fail to assign an MMIO address to the ENA device,
which causes ioremap() to fail.
Experiments show that the ENA device is the only device behind its
bridge, even when multiple ENA devices are present, and that the BIOS
does assign a memory window to the bridge.
We may therefore choose to assign the device an MMIO address at the
start of the bridge's memory window.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a minimal driver for PCI bridges that can be used to locate the
bridge to which a PCI device is attached.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Pretty much all physical machines and off-the-shelf virtual machines
will provide a functional PCI BIOS. We therefore default to using
only the PCI BIOS, with no fallback to an alternative mechanism if the
PCI BIOS fails.
AWS EC2 provides the opportunity to experience some exceptions to this
rule. For example, the t3a.nano instances in eu-west-1 have no
functional PCI BIOS at all. As of commit 83516ba ("[cloud] Use
PCIAPI_DIRECT for cloud images") we therefore use direct Type 1
configuration space accesses in the images built and published for use
in the cloud.
Recent experience has discovered yet more variation in AWS EC2
instances. For example, some of the metal instance types have
multiple PCI host bridges and the direct Type 1 accesses therefore
see only a subset of the PCI devices.
Attempt to accommodate future such variations by making the PCI I/O
API selectable at runtime and choosing ECAM (if available), falling
back to the PCI BIOS (if available), then finally falling back to
direct Type 1 accesses.
This is implemented as a dedicated PCIAPI_CLOUD API, rather than by
having the PCI core select a suitable API at runtime (as was done for
timers in commit 302f1ee ("[time] Allow timer to be selected at
runtime"). The common case will remain that only the PCI BIOS API is
required, and we would prefer to retain the optimisations that come
from inlining the configuration space accesses in this common case.
Cloud images are (at present) disk images rather than ROM images, and
so the increased code size required for this design approach in the
PCIAPI_CLOUD case is acceptable.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The ACPI MCFG table describes a direct mapping of PCI configuration
space into MMIO space. This mapping allows access to extended
configuration space (up to 4096 bytes) and also provides for the
existence of multiple host bridges.
Add support for the ECAM mechanism described by the ACPI MCFG table,
as a selectable PCI I/O API alongside the existing PCI BIOS and Type 1
mechanisms.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow pci_find_next() to discover devices beyond the first PCI
segment, by generalising pci_num_bus() (which implicitly assumes that
there is only a single PCI segment) with pci_discover() (which has the
ability to return an arbitrary contiguous chunk of PCI bus:dev.fn
address space).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The semantics of the bus:dev.fn parameter passed to pci_find_next()
are "find the first existent PCI device at this address or higher",
with the caller expected to increment the address between finding
devices. This does not allow the parameter to distinguish between the
two cases "start from address zero" and "wrapped after incrementing
maximal possible address", which could therefore lead to an infinite
loop in the degenerate case that a device with address ffff:ff:1f.7
really exists.
Fix by checking for wraparound in the caller (which is already
responsible for performing the increment).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Separate the return status code from the returned PCI bus:dev.fn
address, in order to allow pci_find_next() to be used to find devices
with a non-zero PCI segment number.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some versions of the ENA hardware (observed on a c6i.large instance in
eu-west-2) seem to require a receive ring containing at least 128
entries: any smaller ring will never see receive completions or will
stall after the first few completions.
Increase the receive ring size to 128 entries (determined empirically)
for compatibility with these hardware versions. Limit the receive
ring fill level to 16 (as at present) to avoid consuming more memory
than will typically be available in the internal heap.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some versions of the ENA firmware (observed on a c6i.large instance in
eu-west-2) seem to require a host information page, without which the
CREATE_CQ command will fail with ENA_ADMIN_UNKNOWN_ERROR.
These firmware versions also seem to require us to claim that we are a
Linux kernel with a specific driver major version number. This
appears to be a firmware bug, as revealed by Linux kernel commit
1a63443af ("net/amazon: Ensure that driver version is aligned to the
linux kernel"): this commit changed the value of the driver version
number field to be the Linux kernel version, and was hastily reverted
in commit 92040c6da ("net: ena: fix broken interface between ENA
driver and FW") which clarified that the version number field does
actually have some undocumented significance to some versions of the
firmware.
Fix by providing a host information page via the SET_FEATURE command,
incorporating the apparently necessary lies about our identity.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some versions of the ENA firmware (observed on a c6i.large instance in
eu-west-2) will complain if the completion queue's MSI-X vector field
is left empty, even though the queue configuration specifies that
interrupts are not used.
Work around these firmware versions by passing in what appears to be
the magic "no MSI-X vector" value in this field.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The ENA data path design has separate submission and completion
queues. Submission queues must be refilled in strict order (since
there is only a single linear tail pointer used to communicate the
existence of new entries to the hardware), and completion queue
entries include a request identifier copied verbatim from the
submission queue entry. Once the submission queue doorbell has been
rung, software never again reads from the submission queue entry and
nothing ever needs to write back to the submission queue entry since
completions are reported via the separate completion queue.
This design allows the hardware to complete submission queue entries
out of order, provided that it internally caches at least as many
entries as it leaves gaps.
Record and identify I/O buffers by request identifier (using a
circular ring buffer of unique request identifiers), and remove the
assumption that submission queue entries will be completed in order.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The CREATE_CQ command is permitted to return a size smaller than
requested, which could leave us in a situation where the completion
queue could overflow.
Avoid overflow by limiting the submission queue fill level to the
actual size of the completion queue.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Current versions of the E810 PF driver fail to set the number of
in-use queue pairs in response to the CONFIG_VSI_QUEUES message. When
the number of in-use queue pairs is less than the number of available
queue pairs, this results in some packets being directed to
nonexistent receive queues and hence silently dropped.
Work around this PF driver bug by explicitly configuring the number of
available queue pairs via the REQUEST_QUEUES message. This message
triggers a VF reset that, in turn, requires us to reopen the admin
queue and issue an additional GET_RESOURCES message to restore the VF
to a functional state.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The RESET_VF admin queue command does not complete via the usual
mechanism, but instead requires us to poll registers to wait for the
reset to take effect and then reopen the admin queue.
Allow for the existence of other admin queue commands that also
trigger a VF reset, by separating out the logic that waits for the
reset to complete.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Negotiate API version 1.1 in order to allow access to virtual function
opcodes that are disallowed by default on the E810.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a driver for the E810 family of 100 Gigabit Ethernet NICs. The
core datapath is identical to that of the 40 Gigabit XL710, and this
part of the code is shared between both drivers. The admin queue
mechanism is sufficiently similar to make it worth reusing substantial
portions of the code, with separate implementations for several
commands to handle the (unnecessarily) breaking changes in data
structure layouts. The major differences are in the mechanisms for
programming queue contexts (where the E810 abandons TX/RX symmetry)
and for configuring the transmit scheduler and receive filters: these
portions are sufficiently different to justify a separate driver.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove knowledge of the PRTGL_SA[HL] registers, and instead use the
admin queue to set the MAC address and maximum frame size.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove knowledge of the PRTPM_SA[HL] registers, and instead use the
admin queue to retrieve the MAC address.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow for the MAC address to be fetched using an admin queue command,
instead of reading the PRTPM_SA[HL] registers directly.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The PRTGL_SAH register contains the current maximum frame size, and is
not guaranteed on reset to contain the actual maximum frame size
supported by the hardware, which the datasheet specifies as 9728 bytes
(including the 4-byte CRC).
Set the maximum packet size to a hardcoded 9728 bytes instead of
reading from the PRTGL_SAH register.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove knowledge of the GLLAN_RCTL_0 register (which changes location
between the XL810 and E810 register maps), and instead unconditionally
issue the "clear PXE mode" command with the EEXIST error silenced.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The "clear PXE mode" admin queue command will return an EEXIST error
if the device is already in non-PXE mode, but there is no other admin
queue command that can be used to determine whether the device has
already been switched into non-PXE mode.
Provide a mechanism to allow expected errors from a command to be
silenced, to allow the "clear PXE mode" command to be cleanly used
without needing to first check the GLLAN_RCTL_0 register value.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
At least one E810 admin queue command (Query Default Scheduling Tree
Topology) insists upon being provided with a 4kB data buffer, even
when the data to be returned is much smaller.
Work around this requirement by increasing the admin queue data buffer
size to 4kB.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Move knowledge of the virtual function data structures and admin
command definitions from intelxl.h to intelxlvf.h.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove the large static admin data buffer structure embedded within
struct intelxl_nic, and instead copy the response received via the
"send to VF" admin queue event to the (already consumed and completed)
admin command descriptor and data buffer.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The physical and virtual function drivers each care about precisely
one admin queue event type. Simplify event handling by using a
per-driver callback instead of the existing weak function symbol.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The PCI device ID 8086:1889 is for the Intel Ethernet Adaptive Virtual
Function, which is a generic virtual function that can be exposed by
different generations of Intel hardware.
Rename the PCI ID from "xl710-vf-ad" to "iavf" to reflect that the
driver is not XL710-specific.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The E810 requires that receive descriptor rings have at least 64
entries (and are a multiple of 32 entries).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Do not attempt to use the admin commands to get the firmware version
and report the driver version for the virtual function driver, since
these will be rejected by the E810 firmware as invalid commands when
issued by a virtual function. Instead, use the mailbox interface to
negotiate the API version with the physical function driver.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The 100 Gigabit physical function driver requires a virtual function
driver to request that transmit and receive queues are mapped to MSI-X
vector 1 or higher.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The second parameter to intelxlvf_admin_queues() is a boolean used to
select the VF opcode, rather than the raw VF opcode itself.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove knowledge of the PFGEN_CTRL register (which changes location
between XL710 and E810 register maps), and instead use PCIe FLR to
reset the physical function.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use the tail register offset (which exists for all ring types) as the
ring identifier in all relevant debug messages.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
For the sake of completeness, ensure that all 32 bytes of the receive
queue context are programmed (including the unused final 8 bytes).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 8f3e648 ("[intelxl] Use one admin queue buffer per admin queue
descriptor") changed the API for intelxl_admin_command() such that the
caller now constructs the command directly within the next available
descriptor ring entry, rather than relying on intelxl_admin_command()
to copy the descriptor to and from the descriptor ring.
This introduced a regression in intelxl_admin_switch(), since the
second and subsequent iterations of the loop will not have constructed
a valid command in the new descriptor ring entry before calling
intelxl_admin_command().
Fix by constructing the command within the loop.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When applying an ACPI-provided system-specific MAC address, apply it
to netdev->ll_addr rather than netdev->hw_addr. This allows iPXE
scripts to access the permanent MAC address via the ${netX/hwaddr}
setting (and thereby provides scripts with a mechanism to ascertain
that the NIC is using a MAC address other than its own permanent
hardware address).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As per the general pattern for initialisation functions in iPXE,
pci_init() saves code size by assuming that the caller has already
zeroed the underlying storage (e.g. as part of zeroing a larger
containing structure). There are several places within the code where
pci_init() is deliberately used to initialise a transient struct
pci_device without zeroing the entire structure, because the calling
code knows that only the PCI bus:dev.fn address is required to be
initialised (e.g. when reading from PCI configuration space).
Ensure that using pci_init() followed by pci_read_config() will fully
initialise the struct pci_device even if the caller did not previously
zero the underlying storage, since Coverity reports that there are
several places in the code that rely upon this.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Several keyboard layouts define ASCII characters as accessible only
via the AltGr modifier. Add support for this modifier to ensure that
all ASCII characters are accessible.
Experiments suggest that the BIOS console is likely to fail to
generate ASCII characters when the AltGr key is pressed. Work around
this limitation by accepting LShift+RShift (which will definitely
produce an ASCII character) as a synonym for AltGr.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Handle Ctrl and CapsLock key modifiers within key_remap(), to provide
consistent behaviour across different console types.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The key with scancode 86 appears in the position between left shift
and Z on a US keyboard, where it typically fails to exist entirely.
Most US keyboard maps define this nonexistent key as generating "\|",
with the notable exception of "loadkeys" which instead reports it as
generating "<>". Both of these mapping choices duplicate keys that
exist elsewhere in the map, which causes problems for our ASCII-based
remapping mechanism.
Work around these quirks by treating the key as generating "\|" with
the high bit set, and making it subject to remapping. Where the BIOS
generates "\|" as expected, this allows us to remap to the correct
ASCII value.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some keyboard layouts (e.g. "fr") swap letter and punctuation keys.
Apply the logic for upper and lower case and for Ctrl-<key> only after
applying remapping, in order to handle these layouts correctly.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
On some systems (observed with the Thunderbolt ports on a ThinkPad X1
Extreme Gen3 and a ThinkPad P53), if the IOMMU is enabled then the
system firmware will install an ExitBootServices notification event
that disables bus mastering on the Thunderbolt xHCI controller and all
PCI bridges, and destroys any extant IOMMU mappings. This leaves the
xHCI controller unable to perform any DMA operations.
As described in commit 236299b ("[xhci] Avoid DMA during shutdown if
firmware has disabled bus mastering"), any subsequent DMA operation
attempted by the xHCI controller will end up completing after the
operating system kernel has reenabled bus mastering, resulting in a
DMA operation to an area of memory that the hardware is no longer
permitted to access and, on Windows with the Driver Verifier enabled,
a STOP 0xE6 (DRIVER_VERIFIER_DMA_VIOLATION).
That commit avoids triggering any DMA attempts during the shutdown of
the xHCI controller itself. However, this is not a complete solution
since any attached and opened USB device (e.g. a USB NIC) may
asynchronously trigger DMA attempts that happen to occur after bus
mastering has been disabled but before we reset the xHCI controller.
Avoid this problem by installing our own ExitBootServices notification
event at TPL_NOTIFY, thereby causing it to be invoked before the
firmware's own ExitBootServices notification event that disables bus
mastering.
This unsurprisingly causes the shutdown hook itself to be invoked at
TPL_NOTIFY, which causes a fatal error when later code attempts to
raise the TPL to TPL_CALLBACK (which is a lower TPL). Work around
this problem by redefining the "internal" iPXE TPL to be variable, and
set this internal TPL to TPL_NOTIFY when the shutdown hook is invoked.
Avoid calling into an underlying SNP protocol instance from within our
shutdown hook at TPL_NOTIFY, since the underlying SNP driver may
attempt to raise the TPL to TPL_CALLBACK (which would cause a fatal
error). Failing to shut down the underlying SNP device is safe to do
since the underlying device must, in any case, have installed its own
ExitBootServices hook if any shutdown actions are required.
Reported-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Tested-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
On some systems (observed with the Thunderbolt ports on a ThinkPad X1
Extreme Gen3 and a ThinkPad P53), the system firmware will disable bus
mastering on the xHCI controller and all PCI bridges at the point that
ExitBootServices() is called if the IOMMU is enabled. This leaves the
xHCI controller unable to shut down cleanly since all commands will
fail with a timeout.
Commit 85eb961 ("[xhci] Allow for permanent failure of the command
mechanism") allows us to detect that this has happened and respond
cleanly. However, some unidentified hardware component (either the
xHCI controller or one of the PCI bridges) seems to manage to enqueue
the attempted DMA operation and eventually complete it after the
operating system kernel has reenabled bus mastering. This results in
a DMA operation to an area of memory that the hardware is no longer
permitted to access. On Windows with the Driver Verifier enabled,
this will result in a STOP 0xE6 (DRIVER_VERIFIER_DMA_VIOLATION).
Work around this problem by detecting when bus mastering has been
disabled, and immediately failing the device to avoid initiating any
further DMA attempts.
Reported-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown
Benedikt Braunger