Implement a Python equivalent of ogCopyEfiBootLoader as the
function copy_efi_bootloader. This function copies the contents of
the folder of the EFI loader in the ESP into a ogBoot folder at
the root of the partition target of an image creation.
copy_efi_bootloader is a Windows only functionality.
Control non 0 returncode of the lzop and partclone subprocess
in image creation and restoration because this means that either
lzop or partclone has failed.
The implementation must cover cases such as not enough storage
space and log errors into /tmp/command.log and the log file of
the client handling the request.
Check the returncode of lzop and partclone subprocesses and
log the stderr of the process reporting non zero returncode.
Log the whole context of the error when an exception happens.
The previous exception handling was hidding important information
about the cause of the error.
The Windows bootloader only supports a UEFI boot from a GPT
partition. Set enforce_gpt to True in every codepath related to
Windows. When enforce_gpt is set to True get_efi_partition()
raises an exception when an MBR partition scheme is detected.
Make is_uefi_supported() only check for /sys/firmware/efi as
get_efi_partition() will detect a missing ESP or an invalid
partition scheme. Stop using get_efi_partition() inside
is_uefi_supported() as the former is eventually called in every
UEFI related code.
UEFI supports both MBR and GPT as partition schemes and this is
a required change to handle the particular case of Windows not
being able to boot UEFI from a MBR partition scheme.
Provide more information in exception messages as those are the
source of the logging messages. Add information about paths, files
or configuration related to the operation associated to the
exception.
Replace exception types to be more explicit about the nature of
the error.
Improve the exception raising semantics by using the 'from' keyword,
this wraps an older exception into a new one so it is still considered
the same object.
Use only the exception messages as the main resource for error
messages.
The previous error code had string duplication in the form of:
logging.error('msg here')
raise Exception('msg here')
That approach also has the downside of having log duplication as
it had the local logging.err() and a global logging.exception()
inside send_internal_server_error capturing the exception message.
The actual code only requires raising an exception with a proper
error message.
Improve exception messages to give more error context.
Log every AssertionError as a backtrace.
Use the 'raise Exception from e' syntax to modify the a previously
raised exception 'e' into an exception with aditional context or
different type. This also prevents the message that warns about
newer exceptions being launch after an initial exception.
The main function must be able to handle the login of critical
error in the main ogClient class instance. Add a try except block
to the ogCLient run logic and move the relevant error logs into
the except block.
Delegate the error messages to the exception message. This is the
first step towards error message deduplication.
Create ogboot.me and ogboot.secondboot as empty files and
ogboot.firstboot with the value "iniciado" in the root of
the BIOS Windows system partition.
The files must contain data for GRUB to be able to write content,
therefore these are created containing 3072 null bytes.
The Windows boot process is handled by the "pxe" profile.
There the files ogboot.me, ogboot.firstboot and ogboot.secondboot
are used as a state machine to chose between booting Windows and
ogLive.
The first Windows boot happens if ogboot.me and ogboot.firstboot
are identical, then "iniciado" is written in ogboot.firstboot.
We skip this stage as we create ogboot.firstboot with 'iniciado'.
The second Windows boot occurs if ogboot.me and ogboot.secondboot
are boot identical, then "iniciado" is written in ogboot.secondboot.
After the Windows boot ogLive is booted.
Create a bios.py file to hold all the BIOS specific functions.
Implement the _boot_bios_linux in Python. The new boot process
tries to find the vmlinuz and initrd binaries at the desired
partition. Then it tries to load them with kexec with the proper
Grub boot params.
One step closer to the removal of the boot legacy script.
The mage creation process was being interrupted by an error
trying to read the Windows registry by the Hivex library.
Now the exceptions are handled and an error is reported.
The OS probe logic must be able to check a distro programmatically,
add get_linux_distro_id to return an id whitout versioning.
Ensure the availability of 'ubuntu' when we need to ensure certain
features are only used with a supported system.
This change is a preparative for reimplementing the BIOS boot
in order to deprecate the legacy script. All the codepaths to
boot systems located at a partition are now called from the
boot_os_at function enabling an easier structure for the incoming
code.
Checking the existence /sys/firmware/efi as it might appear
sometimes in BIOS installs if the BIOS configuration is not
proper. Checking for the EFI partition is the safest method
to veryfy the install type.
The function getlinuxversion receives a path to the os-release
file. The case of not being able to open it was not handled and
thus causing an unwanted exception.
The json functionality proposed upstream might be merged one day
in efibootmgr so deploying a fork would not be needed anymore.
This change aims to ease the migration once that day comes.
Replace IniciarSesion script in favor of native Python code when booting
a UEFI system into Linux. This completes the implementation of booting
into an OS on a UEFI compliant system.
Replace IniciarSesion script in favor of native Python code when booting
a UEFI system. This applies when running the "session" command.
WIP: Only UEFI boots Windows systems. Raise NotImplementedError
exception trying to boot a Linux system using UEFI.
Add utility module related to the process of booting a system from a
client's partition.
The main utility function to boot a clients system is boot_os_at(), from
which firmware (UEFI or BIOS) and os-family specific private functions are invoked.
This initial commit adds UEFI windows boot function.
Add UEFI related utilities inside a new utility module: uefi.py
_check_efibootmgr_json
======================
Check if the system efibootmgr executable supports JSON output. This is
a private function used only by other functions from uefi.py.
is_uefi_supported
=================
Check if the system supports UEFI firmware.
run_efibootmgr_json
===================
Runs efibootmgr with json output support. Return the JSON output as a
Python dict.
efibootmgr_create_bootentry
===========================
Create nvram boot entry. This bootentry is usually later set to boot
next just once via "BootNext" nvram variable.
efibootmgr_delete_bootentry
===========================
Delete a nvram boot entry. Used to avoid duplicates when booting the
same disk and partition from a given client.
efibootmgr_bootnext
===================
Set nvram "BootNext" variable to a given boot entry so after client
reboot, PXE is not executed and the given boot entry takes precedence.
Add dependency with efibootmgr version >= 18, and efibootmgr JSON output
which is currently out of tree from util-linux repo.
Add a basic OS family enumeration: OSFamily.
Add utility function that probes for an installed Linux or Windows
system, returns the corresponding enum value, OSFamily.UNKNOWN
otherwise.
Add utility function inside disk.py to find, if any, the first ESP
partition of a given disk.
The disk is provided as an integer (starting at 1 following OpenGnsys
scripts usual values), meaning the (n-1)th disk from the disk array
returned from get_disks(). In the future a better mechanism should be
put in place to fetch probed disks from a running client.
This change is part of the upcoming drop of "IniciarSesion" script in
favor of a Python native approach. Specifically regarding UEFI systems.
value extraction did not have error checking and was handled in
a one-liner. The actual implementation expands the parsing logic
and moves it into a function.
Revisit 5056b8f0d5 ("fs: validate ntfsresize dry-run output") that has
introduced a possible infinity loop.
Disentangle this loop while at it: iterate until best smallest size is
found by probing.
do not return the returncode, instead return an integer.
do not use
except CalledProcessError as e:
it causes a another exception while handling exception.
Remount the original image repository.
it should be possible to simplify this further by:
- stacking mounts, no need to umount initial repo and mount it again
when switching to the new repo, because remount back initial repo
might fail (!)
- use check=False and simply check for x.returncode
Remove mbuffer, this is never used.
mbuffer has been never been used since ogClient supports native image restore.
Originally this was used like this:
partclone [...] | mbuffer -q -M 40M | lzop [...]
supposely to speed up partclone in case the device where the read happens is
slowier than the device that is used for writes.
See mbuffer(1) manpage examples.
In any case, this needs benchmarking to really make sure this is helping.
Remove it until that ever happens.
cover more error cases where exceptions need to be raised.
check return code in the invoked subprocess.
restoreImageCustom has been intentionally left behind, it
is unclear what this custom script returns on success and
error.
validate 'Needed relocations: ' is in place before stepping on the split chunks
(2024-01-11 10:28:16) ogClient: [ERROR] - Exception when running "image create" subprocess
Traceback (most recent call last):
File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 454, in image_create
ogReduceFs(disk, partition)
File "/opt/opengnsys/ogClient/src/utils/fs.py", line 105, in ogReduceFs
_reduce_ntfsresize(partdev)
File "/opt/opengnsys/ogClient/src/utils/fs.py", line 235, in _reduce_ntfsresize
extra_size = int(out_resize_dryrun.split('Needed relocations : ')[1].split(' ')[0])*1.1+1024
IndexError: list index out of range
if not present, no need to adjust size
Alejandro Sirgo Rica
OpenGnSys Support Team
Jose M. Guisado