Commit Graph

368 Commits (fa5b37b2a6c2f10d0f380f0da07f23e9288e70f8)
 

Author SHA1 Message Date
OpenGnSys Support Team fa5b37b2a6 live: rename variable that stores json body in refresh()
Just a simple cleanup.
2024-05-21 11:51:29 +02:00
OpenGnSys Support Team b9ca7278a9 live: parttypes: add EFI partition type in MBR partition scheme
Add 0xef partition type for EFI in MBR.
2024-05-14 11:49:54 +02:00
OpenGnSys Support Team 3eb4332b38 live: partcodes: remove reference to website
Remove reference to external website in code.
2024-05-14 11:35:16 +02:00
OpenGnSys Support Team d109e99dbe utils: rename cache_probe() to get_cache_dev_path()
This method reports the /dev path to cache partition, rename it.

Add explicit check if blkid is successful.

And add logging to report that device path to cache is not found.
2024-05-09 11:47:06 +02:00
Alejandro Sirgo Rica 6bfbf6cc7b utils: add error checks to checksum file creation in tip_write_csum
Add a check for potential permission or IO errors during the
creation of the image checksum.
2024-05-07 20:47:05 +02:00
Alejandro Sirgo Rica abea6583d3 utils: add mkdir error report in mount_mkdir
Add exception checks to the os.mkdir operation and log the error
found. The previous implementation was too optimistic and only
handled mount related errors.
2024-05-07 17:24:41 +02:00
Alejandro Sirgo Rica 493d998a4e utils: remove unused json import
The json library was a dependency during the development of the
boot OS functions and it is no longer needed in that file.
2024-05-07 16:08:38 +02:00
Alejandro Sirgo Rica b0d03ce588 utils:fs: fix logging statement in get_filesystem_type
Call the error() logging function from the logging object instead
of the non existent log variable previously referenced.
2024-05-07 12:20:26 +02:00
Alejandro Sirgo Rica de17bb6812 utils:fs: add mkfs logs when return code is not 0
Report mkfs failure for every partition. This does not raise an
exception as that would skip partprobe operations and the mkfs
operations in the next potentially well formated partitions.
2024-05-07 12:20:26 +02:00
OpenGnSys Support Team 9ffe1c81bf live: report LINUX-SWAP instead of SWAP
ogCP expects LINUX-SWAP to specify a swap filesystem.

Add a similar workaround to the one that is done for VFAT for symmetry between
inputs and outputs that circulate over the API.
2024-05-07 11:54:39 +02:00
OpenGnSys Support Team 1ca3639389 live: rewrite log in case tiptorrent client fails
Specify that image file cannot be found in cache because tiptorrent has failed,
otherwise it is confusing.
2024-05-06 19:10:48 +02:00
OpenGnSys Support Team 8ecd57552a live: restore partprobe before building filesystem
Otherwise mkfs silently fails because OS reports out-of-sync partition table.
2024-05-06 19:08:12 +02:00
OpenGnSys Support Team 2dd5105995 live: force flush to disk after partition table is written 2024-05-06 18:51:57 +02:00
OpenGnSys Support Team f42e2ba201 live: partprobe breaks with mounted partitions
partprobe requires that all disk partitions are unmounted.

partprobe needs to be called to report the OS that the partition table
has changed, otherwise ogclient reports incorrect partition information.

iterate over the partition list and mount cache after partprobe is
called.
2024-05-06 18:33:41 +02:00
OpenGnSys Support Team 170d7e1be9 live: umount all partitions before partition setup
If new partition layout is specified, unmount cache and any other partition
under /mnt.
2024-05-06 18:32:54 +02:00
Alejandro Sirgo Rica 8171ddd15f live: fix omited error report in tip_client_get
tip_client_get raises the proper error exceptions but the except
block in _restore_image_tiptorrent overwrites the reported error.
Move the raise statements in _restore_image_tiptorrent outside
of the except block.
2024-05-06 14:23:29 +02:00
OpenGnSys Support Team 5ea6f73343 live: add fat32 to get_parttype()
Use "Microsoft basic data partition" to store FAT32 in case of GPT partition
scheme and 0xB according in case of MBR partition scheme according to
documentation.
2024-04-23 10:56:44 +02:00
OpenGnSys Support Team 84c2944bf3 live: revisit logging for partition setup, image create and restore 2024-04-23 10:55:37 +02:00
OpenGnSys Support Team 7d26f0f69b utils: add logging for checksum validation
Display if checksum validation is correct or not.
2024-04-22 14:56:09 +02:00
OpenGnSys Support Team 8661a99a36 utils: fs: incorrect indentation 2024-04-22 13:56:39 +02:00
OpenGnSys Support Team bddbea942e utils: fs: skip shrink/extend operation for fat
fat does not support this operation, skip it
2024-04-19 12:19:54 +02:00
OpenGnSys Support Team 481ce5794f utils: fs: remove unused variable in _reduce_resize2fs
ret is set but never used in this method
2024-04-19 11:32:53 +02:00
OpenGnSys Support Team 85bbdcfcb8 utils: fs: report error when failing to grow
not really a warning, report an error instead
2024-04-19 11:32:17 +02:00
OpenGnSys Support Team db23ea76f9 utils: fs: set return value in ogReduceFs
otherwise error path uses uninitialized variable

 File "/opt/opengnsys/ogClient/src/utils/fs.py", line 112, in ogReduceFs
 return ret
 UnboundLocalError: local variable 'ret' referenced before assignment
2024-04-19 11:29:24 +02:00
Alejandro Sirgo Rica e19437290d live: improve exception handling in image_create
Reduce the scope of the try except block that controls the case
of deleting the image backup in case of error. Now it only covers
the section of code after backup creation and up to image
verification. Check when the Exception is an OgError to raise
with added context.

Prevent the deletion of the target image in case of error before
the backup creation.

Bundle the backup creation on its own try except block to give
more feedback on a failed backup creation.

Enables a better error management allowing unhandled
exceptions to be reported properly.
2024-04-03 13:31:10 +02:00
Alejandro Sirgo Rica cbe7f8d49c src: use explicit exception types in except Exception blocks
Capture only the relevant exception types in each except block.
The capture of the Exception type means hiding information for
unhandled error cases, even for syntax errors in the codebase.
Using a more fine grained exception filtering improves error
traceability.
2024-04-03 13:31:10 +02:00
Alejandro Sirgo Rica dfde363aa6 src: log backtrace in unhandled error cases
Log an error message in known error cases and log a backtrace
otherwise.

Define a new error type OgError to be used in all the 'raise'
blocks to define the error message to log. The exception
propagates until it reaches send_internal_server_error() where
the exception type is checked. If the type is OgError we log
the exception message. Logs the backtrace for other types.

The initial error implementation printed a backtrace everytime
an error ocurred. The next iteration changed it to only print
a backtrace in a very particular case but ended up omiting too
much information such as syntax errors or unknown error context.
The actual implementation only logs the cases we already cover in
the codebase and logs a bracktrace in the others, enabling a
better debugging experience.
2024-04-03 13:31:10 +02:00
OpenGnSys Support Team c5ccc3c7e2 qmp: incorrect copyright header
Replace incorrect copyright header in qmp.py file.
2024-04-02 13:20:27 +02:00
OpenGnSys Support Team 4712941828 utils: refine log when booting windows UEFI from DOS partition
Refine 97647c32aa utils: add enforce_gpt argument to get_efi_partition()
to provide more explicit error when trying to boot Windows UEFI from DOS
partition.
2024-04-01 14:28:14 +02:00
OpenGnSys Support Team 1aba9d0923 Revert "live: improve lzop and partclone error handling"
This reverts commit 57787dab54.

Read from stderr is blocking if no data is available, revert this patch since
ogClient hangs indefinitely in lzop invocations due to races in process
execution through Popen.
2024-04-01 13:44:17 +02:00
Alejandro Sirgo Rica 474183ab71 utils: fix mount error messages for os probe operations
Replace unexistent mountpoint variable to report a failed
mount operation before an OS probe from a partition.
Improve the semantics of the error message replacing 'at' with
'into'.
Remove the period at the end of the log message.
2024-04-01 13:02:46 +02:00
OpenGnSys Support Team a97fd4acad live: display info logging when restoring image starts
instead of using debug level, this is very useful to track the process.
2024-03-27 17:15:59 +01:00
Alejandro Sirgo Rica 25b00bfd69 live: use .ant image as main image after image creation error
Restore image file from .ant to original file name if new image
creation fails. Remove new imagen and move the .ant image file in
place of the original as previously an error meant a rename of the
image file without a revert to keep the image available.
2024-03-27 10:34:22 +01:00
Alejandro Sirgo Rica c3ce5342c9 utils: make init_cache() use the cache mountpoint
Make init_cache() use the actual cache mountpoint returned by the
function mount_cache() for the creation of the cache directories
instead of a hardcoded path.
2024-03-26 13:32:58 +01:00
Alejandro Sirgo Rica 3aa76e4039 utils: remove redundant return statements from mount_mkdir
Remove return statement the program won't reach and simplify the
return logic with a return from an if and a fallback return
statement.
2024-03-26 13:32:58 +01:00
Alejandro Sirgo Rica 55fadef718 utils: drop ogCopyEfiBootLoader script
Implement a Python equivalent of ogCopyEfiBootLoader as the
function copy_efi_bootloader. This function copies the contents of
the folder of the EFI loader in the ESP into a ogBoot folder at
the root of the partition target of an image creation.
copy_efi_bootloader is a Windows only functionality.
2024-03-26 13:32:58 +01:00
Alejandro Sirgo Rica 57787dab54 live: improve lzop and partclone error handling
Control non 0 returncode of the lzop and partclone subprocess
in image creation and restoration because this means that either
lzop or partclone has failed.
The implementation must cover cases such as not enough storage
space and log errors into /tmp/command.log and the log file of
the client handling the request.
Check the returncode of lzop and partclone subprocesses and
log the stderr of the process reporting non zero returncode.
2024-03-26 13:32:48 +01:00
Alejandro Sirgo Rica 16dcc9b25b live: improve logging in image_create
Log the whole context of the error when an exception happens.
The previous exception handling was hidding important information
about the cause of the error.
2024-03-26 13:23:04 +01:00
Alejandro Sirgo Rica 97647c32aa utils: add enforce_gpt argument to get_efi_partition()
The Windows bootloader only supports a UEFI boot from a GPT
partition. Set enforce_gpt to True in every codepath related to
Windows. When enforce_gpt is set to True get_efi_partition()
raises an exception when an MBR partition scheme is detected.
2024-03-26 13:23:04 +01:00
Alejandro Sirgo Rica 42791a1a7c revert 7f18485: improve uefi detection mechanism
Make is_uefi_supported() only check for /sys/firmware/efi as
get_efi_partition() will detect a missing ESP or an invalid
partition scheme. Stop using get_efi_partition() inside
is_uefi_supported() as the former is eventually called in every
UEFI related code.
UEFI supports both MBR and GPT as partition schemes and this is
a required change to handle the particular case of Windows not
being able to boot UEFI from a MBR partition scheme.
2024-03-26 13:15:58 +01:00
Alejandro Sirgo Rica 049b7a5a2b src: make exception messages more contextual and explicit
Provide more information in exception messages as those are the
source of the logging messages. Add information about paths, files
or configuration related to the operation associated to the
exception.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 8741b2e272 src: change generic exception types to be more explicit
Replace exception types to be more explicit about the nature of
the error.
Improve the exception raising semantics by using the 'from' keyword,
this wraps an older exception into a new one so it is still considered
the same object.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 167fd29634 utils: fix string not being defined as f-strings when they should
Add the string prefix f'' to string instances formated as f-string
but not properly constructed to expand the inline variables.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 657af13351 virtual: add missing check=True to subprocess.run()
Raise exception from subprocess.run() calls as previous code is
trying to capture these exceptions to handle de errors.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 2a4ce65a20 src: centralize error logging into send_internal_server_error
Use only the exception messages as the main resource for error
messages.
The previous error code had string duplication in the form of:
	logging.error('msg here')
	raise Exception('msg here')

That approach also has the downside of having log duplication as
it had the local logging.err() and a global logging.exception()
inside send_internal_server_error capturing the exception message.
The actual code only requires raising an exception with a proper
error message.
Improve exception messages to give more error context.
Log every AssertionError as a backtrace.
Use the 'raise Exception from e' syntax to modify the a previously
raised exception 'e' into an exception with aditional context or
different type. This also prevents the message that warns about
newer exceptions being launch after an initial exception.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 0cbf16461e ogclient: consolidate logging in client core
The main function must be able to handle the login of critical
error in the main ogClient class instance. Add a try except block
to the ogCLient run logic and move the relevant  error logs into
the except block.
Delegate the error messages to the exception message. This is the
first step towards error message deduplication.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 8012562302 utils: implement BIOS boot for windows
Create ogboot.me and ogboot.secondboot as empty files and
ogboot.firstboot with the value "iniciado" in the root of
the BIOS Windows system partition.
The files must contain data for GRUB to be able to write content,
therefore these are created containing 3072 null bytes.
The Windows boot process is handled by the "pxe" profile.
There the files ogboot.me, ogboot.firstboot and ogboot.secondboot
are used as a state machine to chose between booting Windows and
ogLive.
The first Windows boot happens if ogboot.me and ogboot.firstboot
are identical, then "iniciado" is written in ogboot.firstboot.
We skip this stage as we create ogboot.firstboot with 'iniciado'.
The second Windows boot occurs if ogboot.me and ogboot.secondboot
are boot identical, then "iniciado" is written in ogboot.secondboot.
After the Windows boot ogLive is booted.
2024-03-21 10:29:21 +01:00
Alejandro Sirgo Rica ddf08779ae utils: add disk index checks in get_partition_device
Control the possibility of a bad disk index. Handle the checks in
a similar fashion to the get_efi_partition function.
2024-03-21 10:29:13 +01:00
Alejandro Sirgo Rica a3ffdf2370 utils: implement BIOS boot for Linux
Create a bios.py file to hold all the BIOS specific functions.
Implement the _boot_bios_linux in Python. The new boot process
tries to find the vmlinuz and initrd binaries at the desired
partition. Then it tries to load them with kexec with the proper
Grub boot params.
One step closer to the removal of the boot legacy script.
2024-03-21 10:29:13 +01:00
Alejandro Sirgo Rica 37600660f3 live: check if cache partition is available before calling tiptorrent
The image restore command must check if the cache partition is
available. Otherwise if the user forgets to create the cache
tiptorrent fails.
2024-03-21 10:29:06 +01:00