Use the constant OG_CACHE_IMAGE_PATH from cache.py to obtain the
location of the directory where images are stored.
This way the path can be changed from one single point.
Add a 'cache' field into the json payload the client sends to
the server after a restore operation so the server can update
the new cache contents.
Resquest response structure:
{
...
'cache': [
{'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
{'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
]
...
}
Add API REST method to delete cache contents.
Resquest payload structure:
{
'images': ['windows.img', 'linux.img']
}
The client will try to delete as many images in cache as available
with names matching the list of filenames in the 'images' field.
Resquest response structure:
{
'cache': [
{'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
{'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
]
}
Add the list of images in the client's cache partition in the
payload sent to the server.
The information sent is a list of {image_name, img_size, checksum}
elements where img_size is the size of the respective image in bytes.
Resquest response structure:
{
...
'cache': [
{'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
{'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
]
...
}
This method reports the /dev path to cache partition, rename it.
Add explicit check if blkid is successful.
And add logging to report that device path to cache is not found.
ogCP expects LINUX-SWAP to specify a swap filesystem.
Add a similar workaround to the one that is done for VFAT for symmetry between
inputs and outputs that circulate over the API.
partprobe requires that all disk partitions are unmounted.
partprobe needs to be called to report the OS that the partition table
has changed, otherwise ogclient reports incorrect partition information.
iterate over the partition list and mount cache after partprobe is
called.
tip_client_get raises the proper error exceptions but the except
block in _restore_image_tiptorrent overwrites the reported error.
Move the raise statements in _restore_image_tiptorrent outside
of the except block.
Reduce the scope of the try except block that controls the case
of deleting the image backup in case of error. Now it only covers
the section of code after backup creation and up to image
verification. Check when the Exception is an OgError to raise
with added context.
Prevent the deletion of the target image in case of error before
the backup creation.
Bundle the backup creation on its own try except block to give
more feedback on a failed backup creation.
Enables a better error management allowing unhandled
exceptions to be reported properly.
Capture only the relevant exception types in each except block.
The capture of the Exception type means hiding information for
unhandled error cases, even for syntax errors in the codebase.
Using a more fine grained exception filtering improves error
traceability.
Log an error message in known error cases and log a backtrace
otherwise.
Define a new error type OgError to be used in all the 'raise'
blocks to define the error message to log. The exception
propagates until it reaches send_internal_server_error() where
the exception type is checked. If the type is OgError we log
the exception message. Logs the backtrace for other types.
The initial error implementation printed a backtrace everytime
an error ocurred. The next iteration changed it to only print
a backtrace in a very particular case but ended up omiting too
much information such as syntax errors or unknown error context.
The actual implementation only logs the cases we already cover in
the codebase and logs a bracktrace in the others, enabling a
better debugging experience.
This reverts commit 57787dab54.
Read from stderr is blocking if no data is available, revert this patch since
ogClient hangs indefinitely in lzop invocations due to races in process
execution through Popen.
Restore image file from .ant to original file name if new image
creation fails. Remove new imagen and move the .ant image file in
place of the original as previously an error meant a rename of the
image file without a revert to keep the image available.
Implement a Python equivalent of ogCopyEfiBootLoader as the
function copy_efi_bootloader. This function copies the contents of
the folder of the EFI loader in the ESP into a ogBoot folder at
the root of the partition target of an image creation.
copy_efi_bootloader is a Windows only functionality.
Control non 0 returncode of the lzop and partclone subprocess
in image creation and restoration because this means that either
lzop or partclone has failed.
The implementation must cover cases such as not enough storage
space and log errors into /tmp/command.log and the log file of
the client handling the request.
Check the returncode of lzop and partclone subprocesses and
log the stderr of the process reporting non zero returncode.
Log the whole context of the error when an exception happens.
The previous exception handling was hidding important information
about the cause of the error.
Provide more information in exception messages as those are the
source of the logging messages. Add information about paths, files
or configuration related to the operation associated to the
exception.
Replace exception types to be more explicit about the nature of
the error.
Improve the exception raising semantics by using the 'from' keyword,
this wraps an older exception into a new one so it is still considered
the same object.
Use only the exception messages as the main resource for error
messages.
The previous error code had string duplication in the form of:
logging.error('msg here')
raise Exception('msg here')
That approach also has the downside of having log duplication as
it had the local logging.err() and a global logging.exception()
inside send_internal_server_error capturing the exception message.
The actual code only requires raising an exception with a proper
error message.
Improve exception messages to give more error context.
Log every AssertionError as a backtrace.
Use the 'raise Exception from e' syntax to modify the a previously
raised exception 'e' into an exception with aditional context or
different type. This also prevents the message that warns about
newer exceptions being launch after an initial exception.
Create ogboot.me and ogboot.secondboot as empty files and
ogboot.firstboot with the value "iniciado" in the root of
the BIOS Windows system partition.
The files must contain data for GRUB to be able to write content,
therefore these are created containing 3072 null bytes.
The Windows boot process is handled by the "pxe" profile.
There the files ogboot.me, ogboot.firstboot and ogboot.secondboot
are used as a state machine to chose between booting Windows and
ogLive.
The first Windows boot happens if ogboot.me and ogboot.firstboot
are identical, then "iniciado" is written in ogboot.firstboot.
We skip this stage as we create ogboot.firstboot with 'iniciado'.
The second Windows boot occurs if ogboot.me and ogboot.secondboot
are boot identical, then "iniciado" is written in ogboot.secondboot.
After the Windows boot ogLive is booted.
This change is a preparative for reimplementing the BIOS boot
in order to deprecate the legacy script. All the codepaths to
boot systems located at a partition are now called from the
boot_os_at function enabling an easier structure for the incoming
code.
Checking the existence /sys/firmware/efi as it might appear
sometimes in BIOS installs if the BIOS configuration is not
proper. Checking for the EFI partition is the safest method
to veryfy the install type.
Replace IniciarSesion script in favor of native Python code when booting
a UEFI system. This applies when running the "session" command.
WIP: Only UEFI boots Windows systems. Raise NotImplementedError
exception trying to boot a Linux system using UEFI.
do not return the returncode, instead return an integer.
do not use
except CalledProcessError as e:
it causes a another exception while handling exception.
Remount the original image repository.
it should be possible to simplify this further by:
- stacking mounts, no need to umount initial repo and mount it again
when switching to the new repo, because remount back initial repo
might fail (!)
- use check=False and simply check for x.returncode
Remove mbuffer, this is never used.
mbuffer has been never been used since ogClient supports native image restore.
Originally this was used like this:
partclone [...] | mbuffer -q -M 40M | lzop [...]
supposely to speed up partclone in case the device where the read happens is
slowier than the device that is used for writes.
See mbuffer(1) manpage examples.
In any case, this needs benchmarking to really make sure this is helping.
Remove it until that ever happens.
cover more error cases where exceptions need to be raised.
check return code in the invoked subprocess.
restoreImageCustom has been intentionally left behind, it
is unclear what this custom script returns on success and
error.
Alejandro Sirgo Rica
OpenGnSys Support Team
Jose M. Guisado