Reuse os_probe() function from probe.py and change the fallback
case to "unknown" to prevent OS id mismatch in ogserver.
The OS id mismatch causes the images to stop being associated with
partitions after the /refresh command.
Add python implementation of the legacy ogWindowsRegisterPartition
function.
This function configures the system partition to be considered the
new C drive after a Windows image restore.
The drive letter configuration is stored in the SYSTEM hive of
the windows registry. The node MountedDevices contains all the
configuration as key-value pairs.
The C drive key \DosDevices\C: contains a different value based on
the partitioning type.
GPT
the value is DMIO:ID: followed by the 16 byte GUID of the
partition.
Example: DMIO:ID:\xc9\xfc\x1c\x86\x13\x11O\t\xa2\x9c{/\xf1\xdf\xe4)
MBR
The value is a little endian byte sequence with 2 parts.
The first 4 bytes correspond to the disk id. The second part is
the byte offset start of the partition in that disk.
Example: \xe1\\\x9cP\x00\x00\x10j\x18\x00\x00\x00
If we format the MBR value in a more readable way we get
e1 5c 9c 50 00 00 10 6a 18 00 00 00
In this case the disk ID is 509c5ce1.
The partition offset is 186a100000.
This patch adds the following helper functions to:
- get_disk_id_bytes(): to obtain the disk identifier.
- get_part_id_bytes(): to obtain a partition identifier as UUID
for MBR or DMIO:ID format for GPT.
- get_sector_size(): to query the sector size of a specific disk.
Read /sys/class/block/{device_name}/queue/hw_sector_size to obtain the value.
This is MBR specific.
- get_partition_start_offset(): to query the start sector of a specific
partition and disk. Use sfdisk with the -J argument to get fdisk data in
json format. This is MBR specific.
Add 'efi' key into the refresh payload. The value for that key
has the following structure:
'efi': {
'entries': [
{
"order": 0,
"name": "Boot0000",
"active": false,
"description": "grub"
},
{
"order": 1,
"name": "Boot0001",
"active": true,
"description": "UEFI: PXE IP4 Realtek PCIe GBE Family Controller"
}
]
}
If the client is not a EFI system it won't add the 'efi' field.
If an entry is not in the boot order it won't have the 'order' field.
fe40f9c5 ('src: add POST cache/fetch method') broke unicast cache restore.
(2024-09-09 10:05:22) ogClient: [ERROR] - name 'image_name' is not defined
Traceback (most recent call last):
File "/opt/opengnsys/ogClient/src/ogRest.py", line 175, in image_restore
payload = ogRest.operations.image_restore(request, ogRest)
File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 520, in image_restore
self._restore_image_unicast(repo, name, partdev, cache)
File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 242, in _restore_image_unicast
image_path = f'{OG_CACHE_IMAGE_PATH}{image_name}.img'
NameError: name 'image_name' is not defined
Implement configure_fstab() as a replacement of ogConfigureFstab.
Create src/utils/fstab.py to implement the main fstab
configuration functions. Define two fstab helper classes,
FstabBuilder and FstabEntry.
FstabEntry
Represents each line in the fstab file.
Has the values: device, mountpoint, fstype, options, dump_code
and pass_code.
FstabBuilder
Contains a list of FstabEntry. Handles loading of a
preexisting fstab file and the serialization of multiple
FstabEntry into a file.
The fstab configuration has 3 main steps:
Root partition:
- Update the device field with the device where
the new system is installed.
Swap partition:
- Preserve all the swapfile entries in every case.
- If the filesystem has a swap partition: update the device field
in the first fstab swap entry and remove the rest swap entries
pointing to a swap partition. Only one swap partition is supported.
Create a new fstab entry if no preexisting swap entry exists.
- If the system has no swap partition remove every swap partition
entry.
EFI partition:
- Update the device field of the EFI fstab entry if it exists.
Create a new fstab entry if no preexisting EFI entry exists.
Add get_filesystem_id to disk.py to obtain the UUID.
Define every device field as a UUID. That method is more robust
than a plain device path as it works after disks being added or
removed.
instead of
Checking partition "Microsoft Basic Data"...
Checking partition "EFI System"...
show
Searching EFI partition...
EFI partition found at /dev/sda1
and refer to EFI partition consistently in logs.
if image already exists in the cache, skip a second checksum validation.
log shows duplicated entries:
Verifying checksum for example.img, please wait...
Checksum is OK for example.img
Verifying checksum for example.img, please wait...
Checksum is OK for example.img
because tip_check_csum() is called twice in this case.
Restoring an updated image file (with different checksum) could fail while the
old checksum file could remain in place.
Remove image and checksum file before fetching the new files.
.full.sum file in the local cache contains the local checksum for this file,
this checksum is calculated by tip_write_csum(), therefore, there is no need to
calculate this checksum again from the image file in the cache, use this
checksum content instead.
if .full.sum is not available, then cache is inconsistent, tip_check_csum()
is called after checking if image file exists, raise an exception.
checksum could mismatch in two situations:
a) new image version (checksum is different)
b) image is corrupted
in both cases, a new fresh image needs to be retrieved.
this speeds up checksum validation.
Add function to redefine the hostname of a Windows install.
Windows hostnames can't be larger than 15 characters due to
legacy heritage.
Hostname modification is done by modifying 3 registry values in
the ControlSetXXX entry of the SYSTEM hive. ControlSet001 is
generally the entry to be edited but one must query the value
of the 'Current' key in the 'Select' entry of the SYSTEM hive
to retrieve the active ControlSet.
The hostname has to be introduced in the following entries:
path = 'ControlSetXX/Control/ComputerName/ComputerName'
key = 'ComputerName'
path = 'ControlSetXXX/Services/Tcpip/Parameters'
key = 'HostName'
key = 'NV Hostname'
The value to store in those keys is of tipe SZ and has to be
encoded in UCS-2 Little Endian (utf-16-le).
Replace ConfigureOs script by native calls to:
- update BCD via hivex using bcd.py and winreg.py infrastructure.
- restore efi bootloader restore_windows_efi_bootloader().
Call legacy scripts for remaining postinstall procedures to
replace them incrementally.
Define variable CONFIGUREOS_LEGACY_ENABLED as False by default.
Run legacy configureOs when CONFIGUREOS_LEGACY_ENABLED = True.
This serves as a auxiliar method to keep the restoration functional
in case of problems with the new configure_os logic.
Implement update_bcd() as replacement of ogWindowsBootParameters.
The legacy function modified the BCD of a freshly restored system
invoking the privative binary "spartlnx".
The script edits a set of BCD entries needed for a proper system
boot.
Each main BCD entry is identified by an UUID and contain a set of
subnodes, these hold the configuration and entry information.
Each node contains data in the form of key-value.
Common BCD structure:
'Objects'
...
The Boot Manager entry is always identified by the UUID
9dea862c-5cdd-4e70-acc1-f32b344d4795.
Some entries always have the same UUID as identifier such the
Boot Manager while other have different UUID depending on the system.
To identify these entries with a not known UUID we query the value
'Type' of the node 'Description'. This contains a 32 bit value whose
bytes codify the nature of the entry.
We obtain 3 different values as a tuple, each value is the result of
applying a bitmask to the Type value. These masks are 0xf0000000,
0x00f00000 and 0x000fffff. The resulting tuple of 3 values is then
used to obtain the corresponding entry from a map.
The data we modify from the BCD are disk and partition references
to point to the new disk and partition in the system where the
images are restored. Partition and disk information is stored as
UUID in specific offsets inside binary data in the BCD.
To update these we need to obtain the disk and partition UUIDs,
then convert it to bytes as follows:
original UUID: C4C61C51-3456-4733-96AD-AE760A41C392
UUID as bytes: 51 1C C6 C4 56 34 33 47 96 AD AE 76 0A 41 C3 92
The modified entries are: Resume from Hibernation, Windows Boot
Loader OS, Windows Boot Loader Recovery, Windows Recovery,
Boot Loader Settings, Windows Boot Manager and Windows Memory
Diagnostic.
Some of these options could be omited as the system restoration
does not include a recovery partition so in this case all the
recovery related entries just point to the main system partition.
Most entries are edited modifying the value corresponding to the
key 'Element' in 2 subnodes of the 'Elements' node.
These subnodes are '11000001' and '21000001'.
'Objects'
The 2 values stored in these 2 entries is binary data where we
store the partition and disk bytes. We simply replace the byte
representation of out UUIDS in the binary data. Partition is
stored in offset 32 and the disk in offset 56.
The exception is the Bootloader Recovery entry, in which the
partition offset is 84 and the disk offset 108.
Note that the legacy function only does a proper BCD edit in UEFI
systems. The new implementation follows the same behavior with
the possibility of implementing BCD modification under MBR
partitions in the future.
Set the field 16000009 (RECOVERY_ENABLED) to the value x00 to
disable recovery in both Resume from Hibernation and Windows
Boot Loader OS entries in the BCD.
The system install does not include any recovery partition so
it makes no sense to have it enabled.
This commit is preparatory work for the new native postinstall code.
Add winreg.py to the utils folder. Implement hive enum types,
hive handler validation and validated get_* functions for
nodes and registry values.
Implement the utility hive operations through the Hivex library.
This serves as preparatory work for BCD manipulation but it also
has potential to improve registry usage in previous code.
UCS-2 Little Endian is the prefered windows registry text encoding
for binary content. Define a WINDOWS_HIVE_ENCODING global variable
to use when encoding string to write in the win registry.
This commit is preparatory work for the new native postinstall code.
Add restore_windows_efi_bootloader to copy the EFI loader from
the filesystem in the restored system into the EFI partition.
This commit is preparatory work for the new native postinstall code.
Add get_partition_id and get_disk_id to obtain the UUID of a disk
or partition as a string. These values are obtained invoking the
program blkid as a subprocess.
This commit is preparatory work for the new native postinstall code.
If formatting fails, log shows:
mkfs.ntfs reports return code 1 for /dev/sda2
but ogclient reports success to the ogserver.
Raise an exception so front-end gets an indication that formatting has failed.
Alejandro Sirgo Rica
OpenGnSys Support Team