Commit Graph

150 Commits (a1bd0c36f3be137e908540e5ea50354fd9293ca3)

Author SHA1 Message Date
OpenGnSys Support Team 90a9ba9543 live: bogus error in logs when connecting to ogserver
This error is bogus:

 (2024-11-14 09:05:37) ogClient: [ERROR] - Partition query error for /dev/sdb: No medium found

skip if device cannot be opened instead.
2024-11-14 18:12:56 +01:00
Alejandro Sirgo Rica a440b9da47 live: fix error report in partition and format
Fix error codepath and properly mark the operation as failed.
2024-11-05 11:20:59 +01:00
OpenGnSys Support Team 7874323d46 live: use reboot --force --force to make a hard reboot in new live
this is a workaround for the new live system to make a hardware reboot
that allows UEFI to pick up, otherwise it performs a kernel reboot instead.
2024-10-24 17:46:37 +02:00
Alejandro Sirgo Rica 569caa733f cache: add code to cache the oglive boot files
Add update_live_cache() implementing the legacy script
updateBootCache()

Copy the ogvmlinuz and oginitrd.img files into cache after
a partition and format command with an available cache partition.
2024-10-24 10:52:33 +02:00
OpenGnSys Support Team fc21cd988f live: always enable wol when launching ogclient
no need to wait for reboot/poweroff to enable wake-on-lan.
2024-10-24 10:46:44 +02:00
Alejandro Sirgo Rica e8ddbbd075 src: isolate libfdisk operations to enable mount operations
python-libfdisk does not close file descriptor until the cxt
object goes out of scope.

Define get_partition_data and get_disk_data functions to isolate
the python-libfdisk logic and return the data as an object.

Improve error handling of libfdisk operaions in refresh.
2024-10-08 10:07:45 +02:00
Alejandro Sirgo Rica 97c836e0e4 live: improve part not found log in image_create
Report the partition number of the missing partition.
2024-10-08 10:07:45 +02:00
Alejandro Sirgo Rica f942b19eae src: fix cache mount in new ogLive
Mount cache in image_create() image_restore() and cache_fetch().

Remove init_cache() and ensure /opt/opengnsys/images/ exists
within the cache mountpoint if it does not exists in cache_mount().
2024-10-08 10:07:45 +02:00
OpenGnSys Support Team fe960010a0 live: remove useless check in compute_md5
this never returns -1 on error, an exception is rised instead
2024-10-05 08:40:55 +02:00
Alejandro Sirgo Rica 0f167cf29f src: consolidate compute_md5 functions
Add compute_md5 function in src/utils/fs.py

Remove identical md5 functions from src/live/ogOperations.py and
src/utils/tiptorrent.py

Move error checks from ogOperations.py into compute_md5 function in
src/utils/fs.py
2024-10-02 14:50:14 +02:00
Alejandro Sirgo Rica cc70274079 src: check if the system is hibernated before /image/create
Mount the system partition in readonly mode and check for the
hiberfil.sys file if the target system is a Windows.

Fail the image creation process if the target system is hibernated.
2024-10-02 14:25:04 +02:00
OpenGnSys Support Team f1b1532f78 live: refactor poweroff and reboot
- enable WoL
- check for poweroff and reboot before checking for busybox.
- call new shutdown method
2024-10-01 22:12:17 +02:00
OpenGnSys Support Team cf9e1c96fd live: fix EBUSY error in newer kernels with mkfs
python-libfdisk does not close file descriptor to /dev/sda after completing
partitioning. This results EBUSY errors when formatting partitions with mkfs
in newer kernels. Encapsulate code to partition in method so python garbage
collection knows ctx objects can be release then close file descritor to
/dev/sda.

ogRest is not accessible from _partition(), remove check to ogRest.terminated,
actually no need to terminate inmediately when formatting is ongoing, better
leave things in consistent state when stop command is received.
2024-10-01 17:02:49 +02:00
Alejandro Sirgo Rica f0953c969a live: report correct partition size unit in setup() log
Log the correct partition size unit. Use MiB instead of MB.
2024-10-01 16:49:58 +02:00
Alejandro Sirgo Rica e30e934272 src: replace DEVICE env variable with get_ethernet_interface()
Use a python function to obtain the main net interface. Detect
the first ethernet inferface in use.
Stop using the DEVICE environment variable.
2024-10-01 15:32:54 +02:00
OpenGnSys Support Team 8f437bb954 live: remove unused probe
unused since ogserver's commit 87be2ce08 #980 Change initial probe to refresh
2024-09-30 14:42:29 +02:00
OpenGnSys Support Team 9c18ef7392 live: use busybox otherwise poweroff/reboot commands 2024-09-30 14:42:29 +02:00
Alejandro Sirgo Rica 9c2ae873f2 live: disable browser operations if the binary is missing
Add shutils.which checks before every browser operations to
skip the subprocess call if the binary is missing.
2024-09-27 14:27:00 +02:00
Alejandro Sirgo Rica 8754c21694 src: report used and free partition data in bytes
Add "used_size" and "free_size" to the partition data and the
cache data.

Old response from ogClient for /cache/delete, /cache/fetch
and /image/restore:
{
  'cache': [
    {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
    {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
  ]
}

New response:
{
  'cache': {
    'used_size': 4520232322423,
    'free_size': 48273465287452945,
    'images': [
      {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
      {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
    ]
  }
}
2024-09-25 14:35:41 +02:00
Alejandro Sirgo Rica 51258613cc src: verify the fields of the efibootmgr json in /refresh
Don't send the efi data in the /refresh payload if efibootmgr
is missing any of the json keys.

Log the missing keys in case of missing some.
2024-09-25 14:35:41 +02:00
Alejandro Sirgo Rica 595770163a live: add boot entries into /refresh payload
Add 'efi' key into the refresh payload. The value for that key
has the following structure:

'efi': {
  'entries': [
    {
      "order": 0,
      "name": "Boot0000",
      "active": false,
      "description": "grub"
    },
    {
      "order": 1,
      "name": "Boot0001",
      "active": true,
      "description": "UEFI: PXE IP4 Realtek PCIe GBE Family Controller"
    }
  ]
}

If the client is not a EFI system it won't add the 'efi' field.
If an entry is not in the boot order it won't have the 'order' field.
2024-09-10 17:23:18 +02:00
OpenGnSys Support Team 084650e4b1 live: use correct variable name in restore unicast cache
fe40f9c5 ('src: add POST cache/fetch method') broke unicast cache restore.

(2024-09-09 10:05:22) ogClient: [ERROR] - name 'image_name' is not defined
Traceback (most recent call last):
File "/opt/opengnsys/ogClient/src/ogRest.py", line 175, in image_restore
payload = ogRest.operations.image_restore(request, ogRest)
File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 520, in image_restore
self._restore_image_unicast(repo, name, partdev, cache)
File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 242, in _restore_image_unicast
image_path = f'{OG_CACHE_IMAGE_PATH}{image_name}.img'
NameError: name 'image_name' is not defined
2024-09-09 12:23:00 +02:00
OpenGnSys Support Team 05b7a576b0 live: incorrect indentation
f2a2f53074 ('live: remove file from cache with no checksum file') broke
indentation which breaks ogClient.
2024-09-03 19:44:43 +02:00
Alejandro Sirgo Rica 6704abc620 live: add disk index bounds check
Add disk index bounds checks for setup() and image_create().
Prevent backtrace logging when an invalid disk index is used.
2024-09-02 14:25:54 +02:00
OpenGnSys Support Team f2a2f53074 live: remove file from cache with no checksum file
Maybe result of a partial download? then, remove it to leave cache in consistent
state.
2024-09-02 13:59:36 +02:00
OpenGnSys Support Team 9270a6c58f src: append .img to log that report image fetching 2024-08-29 13:10:50 +02:00
OpenGnSys Support Team 3d47b5069a src: use logging.warning()
logging.warn() is deprecated since 3.3.

And use .error() instead when command is unsupported or client is busy, that
should not ever happen.
2024-08-27 12:44:13 +02:00
OpenGnSys Support Team 8839950758 live: report that up-to-date image already exists in cache 2024-08-21 09:28:01 +02:00
OpenGnSys Support Team 1ca3992f71 src: remove call to restoreImageCustom script
remove call to custom version of legacy script, display warning.
2024-08-21 09:21:53 +02:00
Alejandro Sirgo Rica 9fa4a5f651 live: clean files after a failed cache image copy
Remove image files after a failed image copy operation.
2024-08-09 11:05:02 +02:00
Alejandro Sirgo Rica fe40f9c5d6 src: add POST cache/fetch method
Add API REST method to fetch an image.
Consolidate image fetch loging for cache/fetch and image/restore.

Resquest payload structure:
{
    'image': 'linux.img'
    'type': 'TIPTORRENT'
    'repository': '12.141.10.2'
}

The client will try to fetch'image' from 'repository' into cache.

Resquest response structure:
{
    'cache': [
        {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
        {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
    ]
}
2024-08-09 11:04:56 +02:00
OpenGnSys Support Team 9ee5f4adaa live: move filesystem expansion out of OS configuration
just a clean up.
2024-08-06 18:39:37 +02:00
OpenGnSys Support Team 1011957bb7 live: use correct loglevel when reporting issues with listing cache contents
instead of using info to report errors.
2024-07-31 22:29:29 +02:00
OpenGnSys Support Team 1329c0955b live: validate checksum only once if image is already in cache
if image already exists in the cache, skip a second checksum validation.

log shows duplicated entries:

	Verifying checksum for example.img, please wait...
	Checksum is OK for example.img
	Verifying checksum for example.img, please wait...
	Checksum is OK for example.img

because tip_check_csum() is called twice in this case.
2024-07-30 16:19:03 +02:00
OpenGnSys Support Team 1376b1900d live: remove old image and checksum file on download
Restoring an updated image file (with different checksum) could fail while the
old checksum file could remain in place.

Remove image and checksum file before fetching the new files.
2024-07-30 16:11:43 +02:00
OpenGnSys Support Team a846d5e343 live: unicast-direct does not validate checksum
this restore method is unreliable, add warning to the logs.
2024-07-30 00:51:37 +02:00
OpenGnSys Support Team 8395982244 live: validate file existence and checksum for unicast cache
after copying file to cache, validate that file exists in cache and checksum
correct.
2024-07-30 00:51:37 +02:00
Alejandro Sirgo Rica 567fea276e utils: add postinstall.py
Replace ConfigureOs script by native calls to:

- update BCD via hivex using bcd.py and winreg.py infrastructure.
- restore efi bootloader restore_windows_efi_bootloader().

Call legacy scripts for remaining postinstall procedures to
replace them incrementally.

Define variable CONFIGUREOS_LEGACY_ENABLED as False by default.
Run legacy configureOs when CONFIGUREOS_LEGACY_ENABLED = True.
This serves as a auxiliar method to keep the restoration functional
in case of problems with the new configure_os logic.
2024-07-29 15:07:22 +02:00
Alejandro Sirgo Rica e20cda122b utils: rename ogExtendFs to extend_filesystem
Use a more consistent name format for the ogExtendFs function.
2024-07-29 15:07:22 +02:00
OpenGnSys Support Team 2c10b4c92a utils: fs: error out if mkfs fails
If formatting fails, log shows:

	mkfs.ntfs reports return code 1 for /dev/sda2

but ogclient reports success to the ogserver.

Raise an exception so front-end gets an indication that formatting has failed.
2024-07-29 14:22:48 +02:00
OpenGnSys Support Team 59fa3bb120 live: improve logging for file removal from cache
Provide a bit more logging to make it easier to debug issues.
2024-07-21 22:14:07 +02:00
OpenGnSys Support Team fcfa5f9fbc live: add logging to indicate image file copy from samba
For consistency with tiptorrent download.
2024-07-21 21:59:27 +02:00
OpenGnSys Support Team 9d8a95cc74 live: add checksum field to image/create response
Report image checksum to ogserver through HTTP response.
2024-07-15 09:50:30 +02:00
OpenGnSys Support Team 43039749c5 live: check permissions when trying to fetch file via unicast
check sufficient permissions and bail out in case of issues.
2024-06-27 12:16:14 +02:00
OpenGnSys Support Team a9d8cdd210 live: revisit error log when failing to validate checksum
Add explicit check for .full.sum after downloading it.

Rewrite errors log, one of them is misleading when checksum validation
fails, it refers to missing .full.sum, but it could be a different
reason.
2024-06-27 11:46:30 +02:00
Alejandro Sirgo Rica 6282cb41a8 live: add restricted execution mode to shell/run
Try to find the script to run for a shell/run request in
/opt/opengnsys/shell/, restricted mode is enabled if the
script is found.

Excute the script without shell=True and executable=OG_SHELL in
restricted mode.

Restricted mode is a safer execution method as it only executes
code manually defined by the administrator.

Each script needs to define a shebang, this way more than just bash
is supported.
2024-06-25 13:41:41 +02:00
OpenGnSys Support Team 1c9a13cd96 rest: add cmd field to POST /shell/run
echo command that has been run for storage in ogserver, until GET /shell/output
is invoked.
2024-06-21 14:59:48 +02:00
OpenGnSys Support Team 19cd1b9a78 rest: add retcode field to POST /shell/run
provide return code as result to ogserver.

Update virtual mode driver to return dummy value, although this command
is unimplemented, this seems to be broken due to possible TypeError when
accessing result from caller.
2024-06-21 14:47:30 +02:00
Alejandro Sirgo Rica 399a5dceb8 live: handle no cache in tiptorrent and unicast restore
Properly report the lack of cache partition when restoring an
image using TIPTORRENT and UNICAST as methods.
Abort any restore in case of no cache partition.
2024-06-07 10:09:50 +02:00
Alejandro Sirgo Rica 5698aa66d2 utils: fix get_image_info regression
Fix image size, permissions and creation time.
Improve error report related to these parameters now showing the
exact cause of the problem if any occurred during the definition
of image size, file permissions or image creation time values.
2024-06-03 11:29:57 +02:00