Commit Graph

41 Commits (fe40f9c5d66916cad4cee8d5b2d9e172fc1c0477)

Author SHA1 Message Date
Alejandro Sirgo Rica fe40f9c5d6 src: add POST cache/fetch method
Add API REST method to fetch an image.
Consolidate image fetch loging for cache/fetch and image/restore.

Resquest payload structure:
{
    'image': 'linux.img'
    'type': 'TIPTORRENT'
    'repository': '12.141.10.2'
}

The client will try to fetch'image' from 'repository' into cache.

Resquest response structure:
{
    'cache': [
        {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
        {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
    ]
}
2024-08-09 11:04:56 +02:00
OpenGnSys Support Team 1c9a13cd96 rest: add cmd field to POST /shell/run
echo command that has been run for storage in ogserver, until GET /shell/output
is invoked.
2024-06-21 14:59:48 +02:00
OpenGnSys Support Team 19cd1b9a78 rest: add retcode field to POST /shell/run
provide return code as result to ogserver.

Update virtual mode driver to return dummy value, although this command
is unimplemented, this seems to be broken due to possible TypeError when
accessing result from caller.
2024-06-21 14:47:30 +02:00
Alejandro Sirgo Rica 60803fe0ed src: add cache info to the image/restore response
Add a 'cache' field into the json payload the client sends to
the server after a restore operation so the server can update
the new cache contents.

Resquest response structure:
{
    ...
    'cache': [
        {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
        {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
    ]
    ...
}
2024-05-30 17:22:23 +02:00
Alejandro Sirgo Rica 72d8943576 virtual: handle copy error in image restore
Add a proper error report for the shutil.copy operation in
image_restore() instead of silently returning.
2024-05-30 17:22:23 +02:00
Alejandro Sirgo Rica 8de2b785a9 src: add POST cache/delete method
Add API REST method to delete cache contents.

Resquest payload structure:
{
    'images': ['windows.img', 'linux.img']
}

The client will try to delete as many images in cache as available
with names matching the list of filenames in the 'images' field.

Resquest response structure:
{
    'cache': [
        {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'},
        {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'}
    ]
}
2024-05-30 17:22:23 +02:00
Alejandro Sirgo Rica dfde363aa6 src: log backtrace in unhandled error cases
Log an error message in known error cases and log a backtrace
otherwise.

Define a new error type OgError to be used in all the 'raise'
blocks to define the error message to log. The exception
propagates until it reaches send_internal_server_error() where
the exception type is checked. If the type is OgError we log
the exception message. Logs the backtrace for other types.

The initial error implementation printed a backtrace everytime
an error ocurred. The next iteration changed it to only print
a backtrace in a very particular case but ended up omiting too
much information such as syntax errors or unknown error context.
The actual implementation only logs the cases we already cover in
the codebase and logs a bracktrace in the others, enabling a
better debugging experience.
2024-04-03 13:31:10 +02:00
OpenGnSys Support Team c5ccc3c7e2 qmp: incorrect copyright header
Replace incorrect copyright header in qmp.py file.
2024-04-02 13:20:27 +02:00
Alejandro Sirgo Rica 8741b2e272 src: change generic exception types to be more explicit
Replace exception types to be more explicit about the nature of
the error.
Improve the exception raising semantics by using the 'from' keyword,
this wraps an older exception into a new one so it is still considered
the same object.
2024-03-21 10:29:57 +01:00
Alejandro Sirgo Rica 657af13351 virtual: add missing check=True to subprocess.run()
Raise exception from subprocess.run() calls as previous code is
trying to capture these exceptions to handle de errors.
2024-03-21 10:29:57 +01:00
Jose M. Guisado 30fdcceea3 src: improve logging
Adds new logging handler redirecting messages to the log file
located in the Samba shared directory (applies to live mode
clients, i.e: ogLive)

Parses log level configuration from ogclient.json. See:

{
	"opengnsys": {
		...
                "log": "INFO",
		...
	}
	...
}

Adds --debug option to set root logger level to DEBUG when starting
ogClient. Overrides log level from config file.

In addition:

- Replaces any occurence of print with a corresponding logging function.
- Unsets log level for handlers, use root logger level instead.
- Default level for root logger is INFO.
- Replaces level from response log messages to debug (ogRest)
2022-06-08 10:27:06 +02:00
Jose M. Guisado 0c00f64669 #1059 virtual: replace qmp polling for event listening
Polling for a qmp port availability is undesirable, as QEMU only handles
one connection to the qmp port at a time, ogClient may interfere with
cloneer-manager.

Check vm thread now connects to a separate qmp tcp socket, listening for
a shutdown guest event.

When ogClient is run just after ogVDI installation (before guest
installation) it will try to connect until it's possible, ie: after an
iso is specified and a qemu vm is started that exposes the appropiate
qmp tcp port.
2021-09-01 13:29:58 +02:00
OpenGnSys Support Team cb9edc8d95 ogClient is AGPLv3+
Update license header in files.
2021-05-14 00:19:28 +02:00
Jose M. Guisado bd98dd1da0 #995 Add link speed in probe responses
Separates probe method into separate ogclient modes (virtual, vdi) so
future supported OS can easily have a tailored probe responses.

Link speed is retrieved using a minimal ethtool command sent using fcntl
module from python.
2021-05-04 18:30:40 +02:00
Javier Sánchez Parra 0411d2bf94 #1008 Adapt virtual disk setup JSON format
Previous commits updates ogClient and ogServer to support several disks
in Linux (ogLive) mode. This changes disk setup JSON format from an
object to an array. ogClient in virtual mode also need to satisfy this
new format, although it not support several disks.

Adapt ogClient virtual mode to satisfy new disk setup JSON format.

Old format:

    "disk_setup": {
        "disk": 1,
        "partition": 0,
        "code": "0",
        "filesystem": "",
        "os": "",
        "size": 32685957,
        "used_size": 6
    },

New format:

    "disk_setup": [
        {
            "disk": 1,
            "partition": 0,
            "code": "0",
            "filesystem": "",
            "os": "",
            "size": 32685957,
            "used_size": 6
        }
    ],
2020-11-19 13:20:08 +01:00
Roberto Hueso Gómez 3205e0f1e8 Refresh partitions.json after image restore
This is necessary for cloneer-manager to start the guest OS after the next
reboot.
2020-05-29 13:29:16 +02:00
Roberto Hueso Gómez b29b2eb452 Move check_vm_state_loop() into OgVirtualOperations
Improves code encapsulation by moving check_vm_state_loop method into
OgVirtualOperations class. This also fixes import error when running ogclient in
'linux' mode.
2020-05-26 11:24:26 +02:00
Roberto Hueso Gómez 2ab5db74fe Handle other exceptions on virtual /refresh
Ths patch handles different cases when a /refresh request can be made. More
specifically this handles the situation in which the VM is stopped when /refresh
is called but it starts during the processing of /refresh, in this case we just
send the last recorded setup.
2020-05-19 12:40:35 +02:00
Roberto Hueso Gómez de2ce69e46 Refactor OgQMP and adapt operations
This patch:
- Fixes logic errors in the communication with QMP (the order of handshake
  messages was not right).
- Rewrite parts of OgQMP class.
- Enforces better coding practices by using Python's "context managers" to avoid
  forgeting an open socket in case exceptions occur.
- Adapt virtual operations to the use of "context managers" using the "with"
  statement.
2020-05-19 12:34:31 +02:00
Roberto Hueso Gómez 28a25918b2 Update filesystems on /refresh
This updates partitions.json filesystems.
2020-05-18 15:58:22 +02:00
Roberto Hueso Gómez 2e6b73da53 Use virtio-vga to run VMs
This is the device that has the best empirical performance right now and it
seems like it is where most of the qemu development is directed towards for the
future.
2020-05-18 10:28:18 +02:00
Roberto Hueso Gómez 269c7b5055 Rename operation 'execCMD' to 'shellrun'
This patch also ignores calls to shellrun when virtual mode is activated.
2020-05-13 17:30:12 +02:00
Roberto Hueso Gómez b63bd727f9 Change permissions on virtual images for /refresh
Write permission is not needed to get information about each virtual drive so
this patch changes permissions to 'only read'. This can prevent race conditions.
2020-05-13 14:15:44 +02:00
Roberto Hueso Gómez ff988b80b5 Add VNC support for virtual mode VMs
This patch makes possible to interact with guest OS from a remote machine using
VNC.
2020-05-13 14:13:47 +02:00
Roberto Hueso Gómez 404b8c79d0 Extend use of OG_PARTITIONS_CFG_PATH for /setup and /refresh
This is a refactor to consolidate the use of this single variable across all
virtual operations.
2020-05-11 12:13:39 +02:00
Roberto Hueso Gómez 6ca16dd200 Poweroff when no VM and no jobs are running
This patch calls poweroff in virtual mode when no VM is running and no jobs are
being executed. This is useful when the guest OS shutdowns so that the host OS
does not continue to run.
2020-05-11 11:12:11 +02:00
Roberto Hueso Gómez 7bde2a04e9 Add OgVM class to abstract VMs
This patch implements OgVM class. OgVM abstracts VM emulator from its
operations.
2020-04-28 16:40:30 +02:00
Roberto Hueso Gómez c86328181f Poweroff VM before some operations
This patch includes changes to:
- Poweroff VM before running operations that require access to virtual disks.
- Poweroff VM before host system poweroff.
2020-04-28 11:32:05 +02:00
Roberto Hueso Gómez 298e156317 Send last stored setup if VM is running
When virtual mode is activated, send the last stored partitions setup because
it is not possible to access a running virtual drive to get its information.
2020-04-28 11:27:16 +02:00
Roberto Hueso Gómez deb2e075cd Add OgQMP recv method
recv method is useful for receiving information that was not previously
requested (such as "events"). This patch also implements automatic handshake
on OgQMP by sending an "qmp_capabilities" request.
2020-04-28 11:21:49 +02:00
Roberto Hueso Gómez 5444e453ee Fix disk used size calculation
This patch calculates correctly the percentage of disk used. This could cause
an overflow on the ogAdmSever DB.
2020-04-28 11:16:59 +02:00
Roberto Hueso Gómez 1a83ebba23 Control errors in OgQMP for virtual mode
This patch handles possible communication errors between ogclient and Qemu when
making QMP requests.
2020-04-22 13:02:55 +02:00
Roberto Hueso Gómez 2e3d47b7b8 Avoid writting /software output to a file 2020-04-17 16:59:48 +02:00
Roberto Hueso Gómez 7ccc498014 Use samba for create and restore virtual partitions
This requires to configure user and password for samba repositories.
2020-04-17 15:50:35 +02:00
Roberto Hueso Gómez dfb69e9dd5 Use libguestfs for virtual setup 2020-04-17 15:49:21 +02:00
Roberto Hueso Gómez 84e02462c0 Use libguestfs for virtual refresh 2020-04-17 15:49:21 +02:00
Roberto Hueso Gómez 8ec8572da2 Transform absolute paths into relative paths for virtual mode 2020-04-17 15:49:21 +02:00
Roberto Hueso Gomez 0ddf3f287a Add placeholder for virtual execCMD 2020-04-13 12:38:45 +02:00
Roberto Hueso Gomez 7f646fc468 Use 75% of disk space in virtual mode 2020-04-08 13:35:34 +02:00
Roberto Hueso Gomez 99ae598fbd Encapsulate operations in classes 2020-04-08 13:30:16 +02:00
Roberto Hueso Gomez c279325919 Add virtual operations 2020-04-07 13:55:09 +02:00