ogClient can run in "linux" mode. In addition to live or virtual.
Serves as a substitute to the legacy ogagent, which has not received any
updates since 2020/07/23.
Linux mode initially supports remote reboot and poweroff. Requires
updated ogServer with the Linux ogclient state.
ogClient can be set up to run in linux mode by specifying it in
ogclient.json:
{
"opengnsys": {
"ip": "192.168.56.10",
"port": 8889,
"log": "DEBUG",
"mode": "linux",
...
}
Abort command if the connection with the server is lost.
Otherwise, a race condition that leaves ogServer and ogClient
out-of-sync might occur:
1. ogClient is busy running a command (on the worker thread), for example,
image/restore.
2. ogServer is stopped OR ogClient loses connection with ogServer (due to
transient network problem).
3. ogClient reconnects and ogServer sends a refresh command.
4. ogClient worker thread finishes and it sends a reply to image/restore.
5. ogServer gets confused because it expects a reply to the refresh
command, not the old image/restore.
Polling for a qmp port availability is undesirable, as QEMU only handles
one connection to the qmp port at a time, ogClient may interfere with
cloneer-manager.
Check vm thread now connects to a separate qmp tcp socket, listening for
a shutdown guest event.
When ogClient is run just after ogVDI installation (before guest
installation) it will try to connect until it's possible, ie: after an
iso is specified and a qemu vm is started that exposes the appropiate
qmp tcp port.
If server closes the connection, close the socket, reconnect and stop
processing. self.connect() already cleans up the internal state, including the
socket state.
OgClient miscalculates the body size of the request.
ogServer delimits HTTP headers with "\r\n\r\n" to comply with RFC 2616.
But ogClient searches for the first "\r\n" delimiter, hence, ogClient
stops at the first HTTP header field instead of at the end of the header.
Hence, it incorrectly assumes the body starts after the first "\r\n".
This commit updates ogClient to search for the "\r\n\r\n" delimiter.
Example:
POST /shell/run HTTP/1.1\r\n <-- ogClient considers body starts here (WRONG!)
Content-Length: 952\r\n
Content-Type: application/json\r\n
\r\n <-- Here is where the body starts
{"json-body":...}
Add ogClient support to receive, parse and send disk type data from the cloning
engine when refreshing disks configuration.
See also commits with #1037 in ogServer and WebConsole repo.
Separates probe method into separate ogclient modes (virtual, vdi) so
future supported OS can easily have a tailored probe responses.
Link speed is retrieved using a minimal ethtool command sent using fcntl
module from python.
When running image_create operation an underlying script "imageCreate"
is called using the subprocess python module. This script may fail, for
example if the repository in which the image is to be stored has no
sufficent capacity.
Check for non-zero when the imageCreate process has finished.
ogLive related operations are named inside a 'Linux' folder, also its
python class is named OgLinuxOperations. Rename every 'linux' occurrence
with live to further clarify this folder and operations.
- OgLinuxOperations -> OgLiveOperations
- src/linux/ -> src/live/
Fixes: 1377acee ('Rename 'linux' mode to 'live' mode')
ogClient /software gets a representation of the target resource’s state.
GET method is more appropriate than POST.
Change /software method from POST to GET.
Before this patch the ogRest would hang indifinitely in a BUSY state
when a bad request was received. Fix this by returning ogRest state to
IDLE once the corresponding bad request response has been sent.
This accounts for the following cases:
- Unknown GET action
- Unknown POST action
- Unknown HTTP verb
ogClient /setup in linux mode do not support to indicate which table
type the user want to use. It always supposes that the partition table
is MBR/MSDOS.
Add ogClient support to work with GPT tables. Add new field table type
to /setup linux mode that expects a string with "MSDOS" or "GPT".
Example old JSON:
{
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
Example new JSON:
{
"type": "GPT",
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
Previous commits updates ogClient and ogServer to support several disks
in Linux (ogLive) mode. This changes disk setup JSON format from an
object to an array. ogClient in virtual mode also need to satisfy this
new format, although it not support several disks.
Adapt ogClient virtual mode to satisfy new disk setup JSON format.
Old format:
"disk_setup": {
"disk": 1,
"partition": 0,
"code": "0",
"filesystem": "",
"os": "",
"size": 32685957,
"used_size": 6
},
New format:
"disk_setup": [
{
"disk": 1,
"partition": 0,
"code": "0",
"filesystem": "",
"os": "",
"size": 32685957,
"used_size": 6
}
],
ogClient expect to receive information of 1 disk from the Cloning
Engine's script "getConfiguration". ogAdmClient, the deprecated ogLive
client daemon, supported for several disk.
Add ogClient support for several disk.
Image datasize is expressed in kibibytes but the existing REST API field
represent data in bytes.
This commit changes ogClient to send datasize in bytes.
Extend ogClient to include more information about the image that has been
created. This patch modifies ogClient to read an info file created by image
creation script, add this info to the JSON response and then remove the file.
Example of new /image/create response:
{
"disk": "1",
"partition": "1",
"code": "131",
"id": "1",
"name": "ubuntu",
"repository": "192.168.56.10",
"software": "Ubuntu 18.04.5 LTS \naccountsservice 0.6.45\n...",
"clonator": "PARTCLONE",
"compressor": "LZOP",
"filesystem": "EXTFS",
"datasize": 2100000
}
New fields are "clonator", "compressor", "filesystem" and "datasize".
Irina reports that "Partition assistant"/"Asistente de particionado" is
not working. This is happening because ogClient is not reading the full
data ogServer sends when the entire HTTP PDU is larger than 1024.
However, ogClient should read the whole message, reading until read data
length is greater or equal to "Content-Length" header value.
ogClient fails to obtain "Content-Length" value because is looking for
"content-length", be aware of the case sensitivity. It also needs to
take into account the header length because read data length also
includes headers.
This patch updates ogClient to:
1) look for "Content-Length instead of "content-length".
2) compare read date length with content length plus headers
length.
Disconnect gracefully from ogServer after booting OS, the script to start the
OS calls kexec, so everything is gone after it. For Windows, this results in a
reboot.
This commit also improves the disconnect function to make sure the
disconnection is synchronous.
This is useful for debuging and getting information on the processes that are
being executed in ogclient.
syslog outputs are something similar to:
Jun 26 10:36:40 ogAdministrator /ogclient: GET refresh HTTP/1.1
Jun 26 10:36:40 ogAdministrator /ogclient: HTTP/1.0 500 Internal Server Err
When restore image command was sent and the restoration failed, you
could see in WebConsole the image as restored and the command completed,
as it if had not failed. This happened because ogClient did not check
the return code of restoration script.
This commit adds return code check on restore image. So, when return
code is a non-zero value ogClient responses with an error 500. When
ogServer receives this error response, it did not set in the database
the image as restored and command as completed without errors.
From https://www.gnu.org/licenses/gpl-howto.en.html:
"You should also include a copy of the license itself somewhere in the
distribution of your program. All programs, whether they are released under the
GPL or LGPL, should include the text version of the GPL. In GNU programs we
conventionally put the license in a file called COPYING."
This script can be useful to install ogclient on distributions that do not have
an already built ogclient package.
It is necessary to install ogclient in the right path, so the command you have
to use looks similar to:
python3 setup.py install --prefix=/opt/opengnsys/ogclient
--install-lib=/opt/opengnsys/ogclient
--install-script=/opt/opengnsys/ogclient
Improves code encapsulation by moving check_vm_state_loop method into
OgVirtualOperations class. This also fixes import error when running ogclient in
'linux' mode.
Commit 621fb7a7 added class initialization with two parameters. These
two parameters were wrongly initialized, "self." was missing. The call
of these params in other functions of the class was also wrong.
This commit fix the initialization of the parameters of
OgLinuxOperations and fix the calls to these params.
ogClient may runs on an OS that do not have all the dependencies needed
to use virtual functions.
This commit change the behaviour to only import virtual functions when
the ogClient has to work with Virtual Machines. This way ogClient works
on environments which do not need virtual functions.
Ths patch handles different cases when a /refresh request can be made. More
specifically this handles the situation in which the VM is stopped when /refresh
is called but it starts during the processing of /refresh, in this case we just
send the last recorded setup.
This patch:
- Fixes logic errors in the communication with QMP (the order of handshake
messages was not right).
- Rewrite parts of OgQMP class.
- Enforces better coding practices by using Python's "context managers" to avoid
forgeting an open socket in case exceptions occur.
- Adapt virtual operations to the use of "context managers" using the "with"
statement.
This is the device that has the best empirical performance right now and it
seems like it is where most of the qemu development is directed towards for the
future.
ogClient changed its config file format to json. This patch adapts
ogLinuxOperation to use new config file.
Co-authored-by: Roberto Hueso <rhueso@soleta.eu>
This patch waits before polling qemu with QMP so that it has time to be ready
for requests. It also increases wait time in case the host machine is slow.
Write permission is not needed to get information about each virtual drive so
this patch changes permissions to 'only read'. This can prevent race conditions.
Jose M. Guisado
OpenGnSys Support Team
Javier Sánchez Parra
Roberto Hueso Gómez
Javier Sánchez Parra