Commit 141b0797e1 introduced command scheduling rest api to
ogserver. Part of this changeset included
og_json_parse_create_image as a utility funtion to parse the json
payload of a "create image" command.
og_json_parse_create_image did not include the parsing of optional
parameters "description", "center_id" and "group_id". New components
like ogCP or ogCLI use these parameters.
Fix this by adding a struct og_image member to the struct og_msg_params
and assigning it when processing a "create image" command.
This could be extended to further payload parsing for image related
commands in the future.
Delete operation for images stored in the server. It deletes an image
from the database and filesystem, thus images must exists in both places.
POST /image/delete
{
"image": "3"
}
Pending schedule commands can deny ogLive boot of clients due
to filling of pending cmd queue with commands such as "Iniciar Sesión".
For example: Using RemotePC to serve clients that do not boot into
ogLive will fill up the pending command queue with "Iniciar Sesión".
Introduce a safety timeout for pending (scheduled) commands to
avoid this situation.
Fixes task_id re-initialization in commit 33d8cecfe.
Task id is to be initialized with the "session" value and can be done
out of the for loop it was before. Avoiding unnecesary snprintf's with
the same value.
If the scope of a command, procedure or task is related to a group of
computers then actions associated with that schedule can share the same
session value.
og_dbi_add_action was not initializing session value, as it is only used
for visualization purposes in the legacy web interface.
This patch enables og_dbi_add_action to populate session column by
assigning it the id value of the first action related to the scope.
Example: Power off command for a room scope of 4 clients will result in
4 actions, a power off for each client. The first action id will determine
the session value for the rest.
A single command can be schedule for several clients.
Commit 141b0797e1 introduces a regression when scheduling a command for
several clients, the command only executes successfuly for the first
one.
Fix it by reintroducing the usage of the 'session' column to group
a single command into several actions (one for each client), the web
console interface already expects this 'session' column to be set
accordingly.
Adds the possibility to create a task with procedures and other tasks
integrated as steps.
Note: "steps" parameter is optional and "steps" array object order
defines execution order.
Request:
POST /task/add
{
"center": "1",
"name": "task",
"description": "My task",
"steps": [
{
"procedure": 4
},
{
"task": 1
},
{
"procedure": 24
}
]
}
Response:
200 OK
This commit also add task case to procedure's step processing.
Otherwise, gcc prints the following warning:
src/rest.c: In function ‘og_procedure_add_steps’:
src/rest.c:4089:17: warning: enumeration value ‘OG_STEP_TASK’ not handled in switch [-Wswitch]
4089 | switch (step->type) {
| ^~~~~~
Adds the possibility to update a procedure with commands and other
procedures integrated as steps.
Note: "steps" parameter is optional and "steps" array object order
defines execution order. Also, if the body has empty steps, the
procedure loses its previous steps.
Request:
POST /procedure/update
{
"procedure": "34",
"center": "1",
"name": "procedure-edited",
"description": "My procedure edited",
"steps": [
{
"command": "wol",
"params": { "type": "broadcast" }
},
{
"procedure": 24
},
{
"command": "reboot",
"params": {}
}
]
}
Response:
200 OK
ogServer searches queued commands (formerly actions) in the DB by
session. This can lead to problems because session is not an uniquely
identifier but an identifier that several commands can share to group
them.
This commit changes query filter from session to id, ensuring correct
results
This reverts commit d9b6aadf66.
Add ogServer support procedure execution. Now users can send a procedure
and a list of clients to ogServer, then ogServer breaks down the
procedure into commands (formerly actions) and queues them for each
indicated client.
TODO: Do not reply 200 OK when the procedure do not exist.
Request:
POST /procedure/run
{
"clients": ["192.168.56.11", "192.168.56.12"],
"procedure": "33"
}
Response:
200 OK
Delete operation for procedures stored in the database.
POST /procedure/delete
{
"id": "7"
}
If no procedure is found returns 200 OK but a syslog call is issued to
warn so. Such behavior will likely change in the future.
Commit 141b079 introduced a slight change in how rows from table
"acciones" are filtered when queuing a command, from "sesion" column to
"idaccion" column.
This seemed reasonable, as id column is the one autoincrementing. But
remotepc queued commands inserting actions id using a timestamp instead
of the action row id.
See c17ffa5d03/admin/WebConsole/rest/remotepc.php (L188)
Revert said change as long as remotepc keeps such behavior.
Adds the possibility to create a procedure with commands and other
procedures integrated as steps.
Note: "steps" parameter is optional and "steps" array object order
defines execution order.
Request:
POST /procedure/add
{
"center": "1",
"name": "procedure",
"description": "My procedure",
"steps": [
{
"command": "wol",
"params": { "type": "broadcast" }
},
{
"procedure": 22
},
{
"command": "poweroff",
"params": {}
}
]
}
Response:
200 OK
This commit also updates unit tests for /procedure/add POST method to
include steps.
Enables ogserver to schedule commands (also referred as actions in
legacy web console jargon).
This feature enables ogserver to write in the "acciones" table in order
to have full capabilities for command scheduling purposes, thus not
depending in the legacy web console to insert into "acciones" table.
This method adds a procedure associated with a center to the database.
Required payload parameters are center and name, description is
optional.
Note: ogServer does not allow to add more than one procedure with the
same name and center.
Request:
POST /procedure/add
{
"center": "1"
"name": "procedure1"
"description": "My procedure"
}
Response:
200 OK
This commit also adds unit tests for /procedure/add POST method.
This method deletes a room (lab) from the DB and deletes on cascade
computers and computers partitions.
Note: if the room id do not exists in the database, ogserver still
tries to delete it and replies with 200 OK.
Request:
POST /room/delete
{
"id": "1"
}
Response:
200 OK
This method deletes a center from the DB and deletes on cascade
rooms/labs, computers and computers partitions.
Note: if the center id do not exists in the database, ogserver still
tries to delete it and replies with 200 OK.
Request:
POST /center/delete
{
"id": "1"
}
Response:
200 OK
If a probe response contains speedinformation, parse and store
it inside the client struct. Speed is interpreted as an unsigned
integer representing Mbit/s.
Avoids multiple entries of a same client like
{"clients": [{"addr": "192.168.2.230", "state": "WOL_SENT"}, {"addr": "192.168.2.230", "state": "OPG"}]
These can arise when ogserver processes a WoL request for an already
connected client.
When processing the WoL request, search for the target address in the
clients list, if found we avoid creating the wol entry.
Otherwise, copying response json to response buffer could lead to stack
smashing is the json response is too large.
stdout example:
*** stack smashing detected ***: <unknown> terminated
Adds POST method to add centers (organizational unit), required payload
parameter is the name, and an additional comment is optional.
{"name": "ACME"}
{"name": "ACME", "comment": "Some comment"}
This function returns the installed and available ogLiveS in the server
to be booted from.
Request:
GET /oglive/list
NO BODY
Response
200 OK
{
"oglive": [
{
"distribution": "bionic",
"kernel": "5.4.0-40-generic",
"architecture": "amd64",
"revision": "r20200629",
"directory": "ogLive-5.4.0-r20200629",
"iso": "ogLive-bionic-5.4.0-40-generic-amd64-r20200629.85eceaf.iso"
},
{
"distribution": "bionic",
"kernel": "5.0.0-27-generic",
"architecture": "amd64",
"revision": "r20190830",
"directory": "ogLive-5.0.0-r20190830",
"iso": "ogLive-bionic-5.0.0-27-generic-amd64-r20190830.7208cc9.iso"
}
],
"default": 0
}
This commit also adds tests for GET /oglive/test.
==28831== 1 errors in context 1 of 2:
==28831== Invalid read of size 1
==28831== at 0x55AC6FD: inet_aton (inet_addr.c:127)
==28831== by 0x10ECCA: WakeUp (ogAdmServer.c:337)
==28831== by 0x10EED6: Levanta (ogAdmServer.c:292)
==28831== by 0x11651E: og_cmd_wol (rest.c:498)
==28831== by 0x11651E: og_client_state_process_payload_rest (rest.c:3970)
==28831== by 0x110CF3: og_client_read_cb (core.c:143)
==28831== by 0x4E41D72: ev_invoke_pending (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x4E453DD: ev_run (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x10E3E5: ev_loop (ev.h:835)
==28831== by 0x10E3E5: main (main.c:100)
==28831== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Use number of matching ip addresses in the database, skip if zero.
ogServer gets netmask address from computer (ordenadores) table, see
commit a35b7c4. Netmask field is empty in most cases, is only filled
when the user adds computers with dhcpd.conf syntax and cannot be edited
in computer properties view.
Labs/rooms (aulas) table also have netmask field, WebConsole backend
ensures it is not empty and can be edited in lab properties view.
Get netmask from labs table to ensure it is not empty.
Tests for e68fefe were made after 00pm (12:00) so we did not cover <12:00 cases
for immediate commands that are logged (scheduled for the exact moment
they are processed and ignored the fact they are stale so they are executed
right away)
In addition, libdbi was complaining about the data type used to
represent the hours, they were not being inserted properly. From syslog:
failed to query database (og_dbi_schedule_create:3288) 1264: Out of
range value for column 'horas' at row 1
Fix og_tm_hours_mask so <12:00 immediate schedule is handled correctly.
Change return type to uint16_t, as the 'hours' column type is smallint(4)
Fixes e68fefe ("#997 Set stale check flag when processing schedule/create")
Commit e68fefe introduced 'check_stale' flag to better distinguish
real scheduled actions that do not execute if they are stale from
immediate actions that we want them to be logged in the action queue
(by creating a decoy schedule for the exact moment they are processed,
meaning that we ignore if the are stale).
Add this feature into schedule update too, in order to avoid executing
stale commands that were not meant to, ie. real scheduled commands.
Follows e68fefe ("Set stale check flag when processing schedule/create")
After executing an scheduled command/proc/task valgrind reported
leaks inside og_dbi_queue_{command,procedure,task}. String
duplication is not being freed after using them.
==21281== 36 bytes in 1 blocks are definitely lost in loss record 470 of
592
...
==21281== by 0x113DCB: og_dbi_queue_procedure (rest.c:2748)
==21281== by 0x113F91: og_dbi_queue_task (rest.c:2804)
==21281== by 0x114392: og_schedule_run (rest.c:2916)
==21281== by 0x112059: og_agent_timer_cb (schedule.c:441)
...
==21281== by 0x10E2A5: main (main.c:100)
These strdup are not necessary because the dbi result is not freed
before using them, it's safe to use the dbi result's reference to
this string.
Fix previous memleaks when executing scheduled commands, procedures
and tasks.
If you schedule a command in the past, the scheduler executes such
command immediately.
When expanding a schedule that result in commands that run weekly,
commands in the past are also executed, which is not expected.
Fix this by using the check_stale flag (formerly on_start) so
commands in the past that result from expansions are skipped.
image_json object is created to store the json representation of
an image returned by the database. This object is going to be appended
to a json list that will compose the overall root json object.
Use json_array_append_new to let "images" steal the reference of
image_json so when further decref(root) there is no json reference
hanging around.
Allocated strings using dbi_result_get_string_copy are not being freed
after using them. This patch fix this memory leak.
Fixes: e4cb91b ("#990 wol: migrate mac and netmask query to ogServer")
UMA and UPV report that Wake on LAN command (in queue mode) does not
work.
We improved WoL command, now ogServer calculates the broadcast address
of the network to which the client belongs. To calculate this address
ogServer needs the IP and the netmask of the client. We updated ogServer
to retrieve the netmask from the database in non-queue mode, but we forgot
to add this in queue mode.
This patch adds netmask retrieving to queued WoL.
ogServer WoL rest function is expecting ip, mac and netmask from the
request's payload. This makes the client responsible for providing such
data.
ogServer should be the owner of the database, this patch adjusts the
parameter expected in a request payload for /wol action so clients only
need to provide the ips and wol type (broadcast, unicast).
Database is expected to contain valid data for the netmask and mac of
target computers.
og_cmd_get_client_setup is querying the database for computers disk
setup information. This includes the size of a disk or a partition,
which are stored as BIGINT in the database.
Using dbi_result_get_int will result in an error as libdbi expects to
store this kind of data type in a long long instead of an int.
Fix disk size variable declaration and use the correct dbi_result_get
function for this particular data column.
UMA reports that the setup command (in queue mode) does not work.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The setup command has a new field "ttp" since commit 0dd3edd, however,
the ogServer legacy parser was not updated to use this new field.
This patch adds legacy setup support to work with GPT tables. Add new field
table type to legacy setup that expects a string with "MSDOS" or "GPT".
UMA reports that the restore image command (in queue mode) does not work,
the non-queue mode works fine though.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The restore-image command has the field "ptc" which stores a string
with whitespaces, however, ogServer expects no whitespaces ("%s").
Update parser to read until carriage return, whitespaces included.
When executing og_cmd_post_modes ogServer is looking up for the template
related to the mode coming as a parameter. This lookup is only done for
the bios boot mode. (ie. /opt/opengnsys/tftpboot/menu.lst/templates/)
Templates can be created for a given boot mode, for example you can
create a UEFI-only template (it.
/opt/opengnsys/tftpboot/grub/templates).
When a UEFI only template was coming as a parameter to og_cmd_post_modes
the file can't be located because the UEFI template folder is never
tested.
Add UEFI folder to the lookup of pxe templates, try this folder if bios
mode fails.
PS: Later on a bash script "setclientmode" is to be executed, which will
take its own way of updating these pxe files (ie. templates/../), this
script updates every boot mode if an available template is found.
ogServer /setup do not support to indicate which table type the user
want to use. It always supposes that the partition table is MBR/MSDOS.
Add ogServer support to work with GPT tables. Add new field table type
to /setup that expects a string with "MSDOS" or "GPT".
Example old JSON:
{
"clients": [...],
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
Example new JSON:
{
"clients": [...],
"type": "GPT",
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
This function returns the installed and available OSs in a client to be
booted from.
Request:
GET /session
{
"client": ["192.168.56.11"]
}
Response
200 OK
{
"sessions": [
{
"disk": 1,
"name": "Ubuntu 18.04.4 LTS",
"partition": 1
}
]
}
This patch uses client's IP instead of scope ID as a filter for the
request.
Request:
GET /hardware
{
"client": ["192.168.56.11"]
}
Response:
200 OK
{
"hardware": [
{
"description": "BIOS",
"type": "Tipo de proceso de arranque"
},
{
"description": "QEMU Standard PC (i440FX + PIIX, 1996) v.pc-i440fx-5.1",
"type": "Marca y modelo del equipo"
},
{
"description": "Intel Corp. Intel Core Processor (Haswell, no TSX, IBRS) 2GHz v.pc-i440fx-5.1",
"type": "Microprocesadores"
},
{
"description": "QEMU 2049MiB (DIMM 0)",
"type": "Memorias"
},
{
"description": "Red Hat, Inc. Virtio network device v.00",
"type": "Tarjetas de Red"
}
]
}
This function retrieves a list of the software installed in a specific
disk and partition of a client. This list can be updated by using POST
/software beforehand.
Request:
GET /software
{
"client": ["192.168.56.11"],
"disk": 1,
"partition": 1
}
Response:
200 OK
{
"software": [
"grub-pc-bin 2",
"grub-pc 2.02",
"grub2-common 2.02",
"groff-base 1.22.3",
"gpg 2.2.4",
"gnupg-utils 2.2.4",
"gnupg-l10n 2.2.4",
"gnupg 2.2.4",
"gnome-video-effects 0.4.3",
"gnome-user-guide 3.28.2+git20180715",
"gnome-user-docs 3.28.2+git20180715",
"gnome-todo-common 3.28.1",
"gnome-mines 3.28.0",
"gnome-menus 3.13.3",
"gnome-mahjongg 3.22.0",
"gnome-keyring-pkcs11 3.28.0.2",
"gnome-keyring 3.28.0.2",
"gnome-initial-setup 3.28.0",
"accountsservice 0.6.45"
]
}
This extension removes replaces the field 'filename' for 'name' and adds
new fields (software_id, type and id). These new fields are useful when
restoring an image.
Request:
GET /images
Response:
200 OK
{
"disk": {
"free": 37091418112,
"total": 52573995008
},
"images": [
{
"datasize": 5939200000,
"id": 25,
"modified": "Wed Oct 14 11:49:00 2020",
"name": "archlinux",
"permissions": "744",
"size": 1844222333,
"software_id": 19,
"type": 1
}
]
}
In case the DB entry for an image does not exist when POST /image/create
is called, this patch takes care of calling it.
This adds few optional json parameters to the POST /image/create API. If
optional parameters are included then this patch creates the DB entry,
otherwise it just creates the actual image and updates the existing
entry.
Request:
POST /image/create
{
"clients":["192.168.56.11"],
"disk":"1",
"partition":"1",
"name":"archlinux",
"repository":"192.168.56.10",
"id":"24",
"code":"131",
"description":"This is a test",
"group_id":0,
"center_id":1
}
Response:
200 OK
This method deletes a client from the DB.
Request:
POST /client/delete
{
"clients": ["192.168.56.30"]
}
Response:
200 OK
Co-authored-by: Javier Sánchez Parra <jsanchez@soleta.eu>
Use og_server_cfg everywhere. Convert port to string to make it easy for the
dbi API since it expects a string. Remove legacy example configuration file.
This patch is a refactor for og_computer and og_dbi_get_computer_info.
It now uses fixed lenght strings to make it more reliable and avoid
errors if the DB is not returning a null ended string.
This patch selects clients whose modes need to be changed in the current
request. This patch makes POST /mode consistent with the rest of the
REST API where clients are selected and filtered by ip.
CC src/rest.o
src/rest.c: In function ‘og_cmd_post_modes.isra.27’:
src/rest.c:966:4: warning: ‘%s’ directive output may be truncated writing up to 4096 bytes into a region of size between 3018 and 4041 -Wformat-truncation=]
"MODE_FILE='%s'\nMAC='%s'\nDATA='%s'\n"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/rest.c:1085:7:
template_name, scope_name);
~~~~~~~~~~~~~
src/rest.c:967:33: note: format string is defined here
"MODE='PERM'\nTEMPLATE_NAME='%s'",
^~
src/rest.c:965:2: note: ‘snprintf’ output 57 or more bytes (assuming 5176) into a destination of size 4096
snprintf(cmd_params, sizeof(cmd_params),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"MODE_FILE='%s'\nMAC='%s'\nDATA='%s'\n"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"MODE='PERM'\nTEMPLATE_NAME='%s'",
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mode, mac, params, template_name);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now POST /modes does not only write to the database, it also uses the
external bash script 'setclientmode' to generate configured PXE
templates for each machine.
NOTE: This fix requires that room and computer names are unique in the
OpenGnsys DB. This behaviour is inherited from the previous
'setclientmode' script but, eventually, it should be replaced for an
actual scope selector.
This patch implements HTTP POST /modes request which can change the mode of any
particular scope.
Request: POST /modes
{
"scope": {"id": 1,
"type": "computer"},
"mode": "pxe"
}
Response: 200 OK
Some universities have computers in a different subnet where the
ogServer is, but ogServer WoL functionality only supported to send
packet to its own subnets. This commit solves this.
Now ogServer sends two WoL packets per client, one with the broadcast
address of the interface indicated in the config file, the other with
the broadcast address calculated with the address and netmask of the
client.
To ensure that the second WoL works correctly you must configure
correctly the IP and netmask of the clients. Also, you have to configure
the network of your organization to route WoL packet to the correct
subnet.
This patch implements HTTP GET /modes request which returns all modes available
for clients:
Request: GET /modes HTTP/1.0
Response: 200 OK
{
"modes": [
"pxe",
"10",
"13",
"00unknown",
"11",
"19pxeADMIN",
"12"
]
}
Jose M. Guisado
Javier Sánchez Parra
OpenGnSys Support Team
Diego Crespo Quinta
Roberto Hueso Gómez
Irina Gomez