This patch adds database schema management capabilities to ogServer:
- ogServer now tracks the version of its database schema, if no version
is detected, creates a 'version' table with a single row starting at 0.
- ogServer can upgrade its database schema to a newer version if
detected. (ogServer ships required SQL commands to do so)
If ogServer is unable to upgrade the schema at startup (if needed be) it
*will not* start.
Defines schema update v1 which upgrades database engine tables of
ogServer database (usually named 'ogAdmBD') from myISAM to innoDB.
Otherwise, copying response json to response buffer could lead to stack
smashing is the json response is too large.
stdout example:
*** stack smashing detected ***: <unknown> terminated
Several universities have reported that creating a software profile
hangs the machine running the ogServer for a while, sometimes up to
minutes.
Legacy SQL code is producing said bottleneck, responsible for pruning a
intermediate table between "perfilessoft" and "softwares". There is
redundant code, "perfilssoft" should be pruned first, speeding up the
later task of pruning the intermediate table "perfilessoft_softwares"
There is no need to execute:
DELETE FROM perfilessoft_softwares
WHERE idperfilsoft IN (
SELECT idperfilsoft
FROM perfilessoft
WHERE idperfilsoft NOT IN (
SELECT DISTINCT idperfilsoft
from ordenadores_particiones)
AND idperfilsoft NOT IN (
SELECT DISTINCT idperfilsoft from imagenes))
When afterwards "perfilessoft" is going to be pruned and
"perfilessoft_softwares" pruned again:
DELETE FROM perfilessoft WHERE idperfilsoft NOT IN
(SELECT DISTINCT idperfilsoft from ordenadores_particiones)
AND idperfilsoft NOT IN
(SELECT DISTINCT idperfilsoft from imagenes)
DELETE FROM perfilessoft_softwares WHERE idperfilsoft NOT IN
(SELECT idperfilsoft from perfilessoft)
The two latter commands suffice.
This should not happen when using a relational database supporting
foreign keys and ON DELETE CASCADE, like innoDB, which will be adopted
soon.
mktime modifies the struct tm it receives and takes into account whether DST is
active or not (tm_isdst). tm_isdst == 0 adjusts the time, which causes the time
mismatch error.
All fields are being initialized to 0 and therefore it is assumed that the time
that has been passed is not in daylight saving time.
When the value is negative in tm.tm_isdst it delegates to mktime to guess if it
is in daylight saving time or not, this works 99% of the time.
Best way would be that ogserver knows what is its timezone and when daylight
saving applies, so tm_isdst is set to 0 or 1 accordingly.
Meanwhile, "tm_isdst = -1" provides the hotfix.
Adds POST method to add centers (organizational unit), required payload
parameter is the name, and an additional comment is optional.
{"name": "ACME"}
{"name": "ACME", "comment": "Some comment"}
/create/image adds an entry to the database for the given partition
image created when payload contains a "description" attribute. This
insertion into the database is lacking a check for duplicates, which are
not supported for the images table.
Add a prior duplicate check before inserting. Exit with -1 code if an
image with the same name is found.
This function returns the installed and available ogLiveS in the server
to be booted from.
Request:
GET /oglive/list
NO BODY
Response
200 OK
{
"oglive": [
{
"distribution": "bionic",
"kernel": "5.4.0-40-generic",
"architecture": "amd64",
"revision": "r20200629",
"directory": "ogLive-5.4.0-r20200629",
"iso": "ogLive-bionic-5.4.0-40-generic-amd64-r20200629.85eceaf.iso"
},
{
"distribution": "bionic",
"kernel": "5.0.0-27-generic",
"architecture": "amd64",
"revision": "r20190830",
"directory": "ogLive-5.0.0-r20190830",
"iso": "ogLive-bionic-5.0.0-27-generic-amd64-r20190830.7208cc9.iso"
}
],
"default": 0
}
This commit also adds tests for GET /oglive/test.
==28831== 1 errors in context 1 of 2:
==28831== Invalid read of size 1
==28831== at 0x55AC6FD: inet_aton (inet_addr.c:127)
==28831== by 0x10ECCA: WakeUp (ogAdmServer.c:337)
==28831== by 0x10EED6: Levanta (ogAdmServer.c:292)
==28831== by 0x11651E: og_cmd_wol (rest.c:498)
==28831== by 0x11651E: og_client_state_process_payload_rest (rest.c:3970)
==28831== by 0x110CF3: og_client_read_cb (core.c:143)
==28831== by 0x4E41D72: ev_invoke_pending (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x4E453DD: ev_run (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x10E3E5: ev_loop (ev.h:835)
==28831== by 0x10E3E5: main (main.c:100)
==28831== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Use number of matching ip addresses in the database, skip if zero.
ogServer gets netmask address from computer (ordenadores) table, see
commit a35b7c4. Netmask field is empty in most cases, is only filled
when the user adds computers with dhcpd.conf syntax and cannot be edited
in computer properties view.
Labs/rooms (aulas) table also have netmask field, WebConsole backend
ensures it is not empty and can be edited in lab properties view.
Get netmask from labs table to ensure it is not empty.
Tests for e68fefe were made after 00pm (12:00) so we did not cover <12:00 cases
for immediate commands that are logged (scheduled for the exact moment
they are processed and ignored the fact they are stale so they are executed
right away)
In addition, libdbi was complaining about the data type used to
represent the hours, they were not being inserted properly. From syslog:
failed to query database (og_dbi_schedule_create:3288) 1264: Out of
range value for column 'horas' at row 1
Fix og_tm_hours_mask so <12:00 immediate schedule is handled correctly.
Change return type to uint16_t, as the 'hours' column type is smallint(4)
Fixes e68fefe ("#997 Set stale check flag when processing schedule/create")
Commit e68fefe introduced 'check_stale' flag to better distinguish
real scheduled actions that do not execute if they are stale from
immediate actions that we want them to be logged in the action queue
(by creating a decoy schedule for the exact moment they are processed,
meaning that we ignore if the are stale).
Add this feature into schedule update too, in order to avoid executing
stale commands that were not meant to, ie. real scheduled commands.
Follows e68fefe ("Set stale check flag when processing schedule/create")
After executing an scheduled command/proc/task valgrind reported
leaks inside og_dbi_queue_{command,procedure,task}. String
duplication is not being freed after using them.
==21281== 36 bytes in 1 blocks are definitely lost in loss record 470 of
592
...
==21281== by 0x113DCB: og_dbi_queue_procedure (rest.c:2748)
==21281== by 0x113F91: og_dbi_queue_task (rest.c:2804)
==21281== by 0x114392: og_schedule_run (rest.c:2916)
==21281== by 0x112059: og_agent_timer_cb (schedule.c:441)
...
==21281== by 0x10E2A5: main (main.c:100)
These strdup are not necessary because the dbi result is not freed
before using them, it's safe to use the dbi result's reference to
this string.
Fix previous memleaks when executing scheduled commands, procedures
and tasks.
If you schedule a command in the past, the scheduler executes such
command immediately.
When expanding a schedule that result in commands that run weekly,
commands in the past are also executed, which is not expected.
Fix this by using the check_stale flag (formerly on_start) so
commands in the past that result from expansions are skipped.
image_json object is created to store the json representation of
an image returned by the database. This object is going to be appended
to a json list that will compose the overall root json object.
Use json_array_append_new to let "images" steal the reference of
image_json so when further decref(root) there is no json reference
hanging around.
Allocated strings using dbi_result_get_string_copy are not being freed
after using them. This patch fix this memory leak.
Fixes: e4cb91b ("#990 wol: migrate mac and netmask query to ogServer")
UMA and UPV report that Wake on LAN command (in queue mode) does not
work.
We improved WoL command, now ogServer calculates the broadcast address
of the network to which the client belongs. To calculate this address
ogServer needs the IP and the netmask of the client. We updated ogServer
to retrieve the netmask from the database in non-queue mode, but we forgot
to add this in queue mode.
This patch adds netmask retrieving to queued WoL.
ogServer WoL rest function is expecting ip, mac and netmask from the
request's payload. This makes the client responsible for providing such
data.
ogServer should be the owner of the database, this patch adjusts the
parameter expected in a request payload for /wol action so clients only
need to provide the ips and wol type (broadcast, unicast).
Database is expected to contain valid data for the netmask and mac of
target computers.
og_cmd_get_client_setup is querying the database for computers disk
setup information. This includes the size of a disk or a partition,
which are stored as BIGINT in the database.
Using dbi_result_get_int will result in an error as libdbi expects to
store this kind of data type in a long long instead of an int.
Fix disk size variable declaration and use the correct dbi_result_get
function for this particular data column.
UMA reports that the setup command (in queue mode) does not work.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The setup command has a new field "ttp" since commit 0dd3edd, however,
the ogServer legacy parser was not updated to use this new field.
This patch adds legacy setup support to work with GPT tables. Add new field
table type to legacy setup that expects a string with "MSDOS" or "GPT".
UMA reports that the restore image command (in queue mode) does not work,
the non-queue mode works fine though.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The restore-image command has the field "ptc" which stores a string
with whitespaces, however, ogServer expects no whitespaces ("%s").
Update parser to read until carriage return, whitespaces included.
Some distros enable *.* in syslog.conf to add all logging information
to /var/log/syslog. The existing LOG_DEBUG syslog() entries that ogserver
generates fill up the log files very quickly. Remove most of the LOG_DEBUG
syslog() calls.
When executing og_cmd_post_modes ogServer is looking up for the template
related to the mode coming as a parameter. This lookup is only done for
the bios boot mode. (ie. /opt/opengnsys/tftpboot/menu.lst/templates/)
Templates can be created for a given boot mode, for example you can
create a UEFI-only template (it.
/opt/opengnsys/tftpboot/grub/templates).
When a UEFI only template was coming as a parameter to og_cmd_post_modes
the file can't be located because the UEFI template folder is never
tested.
Add UEFI folder to the lookup of pxe templates, try this folder if bios
mode fails.
PS: Later on a bash script "setclientmode" is to be executed, which will
take its own way of updating these pxe files (ie. templates/../), this
script updates every boot mode if an available template is found.
ogServer /setup do not support to indicate which table type the user
want to use. It always supposes that the partition table is MBR/MSDOS.
Add ogServer support to work with GPT tables. Add new field table type
to /setup that expects a string with "MSDOS" or "GPT".
Example old JSON:
{
"clients": [...],
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
Example new JSON:
{
"clients": [...],
"type": "GPT",
"disk": "1",
"cache": "0",
"cache_size": "0",
"partition_setup": [...]
}
ogServer expects to receive information of 1 disk and 4 partitions
from ogClient. Previous ogServer versions support several disks and
partitions.
Add ogServer support for 4 disks and 16 partitions.
Creating a software profile failed. ogServer obtained the information of
a computer without center id because commit cbd9421 removed it
unintentionally, making all computer had center id equal to zero.
This commit restores the gathering of computer center id from the
database.
This function returns the installed and available OSs in a client to be
booted from.
Request:
GET /session
{
"client": ["192.168.56.11"]
}
Response
200 OK
{
"sessions": [
{
"disk": 1,
"name": "Ubuntu 18.04.4 LTS",
"partition": 1
}
]
}
This patch uses client's IP instead of scope ID as a filter for the
request.
Request:
GET /hardware
{
"client": ["192.168.56.11"]
}
Response:
200 OK
{
"hardware": [
{
"description": "BIOS",
"type": "Tipo de proceso de arranque"
},
{
"description": "QEMU Standard PC (i440FX + PIIX, 1996) v.pc-i440fx-5.1",
"type": "Marca y modelo del equipo"
},
{
"description": "Intel Corp. Intel Core Processor (Haswell, no TSX, IBRS) 2GHz v.pc-i440fx-5.1",
"type": "Microprocesadores"
},
{
"description": "QEMU 2049MiB (DIMM 0)",
"type": "Memorias"
},
{
"description": "Red Hat, Inc. Virtio network device v.00",
"type": "Tarjetas de Red"
}
]
}
This function retrieves a list of the software installed in a specific
disk and partition of a client. This list can be updated by using POST
/software beforehand.
Request:
GET /software
{
"client": ["192.168.56.11"],
"disk": 1,
"partition": 1
}
Response:
200 OK
{
"software": [
"grub-pc-bin 2",
"grub-pc 2.02",
"grub2-common 2.02",
"groff-base 1.22.3",
"gpg 2.2.4",
"gnupg-utils 2.2.4",
"gnupg-l10n 2.2.4",
"gnupg 2.2.4",
"gnome-video-effects 0.4.3",
"gnome-user-guide 3.28.2+git20180715",
"gnome-user-docs 3.28.2+git20180715",
"gnome-todo-common 3.28.1",
"gnome-mines 3.28.0",
"gnome-menus 3.13.3",
"gnome-mahjongg 3.22.0",
"gnome-keyring-pkcs11 3.28.0.2",
"gnome-keyring 3.28.0.2",
"gnome-initial-setup 3.28.0",
"accountsservice 0.6.45"
]
}
This extension removes replaces the field 'filename' for 'name' and adds
new fields (software_id, type and id). These new fields are useful when
restoring an image.
Request:
GET /images
Response:
200 OK
{
"disk": {
"free": 37091418112,
"total": 52573995008
},
"images": [
{
"datasize": 5939200000,
"id": 25,
"modified": "Wed Oct 14 11:49:00 2020",
"name": "archlinux",
"permissions": "744",
"size": 1844222333,
"software_id": 19,
"type": 1
}
]
}
In case the DB entry for an image does not exist when POST /image/create
is called, this patch takes care of calling it.
This adds few optional json parameters to the POST /image/create API. If
optional parameters are included then this patch creates the DB entry,
otherwise it just creates the actual image and updates the existing
entry.
Request:
POST /image/create
{
"clients":["192.168.56.11"],
"disk":"1",
"partition":"1",
"name":"archlinux",
"repository":"192.168.56.10",
"id":"24",
"code":"131",
"description":"This is a test",
"group_id":0,
"center_id":1
}
Response:
200 OK
This method deletes a client from the DB.
Request:
POST /client/delete
{
"clients": ["192.168.56.30"]
}
Response:
200 OK
Co-authored-by: Javier Sánchez Parra <jsanchez@soleta.eu>
Use og_server_cfg everywhere. Convert port to string to make it easy for the
dbi API since it expects a string. Remove legacy example configuration file.
This patch is a refactor for og_computer and og_dbi_get_computer_info.
It now uses fixed lenght strings to make it more reliable and avoid
errors if the DB is not returning a null ended string.
Jose M. Guisado
Javier Sánchez Parra
Diego Crespo Quinta
OpenGnSys Support Team
Roberto Hueso Gómez