This patch adds database schema management capabilities to ogServer:
- ogServer now tracks the version of its database schema, if no version
is detected, creates a 'version' table with a single row starting at 0.
- ogServer can upgrade its database schema to a newer version if
detected. (ogServer ships required SQL commands to do so)
If ogServer is unable to upgrade the schema at startup (if needed be) it
*will not* start.
Defines schema update v1 which upgrades database engine tables of
ogServer database (usually named 'ogAdmBD') from myISAM to innoDB.
This commit adds a test for HTTP responses that are too large to fit in
ogServer response buffer.
It also moves the basic sql data used for the other tests to its own
file, easing its reuse in several files.
Python test script launches its own ogServer to run tests. If there is
another ogServer running at the same time, it interferes with tests.
Installing ogServer in a machine results in an enabled ogServer service.
Stop ogServer service before tests just in case the user installed
ogServer in the machine.
Otherwise, copying response json to response buffer could lead to stack
smashing is the json response is too large.
stdout example:
*** stack smashing detected ***: <unknown> terminated
Several universities have reported that creating a software profile
hangs the machine running the ogServer for a while, sometimes up to
minutes.
Legacy SQL code is producing said bottleneck, responsible for pruning a
intermediate table between "perfilessoft" and "softwares". There is
redundant code, "perfilssoft" should be pruned first, speeding up the
later task of pruning the intermediate table "perfilessoft_softwares"
There is no need to execute:
DELETE FROM perfilessoft_softwares
WHERE idperfilsoft IN (
SELECT idperfilsoft
FROM perfilessoft
WHERE idperfilsoft NOT IN (
SELECT DISTINCT idperfilsoft
from ordenadores_particiones)
AND idperfilsoft NOT IN (
SELECT DISTINCT idperfilsoft from imagenes))
When afterwards "perfilessoft" is going to be pruned and
"perfilessoft_softwares" pruned again:
DELETE FROM perfilessoft WHERE idperfilsoft NOT IN
(SELECT DISTINCT idperfilsoft from ordenadores_particiones)
AND idperfilsoft NOT IN
(SELECT DISTINCT idperfilsoft from imagenes)
DELETE FROM perfilessoft_softwares WHERE idperfilsoft NOT IN
(SELECT idperfilsoft from perfilessoft)
The two latter commands suffice.
This should not happen when using a relational database supporting
foreign keys and ON DELETE CASCADE, like innoDB, which will be adopted
soon.
mktime modifies the struct tm it receives and takes into account whether DST is
active or not (tm_isdst). tm_isdst == 0 adjusts the time, which causes the time
mismatch error.
All fields are being initialized to 0 and therefore it is assumed that the time
that has been passed is not in daylight saving time.
When the value is negative in tm.tm_isdst it delegates to mktime to guess if it
is in daylight saving time or not, this works 99% of the time.
Best way would be that ogserver knows what is its timezone and when daylight
saving applies, so tm_isdst is set to 0 or 1 accordingly.
Meanwhile, "tm_isdst = -1" provides the hotfix.
Adds POST method to add centers (organizational unit), required payload
parameter is the name, and an additional comment is optional.
{"name": "ACME"}
{"name": "ACME", "comment": "Some comment"}
/create/image adds an entry to the database for the given partition
image created when payload contains a "description" attribute. This
insertion into the database is lacking a check for duplicates, which are
not supported for the images table.
Add a prior duplicate check before inserting. Exit with -1 code if an
image with the same name is found.
This function returns the installed and available ogLiveS in the server
to be booted from.
Request:
GET /oglive/list
NO BODY
Response
200 OK
{
"oglive": [
{
"distribution": "bionic",
"kernel": "5.4.0-40-generic",
"architecture": "amd64",
"revision": "r20200629",
"directory": "ogLive-5.4.0-r20200629",
"iso": "ogLive-bionic-5.4.0-40-generic-amd64-r20200629.85eceaf.iso"
},
{
"distribution": "bionic",
"kernel": "5.0.0-27-generic",
"architecture": "amd64",
"revision": "r20190830",
"directory": "ogLive-5.0.0-r20190830",
"iso": "ogLive-bionic-5.0.0-27-generic-amd64-r20190830.7208cc9.iso"
}
],
"default": 0
}
This commit also adds tests for GET /oglive/test.
==28831== 1 errors in context 1 of 2:
==28831== Invalid read of size 1
==28831== at 0x55AC6FD: inet_aton (inet_addr.c:127)
==28831== by 0x10ECCA: WakeUp (ogAdmServer.c:337)
==28831== by 0x10EED6: Levanta (ogAdmServer.c:292)
==28831== by 0x11651E: og_cmd_wol (rest.c:498)
==28831== by 0x11651E: og_client_state_process_payload_rest (rest.c:3970)
==28831== by 0x110CF3: og_client_read_cb (core.c:143)
==28831== by 0x4E41D72: ev_invoke_pending (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x4E453DD: ev_run (in /usr/lib/x86_64-linux-gnu/libev.so.4.0.0)
==28831== by 0x10E3E5: ev_loop (ev.h:835)
==28831== by 0x10E3E5: main (main.c:100)
==28831== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Use number of matching ip addresses in the database, skip if zero.
ogServer gets netmask address from computer (ordenadores) table, see
commit a35b7c4. Netmask field is empty in most cases, is only filled
when the user adds computers with dhcpd.conf syntax and cannot be edited
in computer properties view.
Labs/rooms (aulas) table also have netmask field, WebConsole backend
ensures it is not empty and can be edited in lab properties view.
Get netmask from labs table to ensure it is not empty.
Tests for e68fefe were made after 00pm (12:00) so we did not cover <12:00 cases
for immediate commands that are logged (scheduled for the exact moment
they are processed and ignored the fact they are stale so they are executed
right away)
In addition, libdbi was complaining about the data type used to
represent the hours, they were not being inserted properly. From syslog:
failed to query database (og_dbi_schedule_create:3288) 1264: Out of
range value for column 'horas' at row 1
Fix og_tm_hours_mask so <12:00 immediate schedule is handled correctly.
Change return type to uint16_t, as the 'hours' column type is smallint(4)
Fixes e68fefe ("#997 Set stale check flag when processing schedule/create")
Commit e68fefe introduced 'check_stale' flag to better distinguish
real scheduled actions that do not execute if they are stale from
immediate actions that we want them to be logged in the action queue
(by creating a decoy schedule for the exact moment they are processed,
meaning that we ignore if the are stale).
Add this feature into schedule update too, in order to avoid executing
stale commands that were not meant to, ie. real scheduled commands.
Follows e68fefe ("Set stale check flag when processing schedule/create")
After executing an scheduled command/proc/task valgrind reported
leaks inside og_dbi_queue_{command,procedure,task}. String
duplication is not being freed after using them.
==21281== 36 bytes in 1 blocks are definitely lost in loss record 470 of
592
...
==21281== by 0x113DCB: og_dbi_queue_procedure (rest.c:2748)
==21281== by 0x113F91: og_dbi_queue_task (rest.c:2804)
==21281== by 0x114392: og_schedule_run (rest.c:2916)
==21281== by 0x112059: og_agent_timer_cb (schedule.c:441)
...
==21281== by 0x10E2A5: main (main.c:100)
These strdup are not necessary because the dbi result is not freed
before using them, it's safe to use the dbi result's reference to
this string.
Fix previous memleaks when executing scheduled commands, procedures
and tasks.
If you schedule a command in the past, the scheduler executes such
command immediately.
When expanding a schedule that result in commands that run weekly,
commands in the past are also executed, which is not expected.
Fix this by using the check_stale flag (formerly on_start) so
commands in the past that result from expansions are skipped.
image_json object is created to store the json representation of
an image returned by the database. This object is going to be appended
to a json list that will compose the overall root json object.
Use json_array_append_new to let "images" steal the reference of
image_json so when further decref(root) there is no json reference
hanging around.
Allocated strings using dbi_result_get_string_copy are not being freed
after using them. This patch fix this memory leak.
Fixes: e4cb91b ("#990 wol: migrate mac and netmask query to ogServer")
Commit e4cb91b changed MAC and netmask retrieving, now ogServer take these
addresses from the database instead of from the JSON.
Remove MAC and netmask from the JSON body.
UMA and UPV report that Wake on LAN command (in queue mode) does not
work.
We improved WoL command, now ogServer calculates the broadcast address
of the network to which the client belongs. To calculate this address
ogServer needs the IP and the netmask of the client. We updated ogServer
to retrieve the netmask from the database in non-queue mode, but we forgot
to add this in queue mode.
This patch adds netmask retrieving to queued WoL.
ogServer WoL rest function is expecting ip, mac and netmask from the
request's payload. This makes the client responsible for providing such
data.
ogServer should be the owner of the database, this patch adjusts the
parameter expected in a request payload for /wol action so clients only
need to provide the ips and wol type (broadcast, unicast).
Database is expected to contain valid data for the netmask and mac of
target computers.
og_cmd_get_client_setup is querying the database for computers disk
setup information. This includes the size of a disk or a partition,
which are stored as BIGINT in the database.
Using dbi_result_get_int will result in an error as libdbi expects to
store this kind of data type in a long long instead of an int.
Fix disk size variable declaration and use the correct dbi_result_get
function for this particular data column.
UMA reports that the setup command (in queue mode) does not work.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The setup command has a new field "ttp" since commit 0dd3edd, however,
the ogServer legacy parser was not updated to use this new field.
This patch adds legacy setup support to work with GPT tables. Add new field
table type to legacy setup that expects a string with "MSDOS" or "GPT".
UMA reports that the restore image command (in queue mode) does not work,
the non-queue mode works fine though.
WebConsole stores queued commands in the database using the deprecated
SocketHidra legacy string format and ogServer parses them with sscanf().
The restore-image command has the field "ptc" which stores a string
with whitespaces, however, ogServer expects no whitespaces ("%s").
Update parser to read until carriage return, whitespaces included.
Some distros enable *.* in syslog.conf to add all logging information
to /var/log/syslog. The existing LOG_DEBUG syslog() entries that ogserver
generates fill up the log files very quickly. Remove most of the LOG_DEBUG
syslog() calls.
When executing og_cmd_post_modes ogServer is looking up for the template
related to the mode coming as a parameter. This lookup is only done for
the bios boot mode. (ie. /opt/opengnsys/tftpboot/menu.lst/templates/)
Templates can be created for a given boot mode, for example you can
create a UEFI-only template (it.
/opt/opengnsys/tftpboot/grub/templates).
When a UEFI only template was coming as a parameter to og_cmd_post_modes
the file can't be located because the UEFI template folder is never
tested.
Add UEFI folder to the lookup of pxe templates, try this folder if bios
mode fails.
PS: Later on a bash script "setclientmode" is to be executed, which will
take its own way of updating these pxe files (ie. templates/../), this
script updates every boot mode if an available template is found.
OpenGnSys Support Team
Javier Sánchez Parra
Jose M. Guisado
Diego Crespo Quinta
Isabel Arrans