Merge pull request 'refs #1848 decorate oglive methods' (#27) from decorare-oglive-methods into main

Reviewed-on: #27

CHANGELOG.md

@@ -0,0 +1,289 @@
+
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [4.0.0] - 2025-04-24
+
+### Added
+
+- Authn/authz to the oglive agent
+
+## [3.3.0] - 2025-04-14
+
+### Added
+
+- Log stuff to a new json log
+
+## [3.2.0] - 2025-04-10
+
+### Added
+
+- Operating system: periodically ping ogcore
+
+## [3.1.0] - 2025-04-07
+
+### Added
+
+- Oglive: periodically ping ogcore
+
+## [3.0.0] - 2025-03-31
+
+### Changed
+
+- Ignore module provided in the URLs to the API
+
+## [2.0.0] - 2025-03-26
+
+### Changed
+
+- EjecutarScript/ConsolaRemota: expect "scp" parameter encoded in base64
+
+## [1.7.1] - 2025-03-25
+
+### Fixed
+
+- Make cfg2obj more robust
+
+## [1.7.0] - 2025-03-21
+
+### Removed
+
+- Delete the new "ptt" parameter. It's not needed.
+
+## [1.6.0] - 2025-03-12
+
+### Changed
+
+- Don't invoke bash code for some functionalities
+
+## [1.5.0] - 2025-03-12
+
+### Changed
+
+- Accept new "ptt" parameter in /ogAdmCli/Configurar
+
+### Removed
+
+- No longer recognise the unused "che" parameter in /ogAdmCli/Configurar
+
+## [1.4.9] - 2025-02-20
+
+### Changed
+
+- Notify ogcore when agent shuts down within oglive
+
+## [1.4.8] - 2025-02-18
+
+### Changed
+
+- Optionally return disk config in /status
+
+## [1.4.7] - 2025-02-04
+
+### Changed
+
+- Merge server modules
+
+### Added
+
+- Track the progress of children
+
+## [1.4.6] - 2025-01-14
+
+### Changed
+
+- Point to the new menu browser
+
+## [1.4.5] - 2024-11-29
+
+### Added
+
+- Kill long running jobs in oglive
+
+## [1.4.5~pre8] - 2024-11-27
+
+### Added
+
+- Add Configurar() to the CloningEngine module
+
+## [1.4.5~pre7] - 2024-11-20
+
+### Changed
+
+- Use old browser again
+
+## [1.4.5~pre6] - 2024-11-20
+
+### Changed
+
+- Do not use envvars for the operating-system module
+
+## [1.4.5~pre5] - 2024-11-18
+
+### Fixed
+
+- Avoid some KeyErrors
+
+## [1.4.5~pre4] - 2024-11-15
+
+### Fixed
+
+- Don't die when ogcore returns HTTP 4xx or 5xx
+
+### Changed
+
+- Get ogcore IP and port from the environment
+
+## [1.4.5~pre3] - 2024-11-06
+- Kill long running jobs in oglive (not-yet-working draft)
+
+## [1.4.5~pre2] - 2024-11-06
+
+### Fixed
+
+- Remove race condition due to several monitoring threads
+
+### Changed
+
+- Include job_id in asynchronous responses
+
+### Removed
+
+- Remove vim swapfiles from the package contents
+
+## [1.4.5~pre1] - 2024-11-06
+
+### Changed
+
+- CrearImagen: return inventory inline
+
+## [1.4.4] - 2024-10-17
+
+### Fixed
+
+- Use logger.debug() to prevent the windows agent from dying
+
+### Changed
+
+- Make status() call synchronous
+
+## [1.4.3] - 2024-10-17
+
+### Changed
+
+- Use new OGBrowser
+
+## [1.4.2] - 2024-10-15
+
+### Added
+
+- Have ogAdmClient/status return information about network and disks
+
+## [1.4.1] - 2024-10-11
+
+### Fixed
+
+- Bugfix: move data structure to the right class
+
+## [1.4.0] - 2024-10-11
+- Add more functionality
+
+### Changed
+
+- Begin using semantic versioning
+
+## [1.3.8] - 2024-10-01
+
+### Added
+
+- Add more functionality to the ogAdmClient module
+
+## [1.3.7] - 2024-09-27
+
+### Added
+
+- CloningEngine: RESTfully keep a list of long-running jobs
+
+## [1.3.6] - 2024-09-19
+
+### Added
+
+- Add more functionality to the ogAdmClient module
+- Add CloningEngine module
+
+## [1.3.5] - 2024-08-29
+
+### Changed
+
+- Don't unconditionally load modules--dynamically load everything
+
+### Removed
+
+- Remove old, unused code
+
+## [1.3.4] - 2024-07-30
+
+### Added
+
+- Implement JobMgr
+
+## [1.3.1] - 2024-06-26
+
+### Changed
+
+- Migrate the update script from shell to python
+- pyinstaller: include the 'img' subdir
+- take icons from 'img'
+
+## [1.3.0-2] - 2024-04-25
+
+### Fixed
+
+- Add missing dependency on zenity
+
+## [1.3.0] - 2024-04-25
+
+### Changed
+
+- Upgrade to Qt 6
+
+## [1.2.0] - 2020-05-4
+
+### Changed
+
+- Python 3 and Qt 5 compatibility
+
+## [1.1.1b] - 2020-02-7
+
+### Changed
+
+- Use python-distro to detect the distribution version
+
+## [1.1.1] - 2019-05-23
+
+### Changed
+
+- Set connection timeout
+- Compatibility with "Exam Mode" from the University of Seville
+
+## [1.1.0a] - 2019-05-22
+
+### Fixed
+
+- Fix a bug when activating the agent with some network devices
+
+## [1.1.0] - 2016-10-13
+
+### Changed
+
+- Functional OpenGnsys Agent interacting with OpenGnsys Server 1.1.0
+
+## [1.0.0] - 2015-07-18
+
+### Added
+
+- Initial release for OpenGnsys Agent

linux/debian/changelog

@@ -1,3 +1,197 @@
+ogagent (4.0.0-1) stable; urgency=medium
+
+  * Handle authn/authz in the oglive agent
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 24 Apr 2025 13:28:57 +0200
+
+ogagent (3.3.0-1) stable; urgency=medium
+
+  * Create an additional json log file
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 14 Apr 2025 13:50:32 +0200
+
+ogagent (3.2.0-1) stable; urgency=medium
+
+  * Operating system: periodically ping ogcore
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 10 Apr 2025 11:37:35 +0200
+
+ogagent (3.1.0-1) stable; urgency=medium
+
+  * Oglive: periodically ping ogcore
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 07 Apr 2025 11:50:05 +0200
+
+ogagent (3.0.0-1) stable; urgency=medium
+
+  * Ignore module provided in the URLs to the API
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 31 Mar 2025 10:16:07 +0200
+
+ogagent (2.0.0-1) stable; urgency=medium
+
+  * EjecutarScript/ConsolaRemota: expect "scp" parameter encoded in base64
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 26 Mar 2025 10:40:14 +0100
+
+ogagent (1.7.1-1) stable; urgency=medium
+
+  * Make cfg2obj more robust
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 25 Mar 2025 13:31:33 +0100
+
+ogagent (1.7.0-1) stable; urgency=medium
+
+  * Delete the new "ptt" parameter. It's not needed.
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 21 Mar 2025 14:19:56 +0100
+
+ogagent (1.6.0-1) stable; urgency=medium
+
+  * Don't invoke bash code for some functionalities
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 12 Mar 2025 11:59:36 +0100
+
+ogagent (1.5.0-1) stable; urgency=medium
+
+  * Accept new "ptt" parameter in /ogAdmCli/Configurar
+  * No longer recognise the unused "che" parameter in /ogAdmCli/Configurar
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 12 Mar 2025 11:45:37 +0100
+
+ogagent (1.4.9-1) stable; urgency=medium
+
+  * Notify ogcore when agent shuts down within oglive
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 20 Feb 2025 11:58:29 +0100
+
+ogagent (1.4.8-1) stable; urgency=medium
+
+  * Optionally return disk config in /status
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 18 Feb 2025 13:48:54 +0100
+
+ogagent (1.4.7-1) stable; urgency=medium
+
+  * Merge server modules
+  * Track the progress of children
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 04 Feb 2025 14:12:19 +0100
+
+ogagent (1.4.6-1) stable; urgency=medium
+
+  * Point to the new menu browser
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 14 Jan 2025 12:00:24 +0100
+
+ogagent (1.4.5-1) stable; urgency=medium
+
+  * Kill long running jobs in oglive
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 29 Nov 2024 10:22:36 +0100
+
+ogagent (1.4.5~pre8-1) stable; urgency=medium
+
+  * Add Configurar() to the CloningEngine module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 27 Nov 2024 20:02:42 +0100
+
+ogagent (1.4.5~pre7-1) stable; urgency=medium
+
+  * Use old browser again
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 20 Nov 2024 14:24:44 +0100
+
+ogagent (1.4.5~pre6-1) stable; urgency=medium
+
+  * Do not use envvars for the operating-system module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 20 Nov 2024 13:45:21 +0100
+
+ogagent (1.4.5~pre5-1) stable; urgency=medium
+
+  * Avoid some KeyErrors
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 18 Nov 2024 12:14:27 +0100
+
+ogagent (1.4.5~pre4-1) stable; urgency=medium
+
+  * Don't die when ogcore returns HTTP 4xx or 5xx
+  * Get ogcore IP and port from the environment
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 15 Nov 2024 11:43:01 +0100
+
+ogagent (1.4.5~pre3-1) stable; urgency=medium
+
+  * Kill long running jobs in oglive (not-yet-working draft)
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 14:11:32 +0100
+
+ogagent (1.4.5~pre2-1) stable; urgency=medium
+
+  * Remove race condition due to several monitoring threads
+  * Include job_id in asynchronous responses
+  * Remove vim swapfiles from the package contents
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 13:24:03 +0100
+
+ogagent (1.4.5~pre1-1) stable; urgency=medium
+
+  * CrearImagen: return inventory inline
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 12:41:14 +0100
+
+ogagent (1.4.4-1) stable; urgency=medium
+
+  * Use logger.debug() to prevent the windows agent from dying
+  * Make status() call synchronous
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 17 Oct 2024 19:13:58 +0200
+
+ogagent (1.4.3-1) stable; urgency=medium
+
+  * Use new OGBrowser
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 17 Oct 2024 09:43:08 +0200
+
+ogagent (1.4.2-1) stable; urgency=medium
+
+  * Have ogAdmClient/status return information about network and disks
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 15 Oct 2024 10:35:16 +0200
+
+ogagent (1.4.1-1) stable; urgency=medium
+
+  * Bugfix: move data structure to the right class
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 11 Oct 2024 13:51:55 +0200
+
+ogagent (1.4.0-1) stable; urgency=medium
+
+  * Add more functionality
+  * Begin using semantic versioning
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 11 Oct 2024 13:06:51 +0200
+
+ogagent (1.3.8-1) stable; urgency=medium
+
+  * Add more functionality to the ogAdmClient module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 01 Oct 2024 13:41:48 +0200
+
+ogagent (1.3.7-1) stable; urgency=medium
+
+  * CloningEngine: RESTfully keep a list of long-running jobs
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 27 Sep 2024 18:03:16 +0200
+
+ogagent (1.3.6-1) stable; urgency=medium
+
+  * Add more functionality to the ogAdmClient module
+  * Add CloningEngine module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 19 Sep 2024 13:28:17 +0200
+
 ogagent (1.3.5-1) stable; urgency=medium
 
   * Don't unconditionally load modules--dynamically load everything

linux/debian/rules

@@ -22,6 +22,7 @@ install: build
 	dh_prep
 	dh_installdirs
 	$(MAKE) DESTDIR=$(CURDIR)/debian/ogagent install-ogagent
+	find $(CURDIR) -name '*.swp' -exec rm -f '{}' ';'
 binary-arch: build install
 	# emptyness
 binary-indep: build install

ogcore-mock.py

@@ -3,6 +3,7 @@ import os
 import logging
 import json
 import subprocess
+import base64
 
 ## FLASK_APP=/path/to/ogcore-mock.py FLASK_ENV=development FLASK_RUN_CERT=adhoc sudo --preserve-env flask run --host 192.168.1.249 --port 443
 
@@ -14,30 +15,28 @@ logging.basicConfig(level=logging.INFO)
 
 @app.route('/opengnsys/rest/ogagent/<cucu>', methods=['POST'])
 def og_agent(cucu):
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     return jsonify({})
 
 
 
-## agente oglive
+## agente oglive: modulo ogAdmClient
 
-@app.route('/opengnsys/rest/__ogAdmClient/InclusionCliente', methods=['POST'])
+@app.route('/opengnsys/rest/ogAdmClient/InclusionCliente', methods=['POST'])
 def inclusion_cliente():
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     #procesoInclusionCliente() or { return (jsonify { 'res': 0 }) }
     j = request.get_json(force=True)
     iph = j['iph']  ## Toma ip
     cfg = j['cfg']  ## Toma configuracion
-    logging.info(f"iph ({iph}) cfg ({cfg})")
+    logging.info(f'iph ({iph}) cfg ({cfg})')
 
     # dbi->query (sprintf "SELECT ordenadores.*,aulas.idaula,centros.idcentro FROM ordenadores INNER JOIN aulas ON aulas.idaula=ordenadores.idaula INNER JOIN centros ON centros.idcentro=aulas.idcentro WHERE ordenadores.ip = '%s'", iph);
     # if (!dbi_result_next_row(result)) { log_error ('client does not exist in database') }
     # log_debug (sprintf 'Client %s requesting inclusion', iph);
 
     idordenador     = 42              #dbi_result_get_uint(result, "idordenador")
-    nombreordenador = "hal9000"       #dbi_result_get_string(result, "nombreordenador");
+    nombreordenador = 'hal9000'       #dbi_result_get_string(result, "nombreordenador");
     cache           = 42              #dbi_result_get_uint(result, "cache");
     idproautoexec   = 42              #dbi_result_get_uint(result, "idproautoexec");
     idaula          = 42              #dbi_result_get_uint(result, "idaula");
@@ -79,21 +78,20 @@ def _recorreProcedimientos(parametros, fileexe, idp):
 
     return 1
 
-@app.route('/opengnsys/rest/__ogAdmClient/AutoexecCliente', methods=['POST'])
+@app.route('/opengnsys/rest/ogAdmClient/AutoexecCliente', methods=['POST'])
 def autoexec_client():
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     j = request.get_json(force=True)
     iph = j['iph']  ## Toma dirección IP del cliente
     exe = j['exe']  ## Toma identificador del procedimiento inicial
-    logging.info(f"iph ({iph}) exe ({exe})")
+    logging.info(f'iph ({iph}) exe ({exe})')
 
     fileautoexec = '/tmp/Sautoexec-{}'.format(iph)
-    logging.info ("fileautoexec ({})".format (fileautoexec));
+    logging.info ('fileautoexec ({})'.format (fileautoexec));
     try:
         fileexe = open (fileautoexec, 'w')
     except Exception as e:
-        logging.error ("cannot create temporary file: {}".format (e))
+        logging.error ('cannot create temporary file: {}'.format (e))
         return jsonify({})
 
     if (_recorreProcedimientos ('', fileexe, exe)):
@@ -104,15 +102,16 @@ def autoexec_client():
     fileexe.close()
     return res
 
-@app.route('/opengnsys/rest/__ogAdmClient/enviaArchivo', methods=['POST'])
+@app.route('/opengnsys/rest/ogAdmClient/enviaArchivo', methods=['POST'])
 def envia_archivo():
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     j = request.get_json(force=True)
     nfl = j['nfl']  ## Toma nombre completo del archivo
-    logging.info(f"nfl ({nfl})")
+    logging.info(f'nfl ({nfl})')
 
-    return jsonify({'contents': subprocess.run (['cat', nfl], capture_output=True).stdout.decode('utf-8')})
+    contents = subprocess.run (['cat', nfl], capture_output=True).stdout
+    b64 = base64.b64encode (contents).decode ('utf-8')
+    return jsonify({'contents': b64})
 
 def clienteExistente(iph):
     ## esto queda totalmente del lado del servidor, no lo implemento en python
@@ -130,19 +129,18 @@ def buscaComandos(ido):
     ## convertirlo a json, aqui lo pongo a capon
     #return jsonify ({ 'nfn': 'popup', 'title': 'my title', 'message': 'my message', 'ids': ids })
 
-@app.route('/opengnsys/rest/__ogAdmClient/ComandosPendientes', methods=['POST'])
+@app.route('/opengnsys/rest/ogAdmClient/ComandosPendientes', methods=['POST'])
 def comandos_pendientes():
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     j = request.get_json(force=True)
     iph = j['iph']  ## Toma dirección IP
     ido = j['ido']  ## Toma identificador del ordenador
-    logging.info(f"iph ({iph}) ido ({ido})")
+    logging.info(f'iph ({iph}) ido ({ido})')
 
     idx = clienteExistente(iph)      ## Busca índice del cliente
     if not idx:
         ## que devuelvo?? pongamos un 404...
-        abort(404, "Client does not exist")
+        abort(404, 'Client does not exist')
 
     param = buscaComandos(ido)       ## Existen comandos pendientes, buscamos solo uno
     if param is None:
@@ -152,46 +150,89 @@ def comandos_pendientes():
 
     return jsonify(param)
 
-@app.route('/opengnsys/rest/__ogAdmClient/DisponibilidadComandos', methods=['POST'])
+@app.route('/opengnsys/rest/ogAdmClient/DisponibilidadComandos', methods=['POST'])
 def disponibilidad_comandos():
-    c = request
-    logging.info(f"{request.get_json()}")
+    logging.info(f'{request.get_json()}')
     j = request.get_json(force=True)
     iph = j['iph']
     tpc = j['tpc']
-    logging.info(f"iph ({iph}) tpc ({tpc})")
+    logging.info(f'iph ({iph}) tpc ({tpc})')
 
     idx = clienteExistente(iph)      ## Busca índice del cliente
     if not idx:
         ## que devuelvo?? pongamos un 404...
-        abort(404, "Client does not exist")
+        abort(404, 'Client does not exist')
 
     #strcpy(tbsockets[idx].estado, tpc);        ## esto queda totalmente del lado del servidor, no lo implemento en python
 
     return jsonify({})
 
-@app.route('/opengnsys/rest/__ogAdmClient/<cucu>', methods=['POST'])
-def cucu(cucu):
-    c = request
-    j = c.get_json(force=True)
-    logging.info(f"{request.get_json()} {j}")
-    if 'cucu' not in j:
-        abort(400, "missing parameter 'cucu'")
+@app.route('/opengnsys/rest/ogAdmClient/recibeArchivo', methods=['POST'])
+def oac_recibe_archivo():
+    logging.info(f'{request.get_json()}')
+    j = request.get_json(force=True)
+    nfl      = j['nfl']
+    contents = j['contents']
+    logging.info(f'nfl ({nfl}) contents ({contents})')
+    dec = base64.b64decode (contents).decode ('utf-8')
+    logging.info(f'dec ({dec})')
+    return jsonify({'anything':'anything'})   ## if we return {}, then we trigger "if not {}" which happens to be true
+
+@app.route('/opengnsys/rest/clients/status/webhook', methods=['POST'])
+def oac_callback():
+    logging.info(f'{request.get_json()}')
+    return jsonify({'anything':'anything'})
+
+@app.route('/opengnsys/rest/ogAdmClient/<cucu>', methods=['GET', 'POST'])
+def oac_cucu(cucu):
+    #j = request.get_json(force=True)
+    #logging.info(f'{request.get_json()} {j}')
+    #if 'cucu' not in j:
+    #    abort(400, 'missing parameter 'cucu'')
+    #return jsonify({'cucu': j['cucu']})
+    abort (404)
+
+
+
+## agente oglive: modulo CloningEngine
+
+@app.route('/opengnsys/rest/CloningEngine/recibeArchivo', methods=['POST'])
+def ce_recibe_archivo():
+    logging.info(f'{request.get_json()}')
+    j = request.get_json(force=True)
+    nfl      = j['nfl']
+    contents = j['contents']
+    logging.info(f'nfl ({nfl}) contents ({contents})')
+    dec = base64.b64decode (contents).decode ('utf-8')
+    logging.info(f'dec ({dec})')
+    return jsonify({'anything':'anything'})   ## if we return {}, then we trigger "if not {}" which happens to be true
+
+@app.route('/opengnsys/rest/CloningEngine/callback', methods=['POST'])
+def ce_callback():
+    logging.info(f'{request.get_json()}')
+    return jsonify({'anything':'anything'})
+
+@app.route('/opengnsys/rest/CloningEngine/<cucu>', methods=['GET', 'POST'])
+def ce_cucu(cucu):
+    abort (404)
+
 
-    return jsonify({'cucu': j['cucu']})
 
 @app.errorhandler(404)
 def _page_not_found(e):
-    return render_template_string('''<!DOCTYPE html><html>not found</html>''')
+    if type(e.description) is dict:
+        return jsonify (e.description), e.code
+    else:
+        return render_template_string('''<!DOCTYPE html><html>not found</html>'''), e.code
 
 @app.errorhandler(500)
 def _internal_server_error(e):
-    return render_template_string('''<!DOCTYPE html><html>err</html>''')
+    return render_template_string('''<!DOCTYPE html><html>err</html>'''), e.code
 
 @app.errorhandler(Exception)
 def _exception(e):
     print(e)
-    return render_template_string('''<!DOCTYPE html><html>exception</html>''')
+    return render_template_string('''<!DOCTYPE html><html>exception</html>'''), e.code
 
 if __name__ == '__main__':
     app.run(host = '192.168.1.249', port = 443, debug=True)

src/OGAgent.spec

@@ -13,7 +13,7 @@ ogausr_a = Analysis(
 #        ('cfg', 'cfg'),      ## add the entire directory
         ('img', 'img'),      ## add the entire directory
     ],
-    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib'],
+    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],
@@ -26,7 +26,7 @@ ogasvc_a = Analysis(
     pathex=[],
     binaries=[],
     datas=[],
-    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib'],
+    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],

src/OGAgentUser.py

@@ -210,7 +210,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 valid_mods.append(mod)
             except Exception as e:
                 logger.exception()
-                logger.error("Activation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
+                logger.debug ("Activation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
         self.modules[:] = valid_mods  # copy instead of assignment
         # If this is running, it's because he have logged in, inform service of this fact
         self.ipc.sendLogin((operations.getCurrentUser(), operations.getSessionLanguage(),
@@ -223,7 +223,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 mod.deactivate()
             except Exception as e:
                 logger.exception()
-                logger.error("Deactivation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
+                logger.debug ("Deactivation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
 
     def timerFnc(self):
         pass
@@ -236,7 +236,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
             logger.debug('msg: {}, {}'.format(type(msg), msg))
             module, message, data = msg
         except Exception as e:
-            logger.error('Got exception {} processing message {}'.format(e, msg))
+            logger.debug ('Got exception {} processing message {}'.format(e, msg))
             return
 
         for v in self.modules:
@@ -246,9 +246,9 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                     v.processMessage(message, json.loads(data))
                     return
                 except Exception as e:
-                    logger.error('Got exception {} processing generic message on {}'.format(e, v.name))
+                    logger.debug ('Got exception {} processing generic message on {}'.format(e, v.name))
 
-        logger.error('Module {} not found, messsage {} not sent'.format(module, message))
+        logger.debug ('Module {} not found, messsage {} not sent'.format(module, message))
 
     ## when is this run??
     def executeScript(self, script):
@@ -271,7 +271,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 self.deinitialize()
             except Exception:
                 logger.exception()
-                logger.error('Got exception deinitializing modules')
+                logger.debug ('Got exception deinitializing modules')
 
             try:
                 # If we close Client, send Logoff to Broker
@@ -282,10 +282,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
             except Exception:
                 # May we have lost connection with server, simply log and exit in that case
                 logger.exception()
-                #  File "/home/nati/Downloads/work/opengnsys/ogagent/src/OGAgentUser.py", line 286, in cleanup
-                #      logger.exception("Got an exception, processing quit")
-                #TypeError: Logger.exception() takes 1 positional argument but 2 were given
-                #logger.exception("Got an exception, processing quit")
+                logger.debug ('Got an exception, processing quit')
 
             try:
                 # operations.logoff()  # Uncomment this after testing to logoff user
@@ -319,7 +316,7 @@ if __name__ == '__main__':
         trayIcon = OGASystemTray(app)
     except Exception as e:
         logger.exception()
-        logger.error('OGA Service is not running, or it can\'t contact with OGA Server. User Tools stopped: {}'.format(
+        logger.debug ('OGA Service is not running, or it can\'t contact with OGA Server. User Tools stopped: {}'.format(
             utils.exceptionToMessage(e)))
         sys.exit(1)
 
@@ -327,7 +324,7 @@ if __name__ == '__main__':
         trayIcon.initialize()  # Initialize modules, etc..
     except Exception as e:
         logger.exception()
-        logger.error('Exception initializing OpenGnsys User Agent {}'.format(utils.exceptionToMessage(e)))
+        logger.debug ('Exception initializing OpenGnsys User Agent {}'.format(utils.exceptionToMessage(e)))
         trayIcon.quit()
         sys.exit(1)
 

src/VERSION

@@ -1 +1 @@
-1.3.5
+4.0.0

src/cfg/ogagent.cfg

@@ -7,7 +7,7 @@ port=8000
 #path=test_modules/server,more_modules/server
 
 # Remote OpenGnsys Service
-remote=https://192.168.2.10/opengnsys/rest
+remote=https://192.168.2.1/opengnsys/rest
 # Alternate OpenGnsys Service (comment out to enable this option)
 #altremote=https://10.0.2.2/opengnsys/rest
 
@@ -22,12 +22,9 @@ log=DEBUG
 # This section will be passes on activation to module
 [ogAdmClient]
 #path=test_modules/server,more_modules/server
-## this URL will probably be left equal to the other one, but let's see
-remote=https://192.168.2.10/opengnsys/rest
-log=DEBUG
 
-#servidorAdm=192.168.2.1
-#puerto=2008
+remote={}://{}/opengnsys/rest
+log=DEBUG
 pathinterface=/opt/opengnsys/interfaceAdm
-urlMenu=https://192.168.2.1/opengnsys/varios/menubrowser.php
+urlMenu={}://{}/menu-browser
 urlMsg=http://localhost/cgi-bin/httpd-log.sh

src/opengnsys/RESTApi.py

@@ -58,8 +58,12 @@ class ConnectionError(RESTError):
 # Disable warnings log messages
 try:
     import urllib3  # @UnusedImport
+    requests_log = logging.getLogger ('urllib3')
+    requests_log.setLevel (logging.INFO)
 except Exception:
     from requests.packages import urllib3  # @Reimport
+    requests_log = logging.getLogger ('requests.packages.urllib3')
+    requests_log.setLevel (logging.INFO)
 
 try:
     urllib3.disable_warnings()  # @UndefinedVariable
@@ -142,9 +146,15 @@ class REST(object):
                 else:
                     r = requests.post(url, data=data, headers={'content-type': 'application/json'})
 
+            r.raise_for_status()
+            ct = r.headers['Content-Type']
+            if 'application/json' != ct:
+                raise Exception (f'response content-type is not "application/json" but "{ct}"')
             r = json.loads(r.content)  # Using instead of r.json() to make compatible with old requests lib versions
         except requests.exceptions.RequestException as e:
-            raise ConnectionError(e)
+            code = e.response.status_code
+            logger.warning (f'request failed, HTTP code "{code}"')
+            return None
         except Exception as e:
             raise ConnectionError(exceptionToMessage(e))
 
@@ -156,12 +166,17 @@ class REST(object):
         @param data: if None or omitted, message will be a GET, else it will send a POST
         @param processData: if True, data will be serialized to json before sending, else, data will be sent as "raw" 
         """
-        logger.debug('Invoking post message {} with data {}'.format(msg, data))
+        #logger.debug('Invoking post message {} with data {}'.format(msg, data))
 
         if processData and data is not None:
             data = json.dumps(data)
 
         url = self._getUrl(msg)
-        logger.debug('Requesting {}'.format(url))
+        #logger.debug('Requesting {}'.format(url))
 
-        return self._request(url, data)
+        try:
+            res = self._request(url, data)
+            return res
+        except:
+            logger.exception()
+            return None

src/opengnsys/httpserver.py

@@ -30,6 +30,7 @@
 """
 
 
+import os
 import json
 import ssl
 import threading
@@ -77,8 +78,11 @@ class HTTPServerHandler(BaseHTTPRequestHandler):
         except Exception:
             params = {}
 
+        ## quick override because universities do not actually want the module to be extracted out of the URL
+        module = 'ogAdmClient' if os.path.exists ('/scripts/oginit') else 'opengnsys'
+
         for v in self.service.modules:
-            if v.name == path[0]:  # Case Sensitive!!!!
+            if v.name == module:  # Case Sensitive!!!!
                 return v, path[1:], params
             
         return None, path, params
@@ -88,11 +92,32 @@ class HTTPServerHandler(BaseHTTPRequestHandler):
         Locates witch module will process the message based on path (first folder on url path)
         """
         try:
+            if module is None:
+                raise Exception ({ '_httpcode': 404, '_msg': f'Module {path[0]} not found' })
             data = module.processServerMessage(path, get_params, post_params, self)
             self.sendJsonResponse(data)
         except Exception as e:
             logger.exception()
-            self.sendJsonError(500, exceptionToMessage(e))
+            n_args = len (e.args)
+            if 0 == n_args:
+                logger.debug ('Empty exception raised from message processor for "{}"'.format(path[0]))
+                self.sendJsonError(500, exceptionToMessage(e))
+            else:
+                arg0 = e.args[0]
+                if type (arg0) is str:
+                    logger.debug ('Message processor for "{}" returned exception string "{}"'.format(path[0], str(e)))
+                    self.sendJsonError (500, exceptionToMessage(e))
+                elif type (arg0) is dict:
+                    if '_httpcode' in arg0:
+                        logger.debug ('Message processor for "{}" returned HTTP code "{}" with exception string "{}"'.format(path[0], str(arg0['_httpcode']), str(arg0['_msg'])))
+                        self.sendJsonError (arg0['_httpcode'], arg0['_msg'])
+                    else:
+                        logger.debug ('Message processor for "{}" returned exception dict "{}" with no HTTP code'.format(path[0], str(e)))
+                        self.sendJsonError (500, exceptionToMessage(e))
+                else:
+                    logger.debug ('Message processor for "{}" returned non-string and non-dict exception "{}", type "{}"'.format(path[0], str(e), type(e)))
+                    self.sendJsonError (500, exceptionToMessage(e))
+            ## not reached
             
     def do_GET(self):
         module, path, params = self.parseUrl()

src/opengnsys/linux/log.py

@@ -34,6 +34,8 @@ import logging
 import os
 import tempfile
 
+from ..log_format import JsonFormatter
+
 # Logging levels
 OTHER, DEBUG, INFO, WARN, ERROR, FATAL = (10000 * (x + 1) for x in range(6))
 
@@ -48,15 +50,25 @@ class LocalLogger(object):
 
         for logDir in ('/var/log', os.path.expanduser('~'), tempfile.gettempdir()):
             try:
-                fname = os.path.join(logDir, 'opengnsys.log')
-                logging.basicConfig(
-                    filename=fname,
-                    filemode='a',
-                    format='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s',
-                    level=logging.DEBUG
-                )
-                self.logger = logging.getLogger('opengnsys')
-                os.chmod(fname, 0o0600)
+                fname1 = os.path.join (logDir, 'opengnsys.log')
+                fmt1 = logging.Formatter (fmt='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s')
+                fh1 = logging.FileHandler (filename=fname1, mode='a')
+                fh1.setFormatter (fmt1)
+                fh1.setLevel (logging.DEBUG)
+
+                fname2 = os.path.join (logDir, 'opengnsys.json.log')
+                fmt2 = JsonFormatter ({"timestamp": "asctime", "severity": "levelname", "threadName": "threadName", "function": "funcName", "message": "message"}, time_format='%Y-%m-%d %H:%M:%S', msec_format='')
+                fh2 = logging.FileHandler (filename=fname2, mode='a')
+                fh2.setFormatter (fmt2)
+                fh2.setLevel (logging.DEBUG)
+
+                self.logger = logging.getLogger ('opengnsys')
+                self.logger.setLevel (logging.DEBUG)
+                self.logger.addHandler (fh1)
+                self.logger.addHandler (fh2)
+
+                os.chmod (fname1, 0o0600)
+                os.chmod (fname2, 0o0600)
                 return
             except Exception:
                 pass

src/opengnsys/loader.py

@@ -43,6 +43,8 @@ from opengnsys.workers import ServerWorker
 from opengnsys.workers import ClientWorker
 from .log import logger
 
+PY3_12 = sys.version_info[0:2] >= (3, 12)
+
 
 def loadModules(controller, client=False):
     '''
@@ -89,7 +91,11 @@ def loadModules(controller, client=False):
         for (module_loader, name, ispkg) in pkgutil.iter_modules(paths, modPath + '.'):
             if ispkg:
                 logger.debug('Found module package {}'.format(name))
-                module_loader.find_module(name).load_module(name)
+                if PY3_12:
+                    loader = module_loader.find_spec(name).loader
+                else:
+                    loader = module_loader.find_module(name)
+                loader.load_module(name)
 
     
     if controller.config.has_option('opengnsys', 'path') is True:

src/opengnsys/log.py

@@ -79,6 +79,9 @@ class Logger(object):
     def warn(self, message):
         self.log(WARN, message)
 
+    def warning(self, message):
+        self.log(WARN, message)
+
     def info(self, message):
         self.log(INFO, message)
 

src/opengnsys/log_format.py

@@ -0,0 +1,55 @@
+import json
+import logging
+
+class JsonFormatter(logging.Formatter):
+    """
+    Formatter that outputs JSON strings after parsing the LogRecord.
+
+    @param dict fmt_dict: Key: logging format attribute pairs. Defaults to {"message": "message"}.
+    @param str time_format: time.strftime() format string. Default: "%Y-%m-%dT%H:%M:%S"
+    @param str msec_format: Microsecond formatting. Appended at the end. Default: "%s.%03dZ"
+    """
+    def __init__(self, fmt_dict: dict = None, time_format: str = "%Y-%m-%dT%H:%M:%S", msec_format: str = "%s.%03dZ"):
+        self.fmt_dict = fmt_dict if fmt_dict is not None else {"message": "message"}
+        self.default_time_format = time_format
+        self.default_msec_format = msec_format
+        self.datefmt = None
+
+    def usesTime(self) -> bool:
+        """
+        Overwritten to look for the attribute in the format dict values instead of the fmt string.
+        """
+        return "asctime" in self.fmt_dict.values()
+
+    def formatMessage(self, record) -> dict:
+        """
+        Overwritten to return a dictionary of the relevant LogRecord attributes instead of a string.
+        KeyError is raised if an unknown attribute is provided in the fmt_dict.
+        """
+        return {fmt_key: record.__dict__[fmt_val] for fmt_key, fmt_val in self.fmt_dict.items()}
+
+    def format(self, record) -> str:
+        """
+        Mostly the same as the parent's class method, the difference being that a dict is manipulated and dumped as JSON
+        instead of a string.
+        """
+        record.message = record.getMessage()
+
+        if self.usesTime():
+            record.asctime = self.formatTime(record, self.datefmt)
+
+        message_dict = self.formatMessage(record)
+
+        if record.exc_info:
+            # Cache the traceback text to avoid converting it multiple times
+            # (it's constant anyway)
+            if not record.exc_text:
+                record.exc_text = self.formatException(record.exc_info)
+
+        if record.exc_text:
+            message_dict["exc_info"] = record.exc_text
+
+        if record.stack_info:
+            message_dict["stack_info"] = self.formatStack(record.stack_info)
+
+        return json.dumps(message_dict, default=str)

src/opengnsys/modules/server/OpenGnSys/__init__.py

@@ -30,7 +30,6 @@
 @author: Ramón M. Gómez, ramongomez at us dot es
 """
 
-
 import base64
 import os
 import random
@@ -65,7 +64,7 @@ def check_secret(fnc):
             else:
                 raise Exception('Unauthorized operation')
         except Exception as e:
-            logger.error(str(e))
+            logger.debug (str(e))
             raise Exception(e)
 
     return wrapper
@@ -83,7 +82,7 @@ def execution_level(level):
                 else:
                     raise Exception('Unauthorized operation')
             except Exception as e:
-                logger.error(str(e))
+                logger.debug (str(e))
                 raise Exception(e)
 
         return wrapper
@@ -102,24 +101,43 @@ class OpenGnSysWorker(ServerWorker):
     exec_level = None  # Execution level (permitted operations)
     jobmgr = JobMgr()
 
+    ## pings ogcore
+    def mon (self):
+        n = 0
+        while True:
+            time.sleep (1)
+            n += 1
+            if not n % 10:
+                body = {
+                    "iph": self.interface.ip,
+                    "timestamp": int (time.time()),
+                }
+                logger.debug (f'about to send ping ({body})')
+                self.REST.sendMessage ('clients/status/webhook', body)
+
     def onActivation(self):
         """
         Sends OGAgent activation notification to OpenGnsys server
         """
+        if os.path.exists ('/scripts/oginit'):
+            ## estamos en oglive, este modulo no debe cargarse
+            ## esta lógica la saco de src/opengnsys/linux/operations.py, donde hay un if similar
+            raise Exception ('Refusing to load within an ogLive image')
+
         e = None  # Error info
         t = 0  # Count of time
         # Generate random secret to send on activation
         self.random = ''.join(random.choice(string.ascii_lowercase + string.digits) for _ in range(self.length))
         # Ensure cfg has required configuration variables or an exception will be thrown
         try:
-            url = self.service.config.get('opengnsys', 'remote')
+            url = self.service.config.get(self.name, 'remote')
         except NoOptionError as e:
             logger.error("Configuration error: {}".format(e))
             raise e
         self.REST = REST(url)
         # Execution level ('full' by default)
         try:
-            self.exec_level = self.service.config.get('opengnsys', 'level')
+            self.exec_level = self.service.config.get(self.name, 'level')
         except NoOptionError:
             self.exec_level = 'full'
         # Get network interfaces until they are active or timeout (5 minutes)
@@ -156,7 +174,7 @@ class OpenGnSysWorker(ServerWorker):
                     logger.warn (str (e))
                     # Trying to initialize on alternative server, if defined
                     # (used in "exam mode" from the University of Seville)
-                    self.REST = REST(self.service.config.get('opengnsys', 'altremote'))
+                    self.REST = REST(self.service.config.get(self.name, 'altremote'))
                     self.REST.sendMessage('ogagent/started', {'mac': self.interface.mac, 'ip': self.interface.ip,
                                                               'secret': self.random, 'ostype': operations.os_type,
                                                               'osversion': operations.os_version, 'alt_url': True,
@@ -184,6 +202,10 @@ class OpenGnSysWorker(ServerWorker):
         if os.path.isfile(new_hosts_file):
             shutil.copyfile(new_hosts_file, hosts_file)
 
+        threading.Thread (name='monitoring_thread', target=self.mon, daemon=True).start()
+
+        logger.debug ('onActivation ok')
+
     def onDeactivation(self):
         """
         Sends OGAgent stopping notification to OpenGnsys server

src/opengnsys/service.py

@@ -158,7 +158,7 @@ class CommonService(object):
                 validMods.append(mod)
             except Exception as e:
                 logger.exception()
-                logger.error("Activation of {} failed: {}".format(mod.name, exceptionToMessage(e)))
+                logger.debug("Activation of {} failed: {}".format(mod.name, exceptionToMessage(e)))
         
         self.modules[:] = validMods  # copy instead of assignment
         

src/opengnsys/windows/log.py

@@ -36,6 +36,8 @@ import logging
 import os
 import tempfile
 
+from ..log_format import JsonFormatter
+
 # Valid logging levels, from UDS Broker (uds.core.utils.log)
 OTHER, DEBUG, INFO, WARN, ERROR, FATAL = (10000 * (x + 1) for x in range(6))
 
@@ -44,13 +46,24 @@ class LocalLogger(object):
     def __init__(self):
         # tempdir is different for "user application" and "service"
         # service wil get c:\windows\temp, while user will get c:\users\XXX\appdata\local\temp
-        logging.basicConfig(
-            filename=os.path.join(tempfile.gettempdir(), 'opengnsys.log'),
-            filemode='a',
-            format='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s',
-            level=logging.DEBUG
-        )
+
+        fname1 = os.path.join (tempfile.gettempdir(), 'opengnsys.log')
+        fmt1 = logging.Formatter (fmt='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s')
+        fh1 = logging.FileHandler (filename=fname1, mode='a')
+        fh1.setFormatter (fmt1)
+        fh1.setLevel (logging.DEBUG)
+
+        fname2 = os.path.join (tempfile.gettempdir(), 'opengnsys.json.log')
+        fmt2 = JsonFormatter ({"timestamp": "asctime", "severity": "levelname", "threadName": "threadName", "function": "funcName", "message": "message"}, time_format='%Y-%m-%d %H:%M:%S', msec_format='')
+        fh2 = logging.FileHandler (filename=fname2, mode='a')
+        fh2.setFormatter (fmt2)
+        fh2.setLevel (logging.DEBUG)
+
         self.logger = logging.getLogger('opengnsys')
+        self.logger.setLevel (logging.DEBUG)
+        self.logger.addHandler (fh1)
+        self.logger.addHandler (fh2)
+
         self.serviceLogger = False
 
     def log(self, level, message):

src/opengnsys/workers/__init__.py

@@ -1,2 +1,3 @@
 from .server_worker import ServerWorker
 from .client_worker import ClientWorker
+from .oglive_worker import ogLiveWorker, ThreadWithResult

src/opengnsys/workers/oglive_worker.py

@@ -0,0 +1,514 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (c) 2024 Qindel Formación y Servicios S.L.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without modification,
+# are permitted provided that the following conditions are met:
+#
+#    * Redistributions of source code must retain the above copyright notice,
+#      this list of conditions and the following disclaimer.
+#    * Redistributions in binary form must reproduce the above copyright notice,
+#      this list of conditions and the following disclaimer in the documentation
+#      and/or other materials provided with the distribution.
+#    * Neither the name of Virtual Cable S.L. nor the names of its contributors
+#      may be used to endorse or promote products derived from this software
+#      without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+"""
+@author: Natalia Serrano, nserrano at qindel dot com
+"""
+# pylint: disable=unused-wildcard-import,wildcard-import
+
+import os
+import re
+import time
+import random
+import subprocess
+import threading
+import signal
+
+from configparser import NoOptionError
+from opengnsys import REST
+from opengnsys.log import logger
+from .server_worker import ServerWorker
+
+## https://stackoverflow.com/questions/6893968/how-to-get-the-return-value-from-a-thread
+class ThreadWithResult (threading.Thread):
+    def run (self):
+        try:
+            self.result = None
+            if self._target is not None:
+                ## the first arg in self._args is the queue
+                self.pid_q    = self._args[0]
+                self.stdout_q = self._args[1]
+                self._args    = self._args[2:]
+                try:
+                    self.result = self._target (*self._args, **self._kwargs)
+                except Exception as e:
+                    self.result = { 'res': 2, 'der': f'got exception: ({e})' }    ## res=2 as defined in ogAdmClient.c:2048
+        finally:
+            # Avoid a refcycle if the thread is running a function with an argument that has a member that points to the thread.
+            del self._target, self._args, self._kwargs, self.pid_q, self.stdout_q
+
+class ogLiveWorker(ServerWorker):
+    thread_list = {}
+    thread_lock = threading.Lock()
+
+    tbErroresScripts = [
+        "Se han generado errores desconocidos. No se puede continuar la ejecución de este módulo",        ## 0
+		"001-Formato de ejecución incorrecto.",
+		"002-Fichero o dispositivo no encontrado",
+		"003-Error en partición de disco",
+		"004-Partición o fichero bloqueado",
+		"005-Error al crear o restaurar una imagen",
+		"006-Sin sistema operativo",
+		"007-Programa o función BOOLEAN no ejecutable",
+		"008-Error en la creación del archivo de eco para consola remota",
+		"009-Error en la lectura del archivo temporal de intercambio",
+		"010-Error al ejecutar la llamada a la interface de administración",
+		"011-La información retornada por la interface de administración excede de la longitud permitida",
+		"012-Error en el envío de fichero por la red",
+		"013-Error en la creación del proceso hijo",
+		"014-Error de escritura en destino",
+		"015-Sin Cache en el Cliente",
+		"016-No hay espacio en la cache para almacenar fichero-imagen",
+		"017-Error al Reducir el Sistema Archivos",
+		"018-Error al Expandir el Sistema Archivos",
+		"019-Valor fuera de rango o no válido.",
+		"020-Sistema de archivos desconocido o no se puede montar",
+		"021-Error en partición de caché local",
+		"022-El disco indicado no contiene una particion GPT",
+		"023-Error no definido",
+		"024-Error no definido",
+		"025-Error no definido",
+		"026-Error no definido",
+		"027-Error no definido",
+		"028-Error no definido",
+		"029-Error no definido",
+		"030-Error al restaurar imagen - Imagen mas grande que particion",
+		"031-Error al realizar el comando updateCache",
+		"032-Error al formatear",
+		"033-Archivo de imagen corrupto o de otra versión de partclone",
+		"034-Error no definido",
+		"035-Error no definido",
+		"036-Error no definido",
+		"037-Error no definido",
+		"038-Error no definido",
+		"039-Error no definido",
+		"040-Error imprevisto no definido",
+		"041-Error no definido",
+		"042-Error no definido",
+		"043-Error no definido",
+		"044-Error no definido",
+		"045-Error no definido",
+		"046-Error no definido",
+		"047-Error no definido",
+		"048-Error no definido",
+		"049-Error no definido",
+		"050-Error en la generación de sintaxis de transferenica unicast",
+		"051-Error en envio UNICAST de una particion",
+		"052-Error en envio UNICAST de un fichero",
+		"053-Error en la recepcion UNICAST de una particion",
+		"054-Error en la recepcion UNICAST de un fichero",
+		"055-Error en la generacion de sintaxis de transferenica Multicast",
+		"056-Error en envio MULTICAST de un fichero",
+		"057-Error en la recepcion MULTICAST de un fichero",
+		"058-Error en envio MULTICAST de una particion",
+		"059-Error en la recepcion MULTICAST de una particion",
+		"060-Error en la conexion de una sesion UNICAST|MULTICAST con el MASTER",
+		"061-Error no definido",
+		"062-Error no definido",
+		"063-Error no definido",
+		"064-Error no definido",
+		"065-Error no definido",
+		"066-Error no definido",
+		"067-Error no definido",
+		"068-Error no definido",
+		"069-Error no definido",
+		"070-Error al montar una imagen sincronizada.",
+		"071-Imagen no sincronizable (es monolitica).",
+		"072-Error al desmontar la imagen.",
+		"073-No se detectan diferencias entre la imagen basica y la particion.",
+		"074-Error al sincronizar, puede afectar la creacion/restauracion de la imagen.",
+		"Error desconocido",
+	]
+
+    def notifier (self, job_id, result):
+        result['job_id'] = job_id
+        self.REST.sendMessage ('clients/status/webhook', result)
+
+    def killer (self, job_id):
+        logger.debug (f'killer() called, job_id ({job_id})')
+        if job_id not in self.thread_list: return { 'res': 2, 'der': 'Unknown job' }
+
+        with self.thread_lock:
+            if 'thread' not in self.thread_list[job_id]: return { 'res': 2, 'der': 'Job is not running' }
+            t   = self.thread_list[job_id]['thread']
+            pid = self.thread_list[job_id]['child_pid']
+            logger.debug (f'pid ({pid})')
+            try_times = 8
+            sig = signal.SIGTERM
+            msg = f'could not kill pid ({pid}) after ({try_times}) tries'
+            success = 2    ## mimic cmd['res'] in respuestaEjecucionComando(): "1" means success, "2" means failed
+            while True:
+                t.join (0.05)
+                if not t.is_alive():
+                    msg = 'job terminated'
+                    success = 1
+                    logger.debug (msg)
+                    self.thread_list[job_id]['child_pid'] = None
+                    break
+                ## race condition: if the subprocess finishes just here, then we already checked that t.is_alive() is true, but os.path.exists(/proc/pid) will be false below. msg will be 'nothing to kill'.
+                ## this is fine in the first iteration of the loop, before we send any signals. In the rest of iterations, after some signals were sent, msg should be 'job terminated' instead.
+                if pid:
+                    if os.path.exists (f'/proc/{pid}'):
+                        logger.debug (f'sending signal ({sig}) to pid ({pid})')
+                        ## if the process finishes just here, nothing happens: the signal is sent to the void
+                        os.kill (pid, sig)
+                        #subprocess.run (['kill', '--signal', str(sig), str(pid)])
+                    else:
+                        msg = f'pid ({pid}) is gone, nothing to kill'
+                        success = 1
+                        logger.debug (msg)
+                        self.thread_list[job_id]['child_pid'] = None
+                        break
+                else:
+                    msg = 'no PID to kill'
+                    logger.debug (msg)
+
+                if not try_times: break
+                if 4 == try_times: sig = signal.SIGKILL   ## change signal after a few tries
+                try_times -= 1
+                time.sleep (0.4)
+
+        return { 'res':success, 'der':msg }
+
+    def _extract_progress (self, job_id, ary=[]):
+        progress = None
+        for i in ary:
+            if m := re.search (r'^\[([0-9]+)\]', i):     ## look for strings like '[10]', '[60]'
+                #logger.debug (f"matched regex, m.groups ({m.groups()})")
+                progress = float (m.groups()[0]) / 100
+        return progress
+
+    ## monitors child threads, waits for them to finish
+    ## pings ogcore
+    def mon (self):
+        n = 0
+        while True:
+            with self.thread_lock:
+                for k in self.thread_list:
+                    elem = self.thread_list[k]
+                    if 'thread' not in elem: continue
+                    #logger.debug (f'considering thread ({k})')
+
+                    if self.pid_q:
+                        if not self.pid_q.empty():
+                            elem['child_pid'] = self.pid_q.get()
+                            logger.debug (f'queue not empty, got pid ({elem["child_pid"]})')
+
+                    if self.stdout_q:
+                        partial = ''
+                        while not self.stdout_q.empty():
+                            partial += self.stdout_q.get()
+                        lines = partial.splitlines()
+                        if len (lines):
+                            p = self._extract_progress (k, lines)
+                            if p:
+                                m = { "job_id": k, "progress": p }
+                                self.REST.sendMessage ('clients/status/webhook', { "job_id": k, "progress": p })
+
+                    elem['thread'].join (0.05)
+                    if not elem['thread'].is_alive():
+                        logger.debug (f'is no longer alive, k ({k}) thread ({elem["thread"]})')
+                        elem['running'] = False
+                        elem['result'] = elem['thread'].result
+                        del elem['thread']
+                        self.notifier (k, elem['result'])
+
+            time.sleep (1)
+            n += 1
+            if not n % 10:
+                alive_threads = []
+                for k in self.thread_list:
+                    elem = self.thread_list[k]
+                    if 'thread' not in elem: continue
+                    alive_threads.append (k)
+                if alive_threads:
+                    s = ','.join (alive_threads)
+                    logger.debug (f'alive threads: {s}')
+
+                body = {
+                    'iph': self.IPlocal,
+                    'timestamp': int (time.time()),
+                }
+                #logger.debug (f'about to send ping ({body})')
+                self.REST.sendMessage ('clients/status/webhook', body)
+
+    def interfaceAdmin (self, method, parametros=[]):
+        if method in ['Apagar', 'CambiarAcceso', 'Configurar', 'CrearImagen', 'EjecutarScript', 'getConfiguration', 'getIpAddress', 'IniciarSesion', 'InventarioHardware', 'InventarioSoftware', 'Reiniciar', 'RestaurarImagen']:
+            ## python
+            logger.debug (f'({method}) is a python method')
+            exe = '{}/{}.py'.format (self.pathinterface, method)
+            proc = [exe]+parametros
+        else:  ## ConsolaRemota procesaCache
+            ## bash
+            logger.debug (f'({method}) is a bash method')
+            exe = '{}/{}'.format (self.pathinterface, method)
+
+            LANG = os.environ.get ('LANG', 'en_GB.UTF-8').replace ('UTF_8', 'UTF-8')
+            devel_bash_prefix = f'''
+                PATH=/opt/opengnsys/scripts/:$PATH;
+                source /opt/opengnsys/etc/lang.{LANG}.conf;
+                for I in /opt/opengnsys/lib/engine/bin/*.lib; do source $I; done;
+                for i in $(declare -F |cut -f3 -d" "); do export -f $i; done;
+            '''
+
+            if parametros:
+                proc = ['bash', '-c', '{} {} {}'.format (devel_bash_prefix, exe, ' '.join (parametros))]
+            else:
+                proc = ['bash', '-c', '{} {}'.format (devel_bash_prefix, exe)]
+
+        logger.debug ('subprocess.run ("{}")'.format (' '.join (proc)))
+        p = subprocess.Popen (proc, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        if self.pid_q:
+            self.pid_q.put (p.pid)
+        else:
+            ## esto sucede por ejemplo cuando arranca el agente, que estamos en interfaceAdmin() en el mismo hilo, sin _long_running_job ni hilo separado
+            #logger.debug ('no queue--not writing any PID to it')
+            pass
+
+        sout = serr = ''
+        while p.poll() is None:
+            for l in iter (p.stdout.readline, b''):
+                partial = l.decode ('utf-8', 'ignore')
+                if self.stdout_q: self.stdout_q.put (partial)
+                sout += partial
+            for l in iter (p.stderr.readline, b''):
+                partial = l.decode ('utf-8', 'ignore')
+                serr += partial
+            time.sleep (1)
+        sout = sout.strip()
+        serr = serr.strip()
+
+        ## DEBUG
+        logger.debug (f'stdout follows:')
+        for l in sout.splitlines():
+            logger.debug (f'  {l}')
+        #logger.debug (f'stderr follows:')
+        #for l in serr.splitlines():
+        #    logger.debug (f'  {l}')
+        ## /DEBUG
+        if 0 != p.returncode:
+            cmd_txt = ' '.join (proc)
+            logger.error (f'command ({cmd_txt}) failed, stderr follows:')
+            for l in serr.splitlines():
+                logger.error (f'  {l}')
+            raise Exception (f'command ({cmd_txt}) failed, see log for details')
+        return sout
+
+    def tomaIPlocal (self):
+        try:
+            self.IPlocal = self.interfaceAdmin ('getIpAddress')
+        except Exception as e:
+            logger.error (e)
+            logger.error ('No se ha podido recuperar la dirección IP del cliente')
+            return False
+        logger.info ('local IP is "{}"'.format (self.IPlocal))
+        return True
+
+    def tomaMAClocal (self):
+        ## tomaIPlocal() calls interfaceAdm('getIpAddress')
+        ## getIpAddress runs 'ip addr show' and returns the IP address of every network interface except "lo"
+        ## (ie. breaks badly if there's more than one network interface)
+        ## let's make the same assumptions here
+        mac = subprocess.run (["ip -json link show |jq -r '.[] |select (.ifname != \"lo\") |.address'"], shell=True, capture_output=True, text=True)
+        self.mac = mac.stdout.strip()
+        return True
+
+    def enviaMensajeServidor (self, path, obj={}):
+        obj['iph'] = self.IPlocal          ## Ip del ordenador
+        obj['ido'] = self.idordenador      ## Identificador del ordenador
+        obj['npc'] = self.nombreordenador  ## Nombre del ordenador
+        obj['idc'] = self.idcentro         ## Identificador del centro
+        obj['ida'] = self.idaula           ## Identificador del aula
+
+        res = self.REST.sendMessage ('/'.join ([self.name, path]), obj)
+
+        if (type (res) is not dict):
+            logger.error (f'response is not a dict ({res})')
+            return False
+
+        return res
+
+    ## en C, esto envia una trama de respuesta al servidor. Devuelve un boolean
+    ## en python, simplemente termina de construir la respuesta y la devuelve; no envía nada por la red. El caller la usa en return() para enviar implícitamente la respuesta
+    def respuestaEjecucionComando (self, cmd, herror, ids=None):
+        if ids:                 ## Existe seguimiento
+            cmd['ids'] = ids    ## Añade identificador de la sesión
+
+        if 0 == herror:  ## el comando terminó con resultado satisfactorio
+            cmd['res'] = 1
+            cmd['der'] = ''
+        else:            ## el comando tuvo algún error
+            cmd['res'] = 2
+            cmd['der'] = self.tbErroresScripts[herror]
+
+        return cmd
+
+    def cargaPaginaWeb (self, url=None):
+        if (not url): url = self.urlMenu
+        os.system ('pkill -9 browser')
+
+        p = subprocess.Popen (['/usr/bin/browser', '-qws', url])
+        try:
+            p.wait (2)       ## if the process dies before 2 seconds...
+            logger.error ('Error al ejecutar browser, return code "{}"'.format (p.returncode))
+            return False
+        except subprocess.TimeoutExpired:
+            pass
+
+        return True
+
+    def muestraMenu (self):
+        self.cargaPaginaWeb()
+
+    def muestraMensaje (self, idx):
+        self.cargaPaginaWeb (f'{self.urlMsg}?idx={idx}')
+
+    def LeeConfiguracion (self):
+        try:
+            parametroscfg = self.interfaceAdmin ('getConfiguration')   ## Configuración de los Sistemas Operativos del cliente
+        except Exception as e:
+            logger.error (e)
+            logger.error ('No se ha podido recuperar la dirección IP del cliente')
+            return None
+        #logger.debug ('parametroscfg ({})'.format (parametroscfg))
+        return parametroscfg
+
+    def cfg2obj (self, cfg):
+        obj = []
+        ptrPar = cfg.split ('\n')
+        for line in ptrPar:
+            elem = {}
+            ptrCfg = line.split ('\t')
+
+            for item in ptrCfg:
+                if '=' not in item:
+                    logger.warning (f'invalid item ({item})')
+                    continue
+                k, v = item.split ('=', maxsplit=1)
+                elem[k] = v
+
+            obj.append (elem)
+
+        return obj
+
+    def onActivation (self):
+        if not os.path.exists ('/scripts/oginit'):
+            ## no estamos en oglive, este modulo no debe cargarse
+            ## esta lógica la saco de src/opengnsys/linux/operations.py, donde hay un if similar
+            raise Exception ('Refusing to load within an operating system')
+
+        self.pathinterface   = None
+        self.IPlocal         = None     ## Ip del ordenador
+        self.mac             = None     ## MAC del ordenador
+        self.idordenador     = None     ## Identificador del ordenador
+        self.nombreordenador = None     ## Nombre del ordenador
+        self.cache           = None
+        self.idproautoexec   = None
+        self.idcentro        = None     ## Identificador del centro
+        self.idaula          = None     ## Identificador del aula
+        self.pid_q           = None     ## for passing PIDs around
+        self.stdout_q        = None     ## for passing stdout
+        self.progress_jobs   = {}
+
+        ogcore_scheme   = os.environ.get ('OGAGENTCFG_OGCORE_SCHEME',  'https')
+        ogcore_ip       = os.environ.get ('OGAGENTCFG_OGCORE_IP',      '192.168.2.1')
+        ogcore_port     = os.environ.get ('OGAGENTCFG_OGCORE_PORT',    '8443')
+        urlmenu_scheme  = os.environ.get ('OGAGENTCFG_URLMENU_SCHEME', 'https')
+        urlmenu_ip      = os.environ.get ('OGAGENTCFG_URLMENU_IP',     '192.168.2.1')
+        urlmenu_port    = os.environ.get ('OGAGENTCFG_URLMENU_PORT',   '8443')
+        ogcore_ip_port  = ':'.join (map (str, filter (None, [ogcore_ip,  ogcore_port ])))
+        urlmenu_ip_port = ':'.join (map (str, filter (None, [urlmenu_ip, urlmenu_port])))
+        try:
+            url                = self.service.config.get (self.name, 'remote')
+            loglevel           = self.service.config.get (self.name, 'log')
+            self.pathinterface = self.service.config.get (self.name, 'pathinterface')
+            self.urlMenu       = self.service.config.get (self.name, 'urlMenu')
+            self.urlMsg        = self.service.config.get (self.name, 'urlMsg')
+
+            url          = url.format          (ogcore_scheme,  ogcore_ip_port)
+            self.urlMenu = self.urlMenu.format (urlmenu_scheme, urlmenu_ip_port)
+        except NoOptionError as e:
+            logger.error ("Configuration error: {}".format (e))
+            raise e
+        logger.setLevel (loglevel)
+        self.REST = REST (url)
+
+        if not self.tomaIPlocal():
+            raise Exception ('Se han generado errores. No se puede continuar la ejecución de este módulo')
+
+        if not self.tomaMAClocal():
+            raise Exception ('Se han generado errores. No se puede continuar la ejecución de este módulo')
+
+        threading.Thread (name='monitoring_thread', target=self.mon, daemon=True).start()
+
+    def _long_running_job (self, name, f, args):
+        any_job_running = False
+        for k in self.thread_list:
+            if self.thread_list[k]['running']:
+                any_job_running = True
+                break
+        if any_job_running:
+            logger.info ('some job is already running, refusing to launch another one')
+            return { 'job_id': None, 'message': 'some job is already running, refusing to launch another one' }
+
+        job_id = '{}-{}'.format (name, ''.join (random.choice ('0123456789abcdef') for _ in range (8)))
+        import queue
+        self.pid_q    = queue.Queue()   ## a single queue works for us because we never have more than one long_running_job at the same time
+        self.stdout_q = queue.Queue()
+        self.thread_list[job_id] = {
+            'thread': ThreadWithResult (target=f, args=(self.pid_q, self.stdout_q) + args),
+            'starttime': time.time(),
+            'child_pid': None,
+            'running': True,
+            'result': None
+        }
+        self.thread_list[job_id]['thread'].start()
+        return { 'job_id': job_id }
+
+## para matar threads tengo lo siguiente:
+## - aqui en _long_running_job meto una cola en self.pid_q
+## - (self.pid_q fue inicializado a None al instanciar el objeto, para evitar error "objeto no tiene 'pid_q'")
+## - en el thread_list también tengo un child_pid para almacenar el pid de los procesos hijos
+## - al crear el ThreadWithResult le paso la cola, y luego en run() la recojo y la meto en el self.pid_q del thread
+## - en interfaceAdmin() al hacer subprocess.Popen(), recojo el pid y lo escribo en la queue
+## - en mon() recojo pids de la queue y los meto en thread_list 'child_pid'
+##   - algunas funciones llaman a interfaceAdmin más de una vez, y escriben más de un pid en la cola, y en mon() voy recogiendo y actualizando
+##     - por ejemplo EjecutarScript llama a interfaceAdmin() y luego llama a LeeConfiguracion() el cual llama a interfaceAdmin() otra vez
+## - y cuando nos llamen a KillJob, terminamos en killer() el cual coge el 'child_pid' y zas
+##   - pero a lo mejor el child ya terminó
+##   - o a lo mejor el KillJob nos llegó demasiado pronto y todavía no hubo ningún child
+##
+## $ curl --insecure -X POST --data '{"nfn":"EjecutarScript","scp":"cd /usr; sleep 30; pwd; ls","ids":"0"}' https://192.168.2.199:8000/ogAdmClient/EjecutarScript
+## {"job_id": "EjecutarScript-333feb3f"}
+## $ curl --insecure -X POST --data '{"job_id":"EjecutarScript-333feb3f"}' https://192.168.2.199:8000/ogAdmClient/KillJob
+##
+## funciona bien, excepto que el PID no muere xD, ni siquiera haciendo subprocess.run('kill')
+
+## para mostrar el progreso de los jobs reutilizo la misma infra
+## una cola self.stdout_q
+## en interfaceAdmin escribo la stdout parcial que ya venia recogiendo
+## mon() lo recoge y le hace un POST a ogcore

src/opengnsys/workers/server_worker.py

@@ -96,8 +96,8 @@ class ServerWorker(object):
             return self.process(getParams, postParams, server)
         try:
             operation = getattr(self, 'process_' + path[0])
-        except Exception:
-            raise Exception('Message processor for "{}" not found'.format(path[0]))
+        except:
+            raise Exception ({ '_httpcode': 404, '_msg': f'{path[0]}: method not found' })
         
         return operation(path[1:], getParams, postParams, server)