Merge pull request 'refs #1848 decorate oglive methods' (#27) from decorare-oglive-methods into main

Reviewed-on: #27

CHANGELOG.md

@@ -0,0 +1,289 @@
+
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [4.0.0] - 2025-04-24
+
+### Added
+
+- Authn/authz to the oglive agent
+
+## [3.3.0] - 2025-04-14
+
+### Added
+
+- Log stuff to a new json log
+
+## [3.2.0] - 2025-04-10
+
+### Added
+
+- Operating system: periodically ping ogcore
+
+## [3.1.0] - 2025-04-07
+
+### Added
+
+- Oglive: periodically ping ogcore
+
+## [3.0.0] - 2025-03-31
+
+### Changed
+
+- Ignore module provided in the URLs to the API
+
+## [2.0.0] - 2025-03-26
+
+### Changed
+
+- EjecutarScript/ConsolaRemota: expect "scp" parameter encoded in base64
+
+## [1.7.1] - 2025-03-25
+
+### Fixed
+
+- Make cfg2obj more robust
+
+## [1.7.0] - 2025-03-21
+
+### Removed
+
+- Delete the new "ptt" parameter. It's not needed.
+
+## [1.6.0] - 2025-03-12
+
+### Changed
+
+- Don't invoke bash code for some functionalities
+
+## [1.5.0] - 2025-03-12
+
+### Changed
+
+- Accept new "ptt" parameter in /ogAdmCli/Configurar
+
+### Removed
+
+- No longer recognise the unused "che" parameter in /ogAdmCli/Configurar
+
+## [1.4.9] - 2025-02-20
+
+### Changed
+
+- Notify ogcore when agent shuts down within oglive
+
+## [1.4.8] - 2025-02-18
+
+### Changed
+
+- Optionally return disk config in /status
+
+## [1.4.7] - 2025-02-04
+
+### Changed
+
+- Merge server modules
+
+### Added
+
+- Track the progress of children
+
+## [1.4.6] - 2025-01-14
+
+### Changed
+
+- Point to the new menu browser
+
+## [1.4.5] - 2024-11-29
+
+### Added
+
+- Kill long running jobs in oglive
+
+## [1.4.5~pre8] - 2024-11-27
+
+### Added
+
+- Add Configurar() to the CloningEngine module
+
+## [1.4.5~pre7] - 2024-11-20
+
+### Changed
+
+- Use old browser again
+
+## [1.4.5~pre6] - 2024-11-20
+
+### Changed
+
+- Do not use envvars for the operating-system module
+
+## [1.4.5~pre5] - 2024-11-18
+
+### Fixed
+
+- Avoid some KeyErrors
+
+## [1.4.5~pre4] - 2024-11-15
+
+### Fixed
+
+- Don't die when ogcore returns HTTP 4xx or 5xx
+
+### Changed
+
+- Get ogcore IP and port from the environment
+
+## [1.4.5~pre3] - 2024-11-06
+- Kill long running jobs in oglive (not-yet-working draft)
+
+## [1.4.5~pre2] - 2024-11-06
+
+### Fixed
+
+- Remove race condition due to several monitoring threads
+
+### Changed
+
+- Include job_id in asynchronous responses
+
+### Removed
+
+- Remove vim swapfiles from the package contents
+
+## [1.4.5~pre1] - 2024-11-06
+
+### Changed
+
+- CrearImagen: return inventory inline
+
+## [1.4.4] - 2024-10-17
+
+### Fixed
+
+- Use logger.debug() to prevent the windows agent from dying
+
+### Changed
+
+- Make status() call synchronous
+
+## [1.4.3] - 2024-10-17
+
+### Changed
+
+- Use new OGBrowser
+
+## [1.4.2] - 2024-10-15
+
+### Added
+
+- Have ogAdmClient/status return information about network and disks
+
+## [1.4.1] - 2024-10-11
+
+### Fixed
+
+- Bugfix: move data structure to the right class
+
+## [1.4.0] - 2024-10-11
+- Add more functionality
+
+### Changed
+
+- Begin using semantic versioning
+
+## [1.3.8] - 2024-10-01
+
+### Added
+
+- Add more functionality to the ogAdmClient module
+
+## [1.3.7] - 2024-09-27
+
+### Added
+
+- CloningEngine: RESTfully keep a list of long-running jobs
+
+## [1.3.6] - 2024-09-19
+
+### Added
+
+- Add more functionality to the ogAdmClient module
+- Add CloningEngine module
+
+## [1.3.5] - 2024-08-29
+
+### Changed
+
+- Don't unconditionally load modules--dynamically load everything
+
+### Removed
+
+- Remove old, unused code
+
+## [1.3.4] - 2024-07-30
+
+### Added
+
+- Implement JobMgr
+
+## [1.3.1] - 2024-06-26
+
+### Changed
+
+- Migrate the update script from shell to python
+- pyinstaller: include the 'img' subdir
+- take icons from 'img'
+
+## [1.3.0-2] - 2024-04-25
+
+### Fixed
+
+- Add missing dependency on zenity
+
+## [1.3.0] - 2024-04-25
+
+### Changed
+
+- Upgrade to Qt 6
+
+## [1.2.0] - 2020-05-4
+
+### Changed
+
+- Python 3 and Qt 5 compatibility
+
+## [1.1.1b] - 2020-02-7
+
+### Changed
+
+- Use python-distro to detect the distribution version
+
+## [1.1.1] - 2019-05-23
+
+### Changed
+
+- Set connection timeout
+- Compatibility with "Exam Mode" from the University of Seville
+
+## [1.1.0a] - 2019-05-22
+
+### Fixed
+
+- Fix a bug when activating the agent with some network devices
+
+## [1.1.0] - 2016-10-13
+
+### Changed
+
+- Functional OpenGnsys Agent interacting with OpenGnsys Server 1.1.0
+
+## [1.0.0] - 2015-07-18
+
+### Added
+
+- Initial release for OpenGnsys Agent

linux/debian/changelog

@@ -1,3 +1,153 @@
+ogagent (4.0.0-1) stable; urgency=medium
+
+  * Handle authn/authz in the oglive agent
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 24 Apr 2025 13:28:57 +0200
+
+ogagent (3.3.0-1) stable; urgency=medium
+
+  * Create an additional json log file
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 14 Apr 2025 13:50:32 +0200
+
+ogagent (3.2.0-1) stable; urgency=medium
+
+  * Operating system: periodically ping ogcore
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 10 Apr 2025 11:37:35 +0200
+
+ogagent (3.1.0-1) stable; urgency=medium
+
+  * Oglive: periodically ping ogcore
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 07 Apr 2025 11:50:05 +0200
+
+ogagent (3.0.0-1) stable; urgency=medium
+
+  * Ignore module provided in the URLs to the API
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 31 Mar 2025 10:16:07 +0200
+
+ogagent (2.0.0-1) stable; urgency=medium
+
+  * EjecutarScript/ConsolaRemota: expect "scp" parameter encoded in base64
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 26 Mar 2025 10:40:14 +0100
+
+ogagent (1.7.1-1) stable; urgency=medium
+
+  * Make cfg2obj more robust
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 25 Mar 2025 13:31:33 +0100
+
+ogagent (1.7.0-1) stable; urgency=medium
+
+  * Delete the new "ptt" parameter. It's not needed.
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 21 Mar 2025 14:19:56 +0100
+
+ogagent (1.6.0-1) stable; urgency=medium
+
+  * Don't invoke bash code for some functionalities
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 12 Mar 2025 11:59:36 +0100
+
+ogagent (1.5.0-1) stable; urgency=medium
+
+  * Accept new "ptt" parameter in /ogAdmCli/Configurar
+  * No longer recognise the unused "che" parameter in /ogAdmCli/Configurar
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 12 Mar 2025 11:45:37 +0100
+
+ogagent (1.4.9-1) stable; urgency=medium
+
+  * Notify ogcore when agent shuts down within oglive
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 20 Feb 2025 11:58:29 +0100
+
+ogagent (1.4.8-1) stable; urgency=medium
+
+  * Optionally return disk config in /status
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 18 Feb 2025 13:48:54 +0100
+
+ogagent (1.4.7-1) stable; urgency=medium
+
+  * Merge server modules
+  * Track the progress of children
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 04 Feb 2025 14:12:19 +0100
+
+ogagent (1.4.6-1) stable; urgency=medium
+
+  * Point to the new menu browser
+
+ -- OpenGnsys developers <info@opengnsys.es>  Tue, 14 Jan 2025 12:00:24 +0100
+
+ogagent (1.4.5-1) stable; urgency=medium
+
+  * Kill long running jobs in oglive
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 29 Nov 2024 10:22:36 +0100
+
+ogagent (1.4.5~pre8-1) stable; urgency=medium
+
+  * Add Configurar() to the CloningEngine module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 27 Nov 2024 20:02:42 +0100
+
+ogagent (1.4.5~pre7-1) stable; urgency=medium
+
+  * Use old browser again
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 20 Nov 2024 14:24:44 +0100
+
+ogagent (1.4.5~pre6-1) stable; urgency=medium
+
+  * Do not use envvars for the operating-system module
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 20 Nov 2024 13:45:21 +0100
+
+ogagent (1.4.5~pre5-1) stable; urgency=medium
+
+  * Avoid some KeyErrors
+
+ -- OpenGnsys developers <info@opengnsys.es>  Mon, 18 Nov 2024 12:14:27 +0100
+
+ogagent (1.4.5~pre4-1) stable; urgency=medium
+
+  * Don't die when ogcore returns HTTP 4xx or 5xx
+  * Get ogcore IP and port from the environment
+
+ -- OpenGnsys developers <info@opengnsys.es>  Fri, 15 Nov 2024 11:43:01 +0100
+
+ogagent (1.4.5~pre3-1) stable; urgency=medium
+
+  * Kill long running jobs in oglive (not-yet-working draft)
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 14:11:32 +0100
+
+ogagent (1.4.5~pre2-1) stable; urgency=medium
+
+  * Remove race condition due to several monitoring threads
+  * Include job_id in asynchronous responses
+  * Remove vim swapfiles from the package contents
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 13:24:03 +0100
+
+ogagent (1.4.5~pre1-1) stable; urgency=medium
+
+  * CrearImagen: return inventory inline
+
+ -- OpenGnsys developers <info@opengnsys.es>  Wed, 06 Nov 2024 12:41:14 +0100
+
+ogagent (1.4.4-1) stable; urgency=medium
+
+  * Use logger.debug() to prevent the windows agent from dying
+  * Make status() call synchronous
+
+ -- OpenGnsys developers <info@opengnsys.es>  Thu, 17 Oct 2024 19:13:58 +0200
+
 ogagent (1.4.3-1) stable; urgency=medium
 
   * Use new OGBrowser

linux/debian/rules

@@ -22,6 +22,7 @@ install: build
 	dh_prep
 	dh_installdirs
 	$(MAKE) DESTDIR=$(CURDIR)/debian/ogagent install-ogagent
+	find $(CURDIR) -name '*.swp' -exec rm -f '{}' ';'
 binary-arch: build install
 	# emptyness
 binary-indep: build install

ogcore-mock.py

@@ -178,7 +178,7 @@ def oac_recibe_archivo():
     logging.info(f'dec ({dec})')
     return jsonify({'anything':'anything'})   ## if we return {}, then we trigger "if not {}" which happens to be true
 
-@app.route('/opengnsys/rest/ogAdmClient/callback', methods=['POST'])
+@app.route('/opengnsys/rest/clients/status/webhook', methods=['POST'])
 def oac_callback():
     logging.info(f'{request.get_json()}')
     return jsonify({'anything':'anything'})

src/OGAgent.spec

@@ -13,7 +13,7 @@ ogausr_a = Analysis(
 #        ('cfg', 'cfg'),      ## add the entire directory
         ('img', 'img'),      ## add the entire directory
     ],
-    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.CloningEngine', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
+    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],
@@ -26,7 +26,7 @@ ogasvc_a = Analysis(
     pathex=[],
     binaries=[],
     datas=[],
-    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.CloningEngine', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
+    hiddenimports=['win32timezone', 'socketserver', 'http.server', 'urllib', 'opengnsys.modules.client.OpenGnSys', 'opengnsys.modules.server.ogAdmClient', 'opengnsys.modules.server.OpenGnSys'],
     hookspath=[],
     hooksconfig={},
     runtime_hooks=[],

src/OGAgentUser.py

@@ -210,7 +210,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 valid_mods.append(mod)
             except Exception as e:
                 logger.exception()
-                logger.error("Activation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
+                logger.debug ("Activation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
         self.modules[:] = valid_mods  # copy instead of assignment
         # If this is running, it's because he have logged in, inform service of this fact
         self.ipc.sendLogin((operations.getCurrentUser(), operations.getSessionLanguage(),
@@ -223,7 +223,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 mod.deactivate()
             except Exception as e:
                 logger.exception()
-                logger.error("Deactivation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
+                logger.debug ("Deactivation of {} failed: {}".format(mod.name, utils.exceptionToMessage(e)))
 
     def timerFnc(self):
         pass
@@ -236,7 +236,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
             logger.debug('msg: {}, {}'.format(type(msg), msg))
             module, message, data = msg
         except Exception as e:
-            logger.error('Got exception {} processing message {}'.format(e, msg))
+            logger.debug ('Got exception {} processing message {}'.format(e, msg))
             return
 
         for v in self.modules:
@@ -246,9 +246,9 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                     v.processMessage(message, json.loads(data))
                     return
                 except Exception as e:
-                    logger.error('Got exception {} processing generic message on {}'.format(e, v.name))
+                    logger.debug ('Got exception {} processing generic message on {}'.format(e, v.name))
 
-        logger.error('Module {} not found, messsage {} not sent'.format(module, message))
+        logger.debug ('Module {} not found, messsage {} not sent'.format(module, message))
 
     ## when is this run??
     def executeScript(self, script):
@@ -271,7 +271,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
                 self.deinitialize()
             except Exception:
                 logger.exception()
-                logger.error('Got exception deinitializing modules')
+                logger.debug ('Got exception deinitializing modules')
 
             try:
                 # If we close Client, send Logoff to Broker
@@ -282,10 +282,7 @@ class OGASystemTray(QtWidgets.QSystemTrayIcon):
             except Exception:
                 # May we have lost connection with server, simply log and exit in that case
                 logger.exception()
-                #  File "/home/nati/Downloads/work/opengnsys/ogagent/src/OGAgentUser.py", line 286, in cleanup
-                #      logger.exception("Got an exception, processing quit")
-                #TypeError: Logger.exception() takes 1 positional argument but 2 were given
-                #logger.exception("Got an exception, processing quit")
+                logger.debug ('Got an exception, processing quit')
 
             try:
                 # operations.logoff()  # Uncomment this after testing to logoff user
@@ -319,7 +316,7 @@ if __name__ == '__main__':
         trayIcon = OGASystemTray(app)
     except Exception as e:
         logger.exception()
-        logger.error('OGA Service is not running, or it can\'t contact with OGA Server. User Tools stopped: {}'.format(
+        logger.debug ('OGA Service is not running, or it can\'t contact with OGA Server. User Tools stopped: {}'.format(
             utils.exceptionToMessage(e)))
         sys.exit(1)
 
@@ -327,7 +324,7 @@ if __name__ == '__main__':
         trayIcon.initialize()  # Initialize modules, etc..
     except Exception as e:
         logger.exception()
-        logger.error('Exception initializing OpenGnsys User Agent {}'.format(utils.exceptionToMessage(e)))
+        logger.debug ('Exception initializing OpenGnsys User Agent {}'.format(utils.exceptionToMessage(e)))
         trayIcon.quit()
         sys.exit(1)
 

src/VERSION

@@ -1 +1 @@
-1.4.3
+4.0.0

src/cfg/ogagent.cfg

@@ -23,15 +23,8 @@ log=DEBUG
 [ogAdmClient]
 #path=test_modules/server,more_modules/server
 
-remote=https://192.168.2.1/opengnsys/rest
+remote={}://{}/opengnsys/rest
 log=DEBUG
 pathinterface=/opt/opengnsys/interfaceAdm
-urlMenu=https://192.168.2.1/opengnsys/varios/menubrowser.php
-urlMsg=http://localhost/cgi-bin/httpd-log.sh
-
-[CloningEngine]
-remote=https://192.168.2.1/opengnsys/rest
-log=DEBUG
-pathinterface=/opt/opengnsys/interfaceAdm
-urlMenu=https://192.168.2.1/opengnsys/varios/menubrowser.php
+urlMenu={}://{}/menu-browser
 urlMsg=http://localhost/cgi-bin/httpd-log.sh

src/opengnsys/RESTApi.py

@@ -58,8 +58,12 @@ class ConnectionError(RESTError):
 # Disable warnings log messages
 try:
     import urllib3  # @UnusedImport
+    requests_log = logging.getLogger ('urllib3')
+    requests_log.setLevel (logging.INFO)
 except Exception:
     from requests.packages import urllib3  # @Reimport
+    requests_log = logging.getLogger ('requests.packages.urllib3')
+    requests_log.setLevel (logging.INFO)
 
 try:
     urllib3.disable_warnings()  # @UndefinedVariable
@@ -148,7 +152,9 @@ class REST(object):
                 raise Exception (f'response content-type is not "application/json" but "{ct}"')
             r = json.loads(r.content)  # Using instead of r.json() to make compatible with old requests lib versions
         except requests.exceptions.RequestException as e:
-            raise ConnectionError(e)
+            code = e.response.status_code
+            logger.warning (f'request failed, HTTP code "{code}"')
+            return None
         except Exception as e:
             raise ConnectionError(exceptionToMessage(e))
 
@@ -160,12 +166,17 @@ class REST(object):
         @param data: if None or omitted, message will be a GET, else it will send a POST
         @param processData: if True, data will be serialized to json before sending, else, data will be sent as "raw" 
         """
-        logger.debug('Invoking post message {} with data {}'.format(msg, data))
+        #logger.debug('Invoking post message {} with data {}'.format(msg, data))
 
         if processData and data is not None:
             data = json.dumps(data)
 
         url = self._getUrl(msg)
-        logger.debug('Requesting {}'.format(url))
+        #logger.debug('Requesting {}'.format(url))
 
-        return self._request(url, data)
+        try:
+            res = self._request(url, data)
+            return res
+        except:
+            logger.exception()
+            return None

src/opengnsys/httpserver.py

@@ -30,6 +30,7 @@
 """
 
 
+import os
 import json
 import ssl
 import threading
@@ -77,8 +78,11 @@ class HTTPServerHandler(BaseHTTPRequestHandler):
         except Exception:
             params = {}
 
+        ## quick override because universities do not actually want the module to be extracted out of the URL
+        module = 'ogAdmClient' if os.path.exists ('/scripts/oginit') else 'opengnsys'
+
         for v in self.service.modules:
-            if v.name == path[0]:  # Case Sensitive!!!!
+            if v.name == module:  # Case Sensitive!!!!
                 return v, path[1:], params
             
         return None, path, params
@@ -96,22 +100,22 @@ class HTTPServerHandler(BaseHTTPRequestHandler):
             logger.exception()
             n_args = len (e.args)
             if 0 == n_args:
-                logger.error ('Empty exception raised from message processor for "{}"'.format(path[0]))
+                logger.debug ('Empty exception raised from message processor for "{}"'.format(path[0]))
                 self.sendJsonError(500, exceptionToMessage(e))
             else:
                 arg0 = e.args[0]
                 if type (arg0) is str:
-                    logger.error ('Message processor for "{}" returned exception string "{}"'.format(path[0], str(e)))
+                    logger.debug ('Message processor for "{}" returned exception string "{}"'.format(path[0], str(e)))
                     self.sendJsonError (500, exceptionToMessage(e))
                 elif type (arg0) is dict:
                     if '_httpcode' in arg0:
-                        logger.warning ('Message processor for "{}" returned HTTP code "{}" with exception string "{}"'.format(path[0], str(arg0['_httpcode']), str(arg0['_msg'])))
+                        logger.debug ('Message processor for "{}" returned HTTP code "{}" with exception string "{}"'.format(path[0], str(arg0['_httpcode']), str(arg0['_msg'])))
                         self.sendJsonError (arg0['_httpcode'], arg0['_msg'])
                     else:
-                        logger.error ('Message processor for "{}" returned exception dict "{}" with no HTTP code'.format(path[0], str(e)))
+                        logger.debug ('Message processor for "{}" returned exception dict "{}" with no HTTP code'.format(path[0], str(e)))
                         self.sendJsonError (500, exceptionToMessage(e))
                 else:
-                    logger.error ('Message processor for "{}" returned non-string and non-dict exception "{}"'.format(path[0], str(e)))
+                    logger.debug ('Message processor for "{}" returned non-string and non-dict exception "{}", type "{}"'.format(path[0], str(e), type(e)))
                     self.sendJsonError (500, exceptionToMessage(e))
             ## not reached
             

src/opengnsys/linux/log.py

@@ -34,6 +34,8 @@ import logging
 import os
 import tempfile
 
+from ..log_format import JsonFormatter
+
 # Logging levels
 OTHER, DEBUG, INFO, WARN, ERROR, FATAL = (10000 * (x + 1) for x in range(6))
 
@@ -48,15 +50,25 @@ class LocalLogger(object):
 
         for logDir in ('/var/log', os.path.expanduser('~'), tempfile.gettempdir()):
             try:
-                fname = os.path.join(logDir, 'opengnsys.log')
-                logging.basicConfig(
-                    filename=fname,
-                    filemode='a',
-                    format='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s',
-                    level=logging.DEBUG
-                )
-                self.logger = logging.getLogger('opengnsys')
-                os.chmod(fname, 0o0600)
+                fname1 = os.path.join (logDir, 'opengnsys.log')
+                fmt1 = logging.Formatter (fmt='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s')
+                fh1 = logging.FileHandler (filename=fname1, mode='a')
+                fh1.setFormatter (fmt1)
+                fh1.setLevel (logging.DEBUG)
+
+                fname2 = os.path.join (logDir, 'opengnsys.json.log')
+                fmt2 = JsonFormatter ({"timestamp": "asctime", "severity": "levelname", "threadName": "threadName", "function": "funcName", "message": "message"}, time_format='%Y-%m-%d %H:%M:%S', msec_format='')
+                fh2 = logging.FileHandler (filename=fname2, mode='a')
+                fh2.setFormatter (fmt2)
+                fh2.setLevel (logging.DEBUG)
+
+                self.logger = logging.getLogger ('opengnsys')
+                self.logger.setLevel (logging.DEBUG)
+                self.logger.addHandler (fh1)
+                self.logger.addHandler (fh2)
+
+                os.chmod (fname1, 0o0600)
+                os.chmod (fname2, 0o0600)
                 return
             except Exception:
                 pass

src/opengnsys/log_format.py

@@ -0,0 +1,55 @@
+import json
+import logging
+
+class JsonFormatter(logging.Formatter):
+    """
+    Formatter that outputs JSON strings after parsing the LogRecord.
+
+    @param dict fmt_dict: Key: logging format attribute pairs. Defaults to {"message": "message"}.
+    @param str time_format: time.strftime() format string. Default: "%Y-%m-%dT%H:%M:%S"
+    @param str msec_format: Microsecond formatting. Appended at the end. Default: "%s.%03dZ"
+    """
+    def __init__(self, fmt_dict: dict = None, time_format: str = "%Y-%m-%dT%H:%M:%S", msec_format: str = "%s.%03dZ"):
+        self.fmt_dict = fmt_dict if fmt_dict is not None else {"message": "message"}
+        self.default_time_format = time_format
+        self.default_msec_format = msec_format
+        self.datefmt = None
+
+    def usesTime(self) -> bool:
+        """
+        Overwritten to look for the attribute in the format dict values instead of the fmt string.
+        """
+        return "asctime" in self.fmt_dict.values()
+
+    def formatMessage(self, record) -> dict:
+        """
+        Overwritten to return a dictionary of the relevant LogRecord attributes instead of a string.
+        KeyError is raised if an unknown attribute is provided in the fmt_dict.
+        """
+        return {fmt_key: record.__dict__[fmt_val] for fmt_key, fmt_val in self.fmt_dict.items()}
+
+    def format(self, record) -> str:
+        """
+        Mostly the same as the parent's class method, the difference being that a dict is manipulated and dumped as JSON
+        instead of a string.
+        """
+        record.message = record.getMessage()
+
+        if self.usesTime():
+            record.asctime = self.formatTime(record, self.datefmt)
+
+        message_dict = self.formatMessage(record)
+
+        if record.exc_info:
+            # Cache the traceback text to avoid converting it multiple times
+            # (it's constant anyway)
+            if not record.exc_text:
+                record.exc_text = self.formatException(record.exc_info)
+
+        if record.exc_text:
+            message_dict["exc_info"] = record.exc_text
+
+        if record.stack_info:
+            message_dict["stack_info"] = self.formatStack(record.stack_info)
+
+        return json.dumps(message_dict, default=str)

src/opengnsys/modules/server/CloningEngine/__init__.py

@@ -1,331 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-#
-# Copyright (c) 2024 Qindel Formación y Servicios S.L.
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without modification,
-# are permitted provided that the following conditions are met:
-#
-#    * Redistributions of source code must retain the above copyright notice,
-#      this list of conditions and the following disclaimer.
-#    * Redistributions in binary form must reproduce the above copyright notice,
-#      this list of conditions and the following disclaimer in the documentation
-#      and/or other materials provided with the distribution.
-#    * Neither the name of Virtual Cable S.L. nor the names of its contributors
-#      may be used to endorse or promote products derived from this software
-#      without specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
-# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
-# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
-# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
-# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
-# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-"""
-@author: Natalia Serrano, nserrano at qindel dot com
-"""
-
-import base64
-import os
-from pathlib import Path
-
-from opengnsys.log import logger
-from opengnsys.workers import ogLiveWorker
-
-class CloningEngineWorker (ogLiveWorker):
-    name = 'CloningEngine'  # Module name
-    REST = None             # REST object
-
-    def onActivation (self):
-        super().onActivation()
-        logger.info ('onActivation ok')
-
-    def onDeactivation (self):
-        logger.debug ('onDeactivation')
-
-    def InventariandoSoftware (self, dsk, par, sw, nfn):
-        sft_src = f'/tmp/CSft-{self.IPlocal}-{par}'
-        try:
-            self.interfaceAdmin (nfn, [dsk, par, sft_src])
-            herror = 0
-        except:
-            herror = 1
-
-        if herror:
-            logger.warning ('Error al ejecutar el comando')
-            self.muestraMensaje (20)
-        else:
-            if not os.path.exists (sft_src):
-                raise Exception (f'interfaceAdmin({nfn}) returned success but did not create file ({sft_src})')
-            sft_src_contents = Path (sft_src).read_bytes()
-
-            ## Envía fichero de inventario al servidor
-            sft_dst = f'/tmp/Ssft-{self.IPlocal}-{par}'   ## Nombre que tendra el archivo en el Servidor
-            logger.debug ('sending recibeArchivo to server')
-            res = self.enviaMensajeServidor ('recibeArchivo', { 'nfl': sft_dst, 'contents': base64.b64encode (sft_src_contents).decode ('utf-8') })
-            logger.debug (res)
-            if not res:
-                herror = 12   ## Error de envío de fichero por la red
-                raise Exception ('Ha ocurrido algún problema al enviar un archivo por la red')
-            self.muestraMensaje (19)
-
-        if not sw:
-            cmd = {
-                'nfn': 'RESPUESTA_InventarioSoftware',
-                'par': par,
-                'sft': sft_dst,
-            }
-            return self.respuestaEjecucionComando (cmd, herror, 0)
-
-        return {'true':'true'}           ## XXX
-
-    def do_CrearImagen (self, post_params):
-        for k in ['dsk', 'par', 'cpt', 'idi', 'nci', 'ipr', 'nfn', 'ids']:
-            if k not in post_params:
-                logger.error (f'required parameter ({k}) not in POST params')
-                return {}
-
-        dsk = post_params['dsk'] ## Disco
-        par = post_params['par'] ## Número de partición
-        cpt = post_params['cpt'] ## Código de la partición
-        idi = post_params['idi'] ## Identificador de la imagen
-        nci = post_params['nci'] ## Nombre canónico de la imagen
-        ipr = post_params['ipr'] ## Ip del repositorio
-        nfn = post_params['nfn']
-        ids = post_params['ids']
-
-        self.muestraMensaje (7)
-
-        try:
-            res = self.InventariandoSoftware (dsk, par, False, 'InventarioSoftware')   ## Crea inventario Software previamente
-        except:
-            logger.warning ('Error al ejecutar el comando')
-            return {}
-
-        if res:
-            self.muestraMensaje (2)
-            try:
-                self.interfaceAdmin (nfn, [dsk, par, nci, ipr])
-                self.muestraMensaje (9)
-                herror = 0
-            except:
-                logger.warning ('Error al ejecutar el comando')
-                self.muestraMensaje (10)
-                herror = 1
-        else:
-            logger.warning ('Error al ejecutar el comando')
-
-        self.muestraMenu()
-
-        cmd = {
-            'nfn': 'RESPUESTA_CrearImagen',
-            'idi': idi,    ## Identificador de la imagen
-            'dsk': dsk,    ## Número de disco
-            'par': par,    ## Número de partición de donde se creó
-            'cpt': cpt,    ## Tipo o código de partición
-            'ipr': ipr,    ## Ip del repositorio donde se alojó
-        }
-        return self.respuestaEjecucionComando (cmd, herror, ids)
-
-    def do_RestaurarImagen (self, post_params):
-        for k in ['dsk', 'par', 'idi', 'ipr', 'nci', 'ifs', 'ptc', 'nfn', 'ids']:
-            if k not in post_params:
-                logger.error (f'required parameter ({k}) not in POST params')
-                return {}
-
-        dsk = post_params['dsk']
-        par = post_params['par']
-        idi = post_params['idi']
-        ipr = post_params['ipr']
-        nci = post_params['nci']
-        ifs = post_params['ifs']
-        ptc = post_params['ptc']    ## Protocolo de clonación: Unicast, Multicast, Torrent
-        nfn = post_params['nfn']
-        ids = post_params['ids']
-
-        self.muestraMensaje (3)
-
-        try:
-            self.interfaceAdmin (nfn, [dsk, par, nci, ipr, ptc])
-            self.muestraMensaje (11)
-            herror = 0
-        except:
-            logger.warning ('Error al ejecutar el comando')
-            self.muestraMensaje (12)
-            herror = 1
-
-        cfg = self.LeeConfiguracion()
-        if not cfg:
-            logger.warning ('No se ha podido recuperar la configuración de las particiones del disco')
-
-        self.muestraMenu()
-
-        cmd = {
-            'nfn': 'RESPUESTA_RestaurarImagen',
-            'idi': idi,                  ## Identificador de la imagen
-            'dsk': dsk,                  ## Número de disco
-            'par': par,                  ## Número de partición
-            'ifs': ifs,                  ## Identificador del perfil software
-            'cfg': self.cfg2obj(cfg),    ## Configuración de discos
-        }
-        return self.respuestaEjecucionComando (cmd, herror, ids)
-
-    def process_status (self, path, get_params, post_params, server):
-        thr_status = {}
-        for k in self.thread_list:
-            thr_status[k] = {
-                'running': self.thread_list[k]['running'],
-                'result': self.thread_list[k]['result'],
-            }
-        return thr_status
-
-    def do_Configurar (self, post_params):
-        for k in ['nfn', 'dsk', 'cfg', 'ids']:
-            if k not in post_params:
-                logger.error (f'required parameter ({k}) not in POST params')
-                return {}
-
-        nfn = post_params['nfn']
-        dsk = post_params['dsk']
-        cfg = post_params['cfg'].replace('\n','$').replace('\t','#')
-        ids = post_params['ids']
-
-        self.muestraMensaje (4)
-
-        try:
-            self.interfaceAdmin (nfn, [dsk, cfg])
-            self.muestraMensaje (14)
-            herror = 0
-        except:
-            logger.warning ('Error al ejecutar el comando')
-            self.muestraMensaje (13)
-            herror = 1
-
-        cfg = self.LeeConfiguracion()
-        if not cfg:
-            logger.warning ('No se ha podido recuperar la configuración de las particiones del disco')
-            return {}
-
-        cmd = {
-            'nfn': 'RESPUESTA_Configurar',
-            'cfg': self.cfg2obj (cfg),
-        }
-        self.muestraMenu()
-        return self.respuestaEjecucionComando (cmd, herror, ids)
-
-    def do_InventarioHardware (self, post_params):
-        for k in ['nfn', 'ids']:
-            if k not in post_params:
-                logger.error (f'required parameter ({k}) not in POST params')
-                return {}
-
-        nfn = post_params['nfn']
-        ids = post_params['ids']
-
-        self.muestraMensaje (6)
-
-        hrdsrc = f'/tmp/Chrd-{self.IPlocal}'      ## Nombre que tendra el archivo de inventario
-        hrddst = f'/tmp/Shrd-{self.IPlocal}'      ## Nombre que tendra el archivo en el Servidor
-        try:
-            self.interfaceAdmin (nfn, [hrdsrc])
-            hrdsrc_contents = Path (hrdsrc).read_bytes()
-            logger.debug (f'hrdsrc_contents 1 ({hrdsrc_contents})')
-            herror = 0
-        except:
-            logger.warning ('Error al ejecutar el comando')
-            self.muestraMensaje (18)
-            herror = 1
-
-        if herror:
-            hrddst = ''
-        else:
-            logger.debug (f'hrdsrc_contents 2 ({hrdsrc_contents})')
-            ## Envía fichero de inventario al servidor
-            res = self.enviaMensajeServidor ('recibeArchivo', { 'nfl': hrddst, 'contents': base64.b64encode (hrdsrc_contents).decode ('utf-8') })
-            logger.debug (res)
-            if not res:
-                logger.error ('Ha ocurrido algún problema al enviar un archivo por la red')
-                herror = 12     ## Error de envío de fichero por la red
-            self.muestraMensaje (17)
-
-        ## Envia respuesta de ejecución de la función de interface
-        cmd = {
-            'nfn': 'RESPUESTA_InventarioHardware',
-            'hrd': hrddst,
-        }
-        self.muestraMenu()
-        return self.respuestaEjecucionComando (cmd, herror, ids)
-
-    def do_InventarioSoftware (self, post_params):
-        for k in ['nfn', 'dsk', 'par', 'ids']:
-            if k not in post_params:
-                logger.error (f'required parameter ({k}) not in POST params')
-                return {}
-
-        nfn = post_params['nfn']
-        dsk = post_params['dsk']
-        par = post_params['par']
-        ids = post_params['ids']
-
-        self.muestraMensaje (7)
-
-        try:
-            self.InventariandoSoftware (dsk, par, True, 'InventarioSoftware')
-            herror = 0
-        except:
-            logger.warning ('Error al ejecutar el comando')
-            herror = 1
-
-        self.muestraMenu()
-        cmd = {
-            'nfn': 'RESPUESTA_InventarioSoftware',
-        }
-        return self.respuestaEjecucionComando (cmd, herror, ids)
-
-    def process_CrearImagen (self, path, get_params, post_params, server):
-        logger.debug ('in process_CrearImagen, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
-        return self._long_running_job ('CrearImagen', self.do_CrearImagen, args=(post_params,))
-
-    def process_CrearImagenBasica (self, path, get_params, post_params, server):
-        logger.debug ('in process_CrearImagenBasica, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.warning ('this method has been removed')
-        raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
-
-    def process_CrearSoftIncremental (self, path, get_params, post_params, server):
-        logger.debug ('in process_CrearSoftIncremental, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.warning ('this method has been removed')
-        raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
-
-    def process_RestaurarImagen (self, path, get_params, post_params, server):
-        logger.debug ('in process_RestaurarImagen, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
-        return self._long_running_job ('RestaurarImagen', self.do_RestaurarImagen, args=(post_params,))
-
-    def process_RestaurarImagenBasica (self, path, get_params, post_params, server):
-        logger.debug ('in process_RestaurarImagenBasica, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.warning ('this method has been removed')
-        raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
-
-    def process_RestaurarSoftIncremental (self, path, get_params, post_params, server):
-        logger.warning ('in process_RestaurarSoftIncremental')
-        logger.debug ('in process_RestaurarSoftIncremental, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.warning ('this method has been removed')
-        raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
-
-    def process_Configurar (self, path, get_params, post_params, server):
-        logger.debug ('in process_Configurar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        return self._long_running_job ('Configurar', self.do_Configurar, args=(post_params,))
-
-    def process_InventarioHardware (self, path, get_params, post_params, server):
-        logger.debug ('in process_InventarioHardware, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        return self._long_running_job ('InventarioHardware', self.do_InventarioHardware, args=(post_params,))
-
-    def process_InventarioSoftware (self, path, get_params, post_params, server):
-        logger.debug ('in process_InventarioSoftware, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        return self._long_running_job ('InventarioSoftware', self.do_InventarioSoftware, args=(post_params,))

src/opengnsys/modules/server/OpenGnSys/__init__.py

@@ -30,7 +30,6 @@
 @author: Ramón M. Gómez, ramongomez at us dot es
 """
 
-
 import base64
 import os
 import random
@@ -65,7 +64,7 @@ def check_secret(fnc):
             else:
                 raise Exception('Unauthorized operation')
         except Exception as e:
-            logger.error(str(e))
+            logger.debug (str(e))
             raise Exception(e)
 
     return wrapper
@@ -83,7 +82,7 @@ def execution_level(level):
                 else:
                     raise Exception('Unauthorized operation')
             except Exception as e:
-                logger.error(str(e))
+                logger.debug (str(e))
                 raise Exception(e)
 
         return wrapper
@@ -102,6 +101,20 @@ class OpenGnSysWorker(ServerWorker):
     exec_level = None  # Execution level (permitted operations)
     jobmgr = JobMgr()
 
+    ## pings ogcore
+    def mon (self):
+        n = 0
+        while True:
+            time.sleep (1)
+            n += 1
+            if not n % 10:
+                body = {
+                    "iph": self.interface.ip,
+                    "timestamp": int (time.time()),
+                }
+                logger.debug (f'about to send ping ({body})')
+                self.REST.sendMessage ('clients/status/webhook', body)
+
     def onActivation(self):
         """
         Sends OGAgent activation notification to OpenGnsys server
@@ -189,6 +202,8 @@ class OpenGnSysWorker(ServerWorker):
         if os.path.isfile(new_hosts_file):
             shutil.copyfile(new_hosts_file, hosts_file)
 
+        threading.Thread (name='monitoring_thread', target=self.mon, daemon=True).start()
+
         logger.debug ('onActivation ok')
 
     def onDeactivation(self):

src/opengnsys/modules/server/ogAdmClient/__init__.py

@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*-
 #
 # Copyright (c) 2014 Virtual Cable S.L.
-# Copyright (c) 2024 Qindel Formación y Servicios S.L.
+# Copyright (c) 2024-2025 Qindel Formación y Servicios S.L.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without modification,
@@ -33,15 +33,15 @@
 """
 
 import base64
-#import threading
-#import time
 import os
 import signal
+import string
+import random
 import subprocess
 from pathlib import Path
 from urllib.parse import unquote
 
-#from opengnsys import operations
+from opengnsys import VERSION
 from opengnsys.log import logger
 from opengnsys.workers import ogLiveWorker
 
@@ -51,25 +51,43 @@ def check_secret (fnc):
     Decorator to check for received secret key and raise exception if it isn't valid.
     """
     def wrapper (*args, **kwargs):
-        return fnc (*args, **kwargs)
-        #try:
-        #    this, path, get_params, post_params, server = args
-        #    # Accept "status" operation with no arguments or any function with Authorization header
-        #    if fnc.__name__ == 'process_status' and not get_params:
-        #        return fnc (*args, **kwargs)
-        #    elif this.random == server.headers['Authorization']:
-        #        return fnc (*args, **kwargs)
-        #    else:
-        #        raise Exception ('Unauthorized operation')
-        #except Exception as e:
-        #    logger.error (str (e))
-        #    raise Exception (e)
+        try:
+            this, path, get_params, post_params, server = args
+
+            if not server:      ## this happens on startup, eg. onActivation->autoexecCliente->ejecutaArchivo->popup->check_secret
+                return fnc (*args, **kwargs)
+
+            if this.random == server.headers['Authorization']:
+                return fnc (*args, **kwargs)
+            else:
+                raise Exception ('Unauthorized operation')
+        except Exception as e:
+            logger.error (str (e))
+            raise Exception (e)
 
     return wrapper
 
+# Check if operation is permitted
+def execution_level(level):
+    def check_permitted(fnc):
+        def wrapper(*args, **kwargs):
+            levels = ['status', 'halt', 'full']
+            this = args[0]
+            try:
+                if levels.index(level) <= levels.index(this.exec_level):
+                    return fnc(*args, **kwargs)
+                else:
+                    raise Exception('Unauthorized operation')
+            except Exception as e:
+                logger.debug (str(e))
+                raise Exception(e)
+
+        return wrapper
+
+    return check_permitted
+
 class ogAdmClientWorker (ogLiveWorker):
     name          = 'ogAdmClient'  # Module name
-    #interface     = None  # Bound interface for OpenGnsys (el otro modulo lo usa para obtener .ip y .mac
     REST          = None  # REST object
 
     def onDeactivation (self):
@@ -77,64 +95,40 @@ class ogAdmClientWorker (ogLiveWorker):
         Sends OGAgent stopping notification to OpenGnsys server
         """
         logger.debug ('onDeactivation')
-
-    #def processClientMessage (self, message, data):
-    #    logger.debug ('Got OpenGnsys message from client: {}, data {}'.format (message, data))
-
-    #def onLogin (self, data):
-    #    logger.warning ('in onLogin, should not happen')
-
-    #def onLogout (self, user):
-    #    logger.warning ('in onLogout, should not happen')
-
-    #@check_secret
-    #def process_reboot (self, path, get_params, post_params, server):
-    #    """
-    #    Launches a system reboot operation
-    #    :param path:
-    #    :param get_params:
-    #    :param post_params:
-    #    :param server: authorization header
-    #    :return: JSON object {"op": "launched"}
-    #    """
-    #    logger.debug ('Received reboot operation')
-
-    #    # Rebooting thread
-    #    def rebt():
-    #        operations.reboot()
-    #    threading.Thread (target=rebt).start()
-    #    return {'op': 'launched'}
-
-    #@check_secret
-    #def process_poweroff (self, path, get_params, post_params, server):
-    #    """
-    #    Launches a system power off operation
-    #    :param path:
-    #    :param get_params:
-    #    :param post_params:
-    #    :param server: authorization header
-    #    :return: JSON object {"op": "launched"}
-    #    """
-    #    logger.debug ('Received poweroff operation')
-
-    #    # Powering off thread
-    #    def pwoff():
-    #        time.sleep (2)
-    #        operations.poweroff()
-    #    threading.Thread (target=pwoff).start()
-    #    return {'op': 'launched'}
+        self.REST.sendMessage ('ogAdmClient/stopped', {'mac': self.mac, 'ip': self.IPlocal, 'idcentro': self.idcentro, 'idaula': self.idaula,
+                                                       'idordenador': self.idordenador, 'nombreordenador': self.nombreordenador})
 
 
 
 
 
+    def InventariandoSoftware (self, dsk, par, nfn):
+        sft_src = f'/tmp/CSft-{self.IPlocal}-{par}'
+        try:
+            self.interfaceAdmin (nfn, [dsk, par, sft_src])
+            herror = 0
+        except:
+            herror = 1
 
+        if herror:
+            logger.warning ('Error al ejecutar el comando')
+            b64 = ''
+            self.muestraMensaje (20)
+        else:
+            if not os.path.exists (sft_src):
+                raise Exception (f'interfaceAdmin({nfn}) returned success but did not create file ({sft_src})')
+            sft_src_contents = Path (sft_src).read_bytes()
 
+            b64 = base64.b64encode (sft_src_contents).decode ('utf-8')
+            self.muestraMensaje (19)
 
-
-
-    ## process_*        are invoked from opengnsys/httpserver.py:99 "data = module.processServerMessage (path, get_params, post_params, self)" (via opengnsys/workers/server_worker.py)
-    ## process_client_* are invoked from opengnsys/service.py:123   "v.processClientMessage (message, json.loads (data))"                      (via opengnsys/workers/server_worker.py)
+        cmd = {
+            'nfn': 'RESPUESTA_InventarioSoftware',
+            'dsk': dsk,         ## not in the original C code, around ogAdmClient.c:1944
+            'par': par,
+            'contents': b64,
+        }
+        return self.respuestaEjecucionComando (cmd, herror, 0)
 
     def ejecutaArchivo (self,fn):
         logger.debug ('fn ({})'.format (fn))
@@ -145,7 +139,7 @@ class ogAdmClientWorker (ogLiveWorker):
         if buffer:
             for l in buffer.split ('@'):
                 if not len (l): continue
-                logger.debug ('line ({})'.format (l))
+                logger.debug ('line ({})'.format (l.replace ('\r', '\\r')))    ## change \r so as not to mess with the log
                 ## at this point, an option would be fire up a curl to localhost, but we can also parse the params and locally call the desired function:
                 post_params = {}
                 for param in l.split ("\r"):
@@ -174,7 +168,7 @@ class ogAdmClientWorker (ogLiveWorker):
             logger.warning ('Ha ocurrido algún problema en el proceso de inclusión del cliente')
             logger.error ('LeeConfiguracion() failed')
             return False
-        res = self.enviaMensajeServidor ('InclusionCliente', { 'cfg': self.cfg2obj (cfg) })
+        res = self.enviaMensajeServidor ('InclusionCliente', { 'cfg': self.cfg2obj (cfg), 'secret': self.random, 'agent_version': VERSION })
         logger.debug ('res ({})'.format (res))
 
         ## RESPUESTA_InclusionCliente
@@ -280,6 +274,9 @@ class ogAdmClientWorker (ogLiveWorker):
 
     def onActivation (self):
         super().onActivation()
+        self.exec_level = 'full'
+        self.random = ''.join(random.choice(string.ascii_lowercase + string.digits) for _ in range(32))
+
         logger.info ('Inicio de sesion')
         logger.info ('Abriendo sesión en el servidor de Administración')
         if (not self.inclusionCliente()):
@@ -306,34 +303,227 @@ class ogAdmClientWorker (ogLiveWorker):
 
         logger.info ('onActivation ok')
 
-    def do_status (self, post_params):
-        cfg = self.LeeConfiguracion()
-        thr_status = {}
-        for k in self.thread_list:
-            thr_status[k] = {
-                'running': self.thread_list[k]['running'],
-                'result': self.thread_list[k]['result'],
-            }
-        return {
-            'nfn': 'RESPUESTA_status',
-            'mac': self.mac,
-            'ip': self.IPlocal,
-            'cfg': self.cfg2obj (cfg),
-            'threads': thr_status,
+
+
+
+
+
+    def do_CrearImagen (self, post_params):
+        for k in ['dsk', 'par', 'cpt', 'idi', 'nci', 'ipr', 'nfn', 'ids']:
+            if k not in post_params:
+                logger.error (f'required parameter ({k}) not in POST params')
+                return {}
+
+        dsk = post_params['dsk'] ## Disco
+        par = post_params['par'] ## Número de partición
+        cpt = post_params['cpt'] ## Código de la partición
+        idi = post_params['idi'] ## Identificador de la imagen
+        nci = post_params['nci'] ## Nombre canónico de la imagen
+        ipr = post_params['ipr'] ## Ip del repositorio
+        nfn = post_params['nfn']
+        ids = post_params['ids']
+
+        self.muestraMensaje (7)
+
+        try:
+            res = self.InventariandoSoftware (dsk, par, 'InventarioSoftware')   ## Crea inventario Software previamente
+        except:
+            logger.warning ('Error al ejecutar el comando')
+            return {}
+
+        if res['contents']:
+            self.muestraMensaje (2)
+            inv_sft = res['contents']
+            try:
+                self.interfaceAdmin (nfn, [dsk, par, nci, ipr])
+                self.muestraMensaje (9)
+                herror = 0
+            except:
+                logger.warning ('Error al ejecutar el comando')
+                self.muestraMensaje (10)
+                herror = 1
+        else:
+            logger.warning ('Error al ejecutar el comando')
+            herror = 1
+            inv_sft = ''
+
+        self.muestraMenu()
+
+        cmd = {
+            'nfn':     'RESPUESTA_CrearImagen',
+            'idi':     idi,    ## Identificador de la imagen
+            'dsk':     dsk,    ## Número de disco
+            'par':     par,    ## Número de partición de donde se creó
+            'cpt':     cpt,    ## Tipo o código de partición
+            'ipr':     ipr,    ## Ip del repositorio donde se alojó
+            'inv_sft': inv_sft,
         }
+        return self.respuestaEjecucionComando (cmd, herror, ids)
 
-    @check_secret
-    def process_status (self, path, get_params, post_params, server):
-        logger.debug ('in process_status, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        return self._long_running_job ('status', self.do_status, args=(post_params,))
+    def do_RestaurarImagen (self, post_params):
+        for k in ['dsk', 'par', 'idi', 'ipr', 'nci', 'ifs', 'ptc', 'nfn', 'ids']:
+            if k not in post_params:
+                logger.error (f'required parameter ({k}) not in POST params')
+                return {}
 
-    @check_secret
-    def process_popup (self, path, get_params, post_params, server):
-        logger.debug ('in process_popup, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
-        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
-        ## in process_popup, should not happen, path "[]" get_params "{}" post_params "{'title': 'mi titulo', 'message': 'mi mensaje'}" server "<opengnsys.httpserver.HTTPServerHandler object at 0x7fa788cb8fa0>"
-        ## type(post_params) "<class 'dict'>"
-        return {'debug':'test'}
+        dsk = post_params['dsk']
+        par = post_params['par']
+        idi = post_params['idi']
+        ipr = post_params['ipr']
+        nci = post_params['nci']
+        ifs = post_params['ifs']
+        ptc = post_params['ptc']    ## Protocolo de clonación: Unicast, Multicast, Torrent
+        nfn = post_params['nfn']
+        ids = post_params['ids']
+
+        self.muestraMensaje (3)
+
+        try:
+            ## the ptc.split() is useless right now, since interfaceAdmin() does ' '.join(params) in order to spawn a shell
+            ## however we're going to need it in the future (when everything gets translated into python), plus it's harmless now. So let's do it
+            #self.interfaceAdmin (nfn, [dsk, par, nci, ipr, ptc])
+            self.interfaceAdmin (nfn, [dsk, par, nci, ipr] + ptc.split())
+            self.muestraMensaje (11)
+            herror = 0
+        except:
+            logger.warning ('Error al ejecutar el comando')
+            self.muestraMensaje (12)
+            herror = 1
+
+        cfg = self.LeeConfiguracion()
+        if not cfg:
+            logger.warning ('No se ha podido recuperar la configuración de las particiones del disco')
+
+        self.muestraMenu()
+
+        cmd = {
+            'nfn': 'RESPUESTA_RestaurarImagen',
+            'idi': idi,                  ## Identificador de la imagen
+            'dsk': dsk,                  ## Número de disco
+            'par': par,                  ## Número de partición
+            'ifs': ifs,                  ## Identificador del perfil software
+            'cfg': self.cfg2obj(cfg),    ## Configuración de discos
+        }
+        return self.respuestaEjecucionComando (cmd, herror, ids)
+
+    def do_Configurar (self, post_params):
+        for k in ['nfn', 'dsk', 'cfg', 'ids']:
+            if k not in post_params:
+                logger.error (f'required parameter ({k}) not in POST params')
+                return {}
+
+        nfn = post_params['nfn']
+        dsk = post_params['dsk']
+        cfg = post_params['cfg']
+        ids = post_params['ids']
+
+        self.muestraMensaje (4)
+
+        params = []
+        disk_info = cfg.pop (0)
+        logger.debug (f'disk_info ({disk_info})')
+        for k in ['dis', 'tch']:
+            params.append (f'{k}={disk_info[k]}')
+        disk_info_str = '*'.join (params)
+
+        partitions = []
+        for entry in cfg:
+            logger.debug (f'entry ({entry})')
+            params = []
+            for k in ['par', 'cpt', 'sfi', 'tam', 'ope']:
+                params.append (f'{k}={entry[k]}')
+            partitions.append ('*'.join (params))
+        part_info_str = '%'.join (partitions)
+
+        cfg_str = f'{disk_info_str}!{part_info_str}%'
+
+        try:
+            self.interfaceAdmin (nfn, ['ignored', cfg_str])
+            self.muestraMensaje (14)
+            herror = 0
+        except:
+            logger.warning ('Error al ejecutar el comando')
+            self.muestraMensaje (13)
+            herror = 1
+
+        cfg = self.LeeConfiguracion()
+        if not cfg:
+            logger.warning ('No se ha podido recuperar la configuración de las particiones del disco')
+            return {}
+
+        cmd = {
+            'nfn': 'RESPUESTA_Configurar',
+            'cfg': self.cfg2obj (cfg),
+        }
+        self.muestraMenu()
+        return self.respuestaEjecucionComando (cmd, herror, ids)
+
+    def do_InventarioHardware (self, post_params):
+        for k in ['nfn', 'ids']:
+            if k not in post_params:
+                logger.error (f'required parameter ({k}) not in POST params')
+                return {}
+
+        nfn = post_params['nfn']
+        ids = post_params['ids']
+
+        self.muestraMensaje (6)
+
+        hrdsrc = f'/tmp/Chrd-{self.IPlocal}'      ## Nombre que tendra el archivo de inventario
+        hrddst = f'/tmp/Shrd-{self.IPlocal}'      ## Nombre que tendra el archivo en el Servidor
+        try:
+            self.interfaceAdmin (nfn, [hrdsrc])
+            hrdsrc_contents = Path (hrdsrc).read_bytes()
+            logger.debug (f'hrdsrc_contents 1 ({hrdsrc_contents})')
+            herror = 0
+        except:
+            logger.warning ('Error al ejecutar el comando')
+            self.muestraMensaje (18)
+            herror = 1
+
+        if herror:
+            hrddst = ''
+        else:
+            logger.debug (f'hrdsrc_contents 2 ({hrdsrc_contents})')
+            ## Envía fichero de inventario al servidor
+            res = self.enviaMensajeServidor ('recibeArchivo', { 'nfl': hrddst, 'contents': base64.b64encode (hrdsrc_contents).decode ('utf-8') })
+            logger.debug (res)
+            if not res:
+                logger.error ('Ha ocurrido algún problema al enviar un archivo por la red')
+                herror = 12     ## Error de envío de fichero por la red
+            self.muestraMensaje (17)
+
+        ## Envia respuesta de ejecución de la función de interface
+        cmd = {
+            'nfn': 'RESPUESTA_InventarioHardware',
+            'hrd': hrddst,
+        }
+        self.muestraMenu()
+        return self.respuestaEjecucionComando (cmd, herror, ids)
+
+    def do_InventarioSoftware (self, post_params):
+        for k in ['nfn', 'dsk', 'par', 'ids']:
+            if k not in post_params:
+                logger.error (f'required parameter ({k}) not in POST params')
+                return {}
+
+        nfn = post_params['nfn']
+        dsk = post_params['dsk']
+        par = post_params['par']
+        ids = post_params['ids']
+
+        self.muestraMensaje (7)
+
+        try:
+            cmd = self.InventariandoSoftware (dsk, par, 'InventarioSoftware')
+            herror = 0
+        except:
+            logger.warning ('Error al ejecutar el comando')
+            cmd = { 'nfn': 'RESPUESTA_InventarioSoftware' }
+            herror = 1
+
+        self.muestraMenu()
+        return self.respuestaEjecucionComando (cmd, herror, ids)
 
     def do_Actualizar (self, post_params):
         self.muestraMensaje (1)
@@ -379,7 +569,7 @@ class ogAdmClientWorker (ogLiveWorker):
                 return {}
 
         nfn = post_params['nfn']
-        scp = unquote (post_params['scp'])
+        scp = base64.b64decode (unquote (post_params['scp'])).decode ('utf-8')
         filescript = f'/tmp/_script_{self.IPlocal}'
         ecosrc = f'/tmp/_econsola_{self.IPlocal}'
         ecodst = f'/tmp/_Seconsola_{self.IPlocal}'    ## Nombre que tendra el archivo en el Servidor
@@ -474,7 +664,7 @@ class ogAdmClientWorker (ogLiveWorker):
                 return {}
 
         nfn = post_params['nfn']
-        scp = unquote (post_params['scp'])
+        scp = base64.b64decode (unquote (post_params['scp'])).decode ('utf-8')
         ids = post_params['ids']
 
         self.muestraMensaje (8)
@@ -507,16 +697,56 @@ class ogAdmClientWorker (ogLiveWorker):
         self.muestraMenu()
         return self.respuestaEjecucionComando (cmd, herror, ids)
 
+
+
+
+
+    @execution_level('status')
+    def process_status (self, path, get_params, post_params, server):
+        logger.debug ('in process_status, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        full_config = 'full-config' in post_params and post_params['full-config']
+        thr_status = {}
+        for k in self.thread_list:
+            thr_status[k] = {
+                'running': self.thread_list[k]['running'],
+                'result': self.thread_list[k]['result'],
+            }
+        ret = {
+            'nfn': 'RESPUESTA_status',
+            'mac': self.mac,
+            'st': 'OGL',
+            'ip': self.IPlocal,
+            'threads': thr_status,
+        }
+        if full_config:
+            cfg = self.LeeConfiguracion()
+            ret['cfg'] = self.cfg2obj (cfg)
+        return ret
+
+    @check_secret
+    def process_popup (self, path, get_params, post_params, server):
+        logger.debug ('in process_popup, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
+        ## in process_popup, should not happen, path "[]" get_params "{}" post_params "{'title': 'mi titulo', 'message': 'mi mensaje'}" server "<opengnsys.httpserver.HTTPServerHandler object at 0x7fa788cb8fa0>"
+        ## type(post_params) "<class 'dict'>"
+        return {'debug':'test'}
+
+    @execution_level('full')
+    @check_secret
     def process_Actualizar (self, path, get_params, post_params, server):
         logger.debug ('in process_Actualizar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('Actualizar', self.do_Actualizar, args=(post_params,))
 
+    @execution_level('full')
+    @check_secret
     def process_Purgar (self, path, get_params, post_params, server):
         logger.debug ('in process_Purgar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         os.kill (os.getpid(), signal.SIGTERM)
         return {}
         #exit (0)      ## ogAdmClient.c:905
 
+    @execution_level('full')
+    @check_secret
     def process_Comando (self, path, get_params, post_params, server):
         logger.debug ('in process_Comando, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('Comando', self.do_Comando, args=(post_params,))
@@ -525,10 +755,14 @@ class ogAdmClientWorker (ogLiveWorker):
         logger.debug ('in process_Sondeo, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return {}      ## ogAdmClient.c:920
 
+    @execution_level('full')
+    @check_secret
     def process_ConsolaRemota (self, path, get_params, post_params, server):
         logger.debug ('in process_ConsolaRemota, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('ConsolaRemota', self.do_ConsolaRemota, args=(post_params,))
 
+    @execution_level('full')
+    @check_secret
     def process_Arrancar (self, path, get_params, post_params, server):
         logger.debug ('in process_Arrancar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
 
@@ -545,22 +779,97 @@ class ogAdmClientWorker (ogLiveWorker):
         }
         return self.respuestaEjecucionComando (cmd, 0, ids)
 
+    @execution_level('halt')
+    @check_secret
     def process_Apagar (self, path, get_params, post_params, server):
         logger.debug ('in process_Apagar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('Apagar', self.do_Apagar, args=(post_params,))
 
+    @execution_level('halt')
+    @check_secret
     def process_Reiniciar (self, path, get_params, post_params, server):
         logger.debug ('in process_Reiniciar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('Reiniciar', self.do_Reiniciar, args=(post_params,))
 
+    @execution_level('full')
+    @check_secret
     def process_IniciarSesion (self, path, get_params, post_params, server):
         logger.debug ('in process_IniciarSesion, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('IniciarSesion', self.do_IniciarSesion, args=(post_params,))
 
+    @execution_level('full')
+    @check_secret
     def process_EjecutarScript (self, path, get_params, post_params, server):
         logger.debug ('in process_EjecutarScript, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return self._long_running_job ('EjecutarScript', self.do_EjecutarScript, args=(post_params,))
 
+    @execution_level('full')
+    @check_secret
     def process_EjecutaComandosPendientes (self, path, get_params, post_params, server):
         logger.debug ('in process_EjecutaComandosPendientes, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
         return {'true':'true'}         ## ogAdmClient.c:2138
+
+    @execution_level('full')
+    @check_secret
+    def process_CrearImagen (self, path, get_params, post_params, server):
+        logger.debug ('in process_CrearImagen, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
+        return self._long_running_job ('CrearImagen', self.do_CrearImagen, args=(post_params,))
+
+    #def process_CrearImagenBasica (self, path, get_params, post_params, server):
+    #    logger.debug ('in process_CrearImagenBasica, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+    #    logger.warning ('this method has been removed')
+    #    raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
+
+    #def process_CrearSoftIncremental (self, path, get_params, post_params, server):
+    #    logger.debug ('in process_CrearSoftIncremental, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+    #    logger.warning ('this method has been removed')
+    #    raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
+
+    @execution_level('full')
+    @check_secret
+    def process_RestaurarImagen (self, path, get_params, post_params, server):
+        logger.debug ('in process_RestaurarImagen, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        logger.debug ('type(post_params) "{}"'.format (type (post_params)))
+        return self._long_running_job ('RestaurarImagen', self.do_RestaurarImagen, args=(post_params,))
+
+    #def process_RestaurarImagenBasica (self, path, get_params, post_params, server):
+    #    logger.debug ('in process_RestaurarImagenBasica, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+    #    logger.warning ('this method has been removed')
+    #    raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
+
+    #def process_RestaurarSoftIncremental (self, path, get_params, post_params, server):
+    #    logger.warning ('in process_RestaurarSoftIncremental')
+    #    logger.debug ('in process_RestaurarSoftIncremental, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+    #    logger.warning ('this method has been removed')
+    #    raise Exception ({ '_httpcode': 404, '_msg': 'This method has been removed' })
+
+    ## una partición + cache en disco de 30 Gb:
+    @execution_level('full')
+    @check_secret
+    def process_Configurar (self, path, get_params, post_params, server):
+        logger.debug ('in process_Configurar, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        return self._long_running_job ('Configurar', self.do_Configurar, args=(post_params,))
+
+    @execution_level('full')
+    @check_secret
+    def process_InventarioHardware (self, path, get_params, post_params, server):
+        logger.debug ('in process_InventarioHardware, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        return self._long_running_job ('InventarioHardware', self.do_InventarioHardware, args=(post_params,))
+
+    @execution_level('full')
+    @check_secret
+    def process_InventarioSoftware (self, path, get_params, post_params, server):
+        logger.debug ('in process_InventarioSoftware, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        return self._long_running_job ('InventarioSoftware', self.do_InventarioSoftware, args=(post_params,))
+
+    @execution_level('full')
+    @check_secret
+    def process_KillJob (self, path, get_params, post_params, server):
+        logger.debug ('in process_KillJob, path "{}" get_params "{}" post_params "{}" server "{}"'.format (path, get_params, post_params, server))
+        jid = post_params['job_id']
+        r = self.killer (jid)
+        logger.debug (f'r bef ({r})')
+        r.update ({ 'nfn':'RESPUESTA_KillJob', 'job':jid })
+        logger.debug (f'r aft ({r})')
+        return r

src/opengnsys/windows/log.py

@@ -36,6 +36,8 @@ import logging
 import os
 import tempfile
 
+from ..log_format import JsonFormatter
+
 # Valid logging levels, from UDS Broker (uds.core.utils.log)
 OTHER, DEBUG, INFO, WARN, ERROR, FATAL = (10000 * (x + 1) for x in range(6))
 
@@ -44,13 +46,24 @@ class LocalLogger(object):
     def __init__(self):
         # tempdir is different for "user application" and "service"
         # service wil get c:\windows\temp, while user will get c:\users\XXX\appdata\local\temp
-        logging.basicConfig(
-            filename=os.path.join(tempfile.gettempdir(), 'opengnsys.log'),
-            filemode='a',
-            format='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s',
-            level=logging.DEBUG
-        )
+
+        fname1 = os.path.join (tempfile.gettempdir(), 'opengnsys.log')
+        fmt1 = logging.Formatter (fmt='%(levelname)s %(asctime)s (%(threadName)s) (%(funcName)s) %(message)s')
+        fh1 = logging.FileHandler (filename=fname1, mode='a')
+        fh1.setFormatter (fmt1)
+        fh1.setLevel (logging.DEBUG)
+
+        fname2 = os.path.join (tempfile.gettempdir(), 'opengnsys.json.log')
+        fmt2 = JsonFormatter ({"timestamp": "asctime", "severity": "levelname", "threadName": "threadName", "function": "funcName", "message": "message"}, time_format='%Y-%m-%d %H:%M:%S', msec_format='')
+        fh2 = logging.FileHandler (filename=fname2, mode='a')
+        fh2.setFormatter (fmt2)
+        fh2.setLevel (logging.DEBUG)
+
         self.logger = logging.getLogger('opengnsys')
+        self.logger.setLevel (logging.DEBUG)
+        self.logger.addHandler (fh1)
+        self.logger.addHandler (fh2)
+
         self.serviceLogger = False
 
     def log(self, level, message):

src/opengnsys/workers/oglive_worker.py

@@ -31,10 +31,12 @@
 # pylint: disable=unused-wildcard-import,wildcard-import
 
 import os
+import re
 import time
 import random
 import subprocess
 import threading
+import signal
 
 from configparser import NoOptionError
 from opengnsys import REST
@@ -47,16 +49,21 @@ class ThreadWithResult (threading.Thread):
         try:
             self.result = None
             if self._target is not None:
+                ## the first arg in self._args is the queue
+                self.pid_q    = self._args[0]
+                self.stdout_q = self._args[1]
+                self._args    = self._args[2:]
                 try:
                     self.result = self._target (*self._args, **self._kwargs)
                 except Exception as e:
                     self.result = { 'res': 2, 'der': f'got exception: ({e})' }    ## res=2 as defined in ogAdmClient.c:2048
         finally:
             # Avoid a refcycle if the thread is running a function with an argument that has a member that points to the thread.
-            del self._target, self._args, self._kwargs
+            del self._target, self._args, self._kwargs, self.pid_q, self.stdout_q
 
 class ogLiveWorker(ServerWorker):
     thread_list = {}
+    thread_lock = threading.Lock()
 
     tbErroresScripts = [
         "Se han generado errores desconocidos. No se puede continuar la ejecución de este módulo",        ## 0
@@ -137,58 +144,179 @@ class ogLiveWorker(ServerWorker):
 		"Error desconocido",
 	]
 
-    def notifier (self, result):
-        logger.debug (f'notifier() called, result ({result})')
-        res = self.REST.sendMessage ('/'.join ([self.name, 'callback']), result)
+    def notifier (self, job_id, result):
+        result['job_id'] = job_id
+        self.REST.sendMessage ('clients/status/webhook', result)
 
+    def killer (self, job_id):
+        logger.debug (f'killer() called, job_id ({job_id})')
+        if job_id not in self.thread_list: return { 'res': 2, 'der': 'Unknown job' }
+
+        with self.thread_lock:
+            if 'thread' not in self.thread_list[job_id]: return { 'res': 2, 'der': 'Job is not running' }
+            t   = self.thread_list[job_id]['thread']
+            pid = self.thread_list[job_id]['child_pid']
+            logger.debug (f'pid ({pid})')
+            try_times = 8
+            sig = signal.SIGTERM
+            msg = f'could not kill pid ({pid}) after ({try_times}) tries'
+            success = 2    ## mimic cmd['res'] in respuestaEjecucionComando(): "1" means success, "2" means failed
+            while True:
+                t.join (0.05)
+                if not t.is_alive():
+                    msg = 'job terminated'
+                    success = 1
+                    logger.debug (msg)
+                    self.thread_list[job_id]['child_pid'] = None
+                    break
+                ## race condition: if the subprocess finishes just here, then we already checked that t.is_alive() is true, but os.path.exists(/proc/pid) will be false below. msg will be 'nothing to kill'.
+                ## this is fine in the first iteration of the loop, before we send any signals. In the rest of iterations, after some signals were sent, msg should be 'job terminated' instead.
+                if pid:
+                    if os.path.exists (f'/proc/{pid}'):
+                        logger.debug (f'sending signal ({sig}) to pid ({pid})')
+                        ## if the process finishes just here, nothing happens: the signal is sent to the void
+                        os.kill (pid, sig)
+                        #subprocess.run (['kill', '--signal', str(sig), str(pid)])
+                    else:
+                        msg = f'pid ({pid}) is gone, nothing to kill'
+                        success = 1
+                        logger.debug (msg)
+                        self.thread_list[job_id]['child_pid'] = None
+                        break
+                else:
+                    msg = 'no PID to kill'
+                    logger.debug (msg)
+
+                if not try_times: break
+                if 4 == try_times: sig = signal.SIGKILL   ## change signal after a few tries
+                try_times -= 1
+                time.sleep (0.4)
+
+        return { 'res':success, 'der':msg }
+
+    def _extract_progress (self, job_id, ary=[]):
+        progress = None
+        for i in ary:
+            if m := re.search (r'^\[([0-9]+)\]', i):     ## look for strings like '[10]', '[60]'
+                #logger.debug (f"matched regex, m.groups ({m.groups()})")
+                progress = float (m.groups()[0]) / 100
+        return progress
+
+    ## monitors child threads, waits for them to finish
+    ## pings ogcore
     def mon (self):
+        n = 0
         while True:
-            #print ('mon(): iterating')
-            for k in self.thread_list:
-                elem = self.thread_list[k]
-                if 'thread' not in elem: continue
-                logger.debug (f'considering thread ({k})')
-                try: elem['thread'].join (0.05)
-                except RuntimeError: pass       ## race condition: a thread is created and this code runs before it is start()ed
-                if not elem['thread'].is_alive():
-                    logger.debug (f'is no longer alive, k ({k}) thread ({elem["thread"]})')
-                    elem['running'] = False
-                    elem['result'] = elem['thread'].result
-                    del elem['thread']
-                    self.notifier (elem['result'])
+            with self.thread_lock:
+                for k in self.thread_list:
+                    elem = self.thread_list[k]
+                    if 'thread' not in elem: continue
+                    #logger.debug (f'considering thread ({k})')
+
+                    if self.pid_q:
+                        if not self.pid_q.empty():
+                            elem['child_pid'] = self.pid_q.get()
+                            logger.debug (f'queue not empty, got pid ({elem["child_pid"]})')
+
+                    if self.stdout_q:
+                        partial = ''
+                        while not self.stdout_q.empty():
+                            partial += self.stdout_q.get()
+                        lines = partial.splitlines()
+                        if len (lines):
+                            p = self._extract_progress (k, lines)
+                            if p:
+                                m = { "job_id": k, "progress": p }
+                                self.REST.sendMessage ('clients/status/webhook', { "job_id": k, "progress": p })
+
+                    elem['thread'].join (0.05)
+                    if not elem['thread'].is_alive():
+                        logger.debug (f'is no longer alive, k ({k}) thread ({elem["thread"]})')
+                        elem['running'] = False
+                        elem['result'] = elem['thread'].result
+                        del elem['thread']
+                        self.notifier (k, elem['result'])
 
             time.sleep (1)
+            n += 1
+            if not n % 10:
+                alive_threads = []
+                for k in self.thread_list:
+                    elem = self.thread_list[k]
+                    if 'thread' not in elem: continue
+                    alive_threads.append (k)
+                if alive_threads:
+                    s = ','.join (alive_threads)
+                    logger.debug (f'alive threads: {s}')
+
+                body = {
+                    'iph': self.IPlocal,
+                    'timestamp': int (time.time()),
+                }
+                #logger.debug (f'about to send ping ({body})')
+                self.REST.sendMessage ('clients/status/webhook', body)
 
     def interfaceAdmin (self, method, parametros=[]):
-        exe = '{}/{}'.format (self.pathinterface, method)
-        ## for development only. Will be removed when the referenced bash code (/opt/opengnsys/lib/engine/bin/*.lib) is translated into python
-        devel_bash_prefix = '''
-            PATH=/opt/opengnsys/scripts/:$PATH;
-            for I in /opt/opengnsys/lib/engine/bin/*.lib; do source $I; done;
-            for i in $(declare -F |cut -f3 -d" "); do export -f $i; done;
-        '''
+        if method in ['Apagar', 'CambiarAcceso', 'Configurar', 'CrearImagen', 'EjecutarScript', 'getConfiguration', 'getIpAddress', 'IniciarSesion', 'InventarioHardware', 'InventarioSoftware', 'Reiniciar', 'RestaurarImagen']:
+            ## python
+            logger.debug (f'({method}) is a python method')
+            exe = '{}/{}.py'.format (self.pathinterface, method)
+            proc = [exe]+parametros
+        else:  ## ConsolaRemota procesaCache
+            ## bash
+            logger.debug (f'({method}) is a bash method')
+            exe = '{}/{}'.format (self.pathinterface, method)
 
-        if parametros:
-            proc = ['bash', '-c', '{} bash -x {} {}'.format (devel_bash_prefix, exe, ' '.join (parametros))]
+            LANG = os.environ.get ('LANG', 'en_GB.UTF-8').replace ('UTF_8', 'UTF-8')
+            devel_bash_prefix = f'''
+                PATH=/opt/opengnsys/scripts/:$PATH;
+                source /opt/opengnsys/etc/lang.{LANG}.conf;
+                for I in /opt/opengnsys/lib/engine/bin/*.lib; do source $I; done;
+                for i in $(declare -F |cut -f3 -d" "); do export -f $i; done;
+            '''
+
+            if parametros:
+                proc = ['bash', '-c', '{} {} {}'.format (devel_bash_prefix, exe, ' '.join (parametros))]
+            else:
+                proc = ['bash', '-c', '{} {}'.format (devel_bash_prefix, exe)]
+
+        logger.debug ('subprocess.run ("{}")'.format (' '.join (proc)))
+        p = subprocess.Popen (proc, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        if self.pid_q:
+            self.pid_q.put (p.pid)
         else:
-            proc = ['bash', '-c', '{} bash -x {}'.format (devel_bash_prefix, exe)]
-        logger.debug ('subprocess.run ("{}", capture_output=True)'.format (proc))
-        p = subprocess.run (proc, capture_output=True)
+            ## esto sucede por ejemplo cuando arranca el agente, que estamos en interfaceAdmin() en el mismo hilo, sin _long_running_job ni hilo separado
+            #logger.debug ('no queue--not writing any PID to it')
+            pass
+
+        sout = serr = ''
+        while p.poll() is None:
+            for l in iter (p.stdout.readline, b''):
+                partial = l.decode ('utf-8', 'ignore')
+                if self.stdout_q: self.stdout_q.put (partial)
+                sout += partial
+            for l in iter (p.stderr.readline, b''):
+                partial = l.decode ('utf-8', 'ignore')
+                serr += partial
+            time.sleep (1)
+        sout = sout.strip()
+        serr = serr.strip()
+
         ## DEBUG
-        logger.info (f'stdout follows:')
-        for l in p.stdout.strip().decode ('utf-8').splitlines():
-            logger.info (f'  {l}')
-        logger.info (f'stderr follows:')
-        for l in p.stderr.strip().decode ('utf-8').splitlines():
-            logger.info (f'  {l}')
+        logger.debug (f'stdout follows:')
+        for l in sout.splitlines():
+            logger.debug (f'  {l}')
+        #logger.debug (f'stderr follows:')
+        #for l in serr.splitlines():
+        #    logger.debug (f'  {l}')
         ## /DEBUG
         if 0 != p.returncode:
             cmd_txt = ' '.join (proc)
             logger.error (f'command ({cmd_txt}) failed, stderr follows:')
-            for l in p.stderr.strip().decode ('utf-8').splitlines():
+            for l in serr.splitlines():
                 logger.error (f'  {l}')
             raise Exception (f'command ({cmd_txt}) failed, see log for details')
-        return p.stdout.strip().decode ('utf-8')
+        return sout
 
     def tomaIPlocal (self):
         try:
@@ -219,9 +347,7 @@ class ogLiveWorker(ServerWorker):
         res = self.REST.sendMessage ('/'.join ([self.name, path]), obj)
 
         if (type (res) is not dict):
-            #logger.error ('No se ha podido establecer conexión con el Servidor de Administración')   ## Error de conexión con el servidor
-            logger.debug (f'res ({res})')
-            logger.error ('Error al enviar trama ***send() fallo')
+            logger.error (f'response is not a dict ({res})')
             return False
 
         return res
@@ -237,20 +363,18 @@ class ogLiveWorker(ServerWorker):
             cmd['der'] = ''
         else:            ## el comando tuvo algún error
             cmd['res'] = 2
-            cmd['der'] = self.tbErroresScripts[herror]           ## XXX
+            cmd['der'] = self.tbErroresScripts[herror]
 
         return cmd
 
     def cargaPaginaWeb (self, url=None):
         if (not url): url = self.urlMenu
-        os.system ('pkill -9 OGBrowser')
+        os.system ('pkill -9 browser')
 
-        p = subprocess.Popen (['/usr/bin/OGBrowser', '-qws', url])
+        p = subprocess.Popen (['/usr/bin/browser', '-qws', url])
         try:
             p.wait (2)       ## if the process dies before 2 seconds...
-            logger.error ('Error al ejecutar la llamada a la interface de administración')
-            logger.error ('Error en la creación del proceso hijo')
-            logger.error ('return code "{}"'.format (p.returncode))
+            logger.error ('Error al ejecutar browser, return code "{}"'.format (p.returncode))
             return False
         except subprocess.TimeoutExpired:
             pass
@@ -270,7 +394,7 @@ class ogLiveWorker(ServerWorker):
             logger.error (e)
             logger.error ('No se ha podido recuperar la dirección IP del cliente')
             return None
-        logger.debug ('parametroscfg ({})'.format (parametroscfg))
+        #logger.debug ('parametroscfg ({})'.format (parametroscfg))
         return parametroscfg
 
     def cfg2obj (self, cfg):
@@ -281,7 +405,10 @@ class ogLiveWorker(ServerWorker):
             ptrCfg = line.split ('\t')
 
             for item in ptrCfg:
-                k, v = item.split ('=')
+                if '=' not in item:
+                    logger.warning (f'invalid item ({item})')
+                    continue
+                k, v = item.split ('=', maxsplit=1)
                 elem[k] = v
 
             obj.append (elem)
@@ -303,13 +430,27 @@ class ogLiveWorker(ServerWorker):
         self.idproautoexec   = None
         self.idcentro        = None     ## Identificador del centro
         self.idaula          = None     ## Identificador del aula
+        self.pid_q           = None     ## for passing PIDs around
+        self.stdout_q        = None     ## for passing stdout
+        self.progress_jobs   = {}
 
+        ogcore_scheme   = os.environ.get ('OGAGENTCFG_OGCORE_SCHEME',  'https')
+        ogcore_ip       = os.environ.get ('OGAGENTCFG_OGCORE_IP',      '192.168.2.1')
+        ogcore_port     = os.environ.get ('OGAGENTCFG_OGCORE_PORT',    '8443')
+        urlmenu_scheme  = os.environ.get ('OGAGENTCFG_URLMENU_SCHEME', 'https')
+        urlmenu_ip      = os.environ.get ('OGAGENTCFG_URLMENU_IP',     '192.168.2.1')
+        urlmenu_port    = os.environ.get ('OGAGENTCFG_URLMENU_PORT',   '8443')
+        ogcore_ip_port  = ':'.join (map (str, filter (None, [ogcore_ip,  ogcore_port ])))
+        urlmenu_ip_port = ':'.join (map (str, filter (None, [urlmenu_ip, urlmenu_port])))
         try:
             url                = self.service.config.get (self.name, 'remote')
             loglevel           = self.service.config.get (self.name, 'log')
             self.pathinterface = self.service.config.get (self.name, 'pathinterface')
             self.urlMenu       = self.service.config.get (self.name, 'urlMenu')
             self.urlMsg        = self.service.config.get (self.name, 'urlMsg')
+
+            url          = url.format          (ogcore_scheme,  ogcore_ip_port)
+            self.urlMenu = self.urlMenu.format (urlmenu_scheme, urlmenu_ip_port)
         except NoOptionError as e:
             logger.error ("Configuration error: {}".format (e))
             raise e
@@ -335,11 +476,39 @@ class ogLiveWorker(ServerWorker):
             return { 'job_id': None, 'message': 'some job is already running, refusing to launch another one' }
 
         job_id = '{}-{}'.format (name, ''.join (random.choice ('0123456789abcdef') for _ in range (8)))
+        import queue
+        self.pid_q    = queue.Queue()   ## a single queue works for us because we never have more than one long_running_job at the same time
+        self.stdout_q = queue.Queue()
         self.thread_list[job_id] = {
-            'thread': ThreadWithResult (target=f, args=args),
+            'thread': ThreadWithResult (target=f, args=(self.pid_q, self.stdout_q) + args),
             'starttime': time.time(),
+            'child_pid': None,
             'running': True,
             'result': None
         }
         self.thread_list[job_id]['thread'].start()
         return { 'job_id': job_id }
+
+## para matar threads tengo lo siguiente:
+## - aqui en _long_running_job meto una cola en self.pid_q
+## - (self.pid_q fue inicializado a None al instanciar el objeto, para evitar error "objeto no tiene 'pid_q'")
+## - en el thread_list también tengo un child_pid para almacenar el pid de los procesos hijos
+## - al crear el ThreadWithResult le paso la cola, y luego en run() la recojo y la meto en el self.pid_q del thread
+## - en interfaceAdmin() al hacer subprocess.Popen(), recojo el pid y lo escribo en la queue
+## - en mon() recojo pids de la queue y los meto en thread_list 'child_pid'
+##   - algunas funciones llaman a interfaceAdmin más de una vez, y escriben más de un pid en la cola, y en mon() voy recogiendo y actualizando
+##     - por ejemplo EjecutarScript llama a interfaceAdmin() y luego llama a LeeConfiguracion() el cual llama a interfaceAdmin() otra vez
+## - y cuando nos llamen a KillJob, terminamos en killer() el cual coge el 'child_pid' y zas
+##   - pero a lo mejor el child ya terminó
+##   - o a lo mejor el KillJob nos llegó demasiado pronto y todavía no hubo ningún child
+##
+## $ curl --insecure -X POST --data '{"nfn":"EjecutarScript","scp":"cd /usr; sleep 30; pwd; ls","ids":"0"}' https://192.168.2.199:8000/ogAdmClient/EjecutarScript
+## {"job_id": "EjecutarScript-333feb3f"}
+## $ curl --insecure -X POST --data '{"job_id":"EjecutarScript-333feb3f"}' https://192.168.2.199:8000/ogAdmClient/KillJob
+##
+## funciona bien, excepto que el PID no muere xD, ni siquiera haciendo subprocess.run('kill')
+
+## para mostrar el progreso de los jobs reutilizo la misma infra
+## una cola self.stdout_q
+## en interfaceAdmin escribo la stdout parcial que ya venia recogiendo
+## mon() lo recoge y le hace un POST a ogcore