job manager #5

Merged
nserrano merged 6 commits from ogagent-jobs into main 2024-08-07 14:03:24 +02:00
Collaborator

Historia de usuario: https://ognproject.evlt.uma.es/redmine/issues/494

Preparar el entorno

Ejecutar una VM linux, windows o macos (por ejemplo con virtualbox, docukrr o qemu).

Crear paquete de ogagent-1.3.4 (wiki redmine). Subirlo a la VM e instalarlo.

Configurar IP del ogserver de mentira (usar la IP del portátil de uno):

# sed -i -e '/^remote=/s/192.168.2.10/192.168.1.249/' /usr/share/OGAgent/cfg/ogagent.cfg

Ejecutar ogserver de mentira:

$ sudo perl -MDateTime -Mojo -E 'a("/*foo" => sub { printf "%s %s %s %s\n", DateTime->now->strftime ("%Y%m%d %H%M%S%z"), $_->tx->remote_address, $_->param("foo"), $_->req->body; $_->render (text => "{}") })->start ("daemon", "-l", "https://192.168.1.249:443")'

Reiniciar la VM y comprobar que el agente arranca.

Subir a la VM un script que tarde un tiempo en ejecutarse. Llamarlo printer.py. Por ejemplo este:

import time
import sys
import random
r=random.random()

def _out(txt,zzz=3):
    sys.stdout.write("{} {}".format(r,txt))
    sys.stdout.flush()
    time.sleep(zzz)

def _err(txt,zzz=3):
    sys.stderr.write("{} {}".format(r,txt))
    sys.stderr.flush()
    time.sleep(zzz)

_out ("Hello\n")
_out ("World!\n")
_err ("That's\n")
_out ("Actually,\n")
_err ("an error\n")
_out ("I'm fine\n")

Exportar un par de variables para que el PR sea más fácil de seguir:

$ export CLI_IP=192.168.1.43
$ export TOKEN=elquesea

Ejecutar pwd en el lado de sistema

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo pwd |base64 -w 0)'","client":"false"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched", "jobid": "cc7707b2"}

Para obtener la lista de trabajos hay que llamar a preparescripts, esperar un poco y llamar a getscripts:

$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  {
    "pid": 1144,
    "starttime": "2024-08-01 11:22:38.967481+0000",
    "script": "pwd\n",
    "client": false,
    "status": "finished",
    "stdout": "/\n",
    "stderr": "",
    "jobid": "cc7707b2",
    "rc": 0
  }
]

Ejecutar pwd en el lado de usuario

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo pwd |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched"}

Debido a la arquitectura asíncrona del agente, el jobid no está inmediatamente disponible y no sale en la salida de la llamada a /script.

Consultar jobs:

$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  {
    "pid": 1144,
    "starttime": "2024-08-01 11:22:38.967481+0000",
    "script": "pwd\n",
    "client": false,
    "status": "finished",
    "stdout": "/\n",
    "stderr": "",
    "jobid": "cc7707b2",
    "rc": 0
  },
  {
    "pid": 1150,
    "starttime": "2024-08-01 11:23:37.751604+0000",
    "script": "pwd\n",
    "client": true,
    "status": "finished",
    "stdout": "/usr/share/OGAgent\n",
    "stderr": "",
    "jobid": "80618024",
    "rc": 0
  }
]

Ejecutar dos instancias del printer.py en paralelo

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched"}

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched"}

Consultarlos varias veces y ver el progreso:

  • stdout y stderr se van actualizando
  • cuando cada trabajo termina, status cambia a finished y aparece el rc
$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  ...
  {
    "pid": 1156,
    "starttime": "2024-08-01 11:24:57.236169+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "running",
    "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n",
    "stderr": "0.5838482699704048 That's\n",
    "jobid": "ef59b617"
  },
  {
    "pid": 1161,
    "starttime": "2024-08-01 11:25:03.105171+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "running",
    "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n",
    "stderr": "",
    "jobid": "b8a15dad"
  }
]
$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  ...
  {
    "pid": 1156,
    "starttime": "2024-08-01 11:24:57.236169+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "finished",
    "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n0.5838482699704048 I'm fine\n",
    "stderr": "0.5838482699704048 That's\n0.5838482699704048 an error\n",
    "jobid": "ef59b617",
    "rc": 0
  },
  {
    "pid": 1161,
    "starttime": "2024-08-01 11:25:03.105171+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "running",
    "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n0.5256179916328773 Actually,\n0.5256179916328773 I'm fine\n",
    "stderr": "0.5256179916328773 That's\n0.5256179916328773 an error\n",
    "jobid": "b8a15dad"
  }
]
$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  ...
  {
    "pid": 1156,
    "starttime": "2024-08-01 11:24:57.236169+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "finished",
    "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n0.5838482699704048 I'm fine\n",
    "stderr": "0.5838482699704048 That's\n0.5838482699704048 an error\n",
    "jobid": "ef59b617",
    "rc": 0
  },
  {
    "pid": 1161,
    "starttime": "2024-08-01 11:25:03.105171+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": true,
    "status": "finished",
    "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n0.5256179916328773 Actually,\n0.5256179916328773 I'm fine\n",
    "stderr": "0.5256179916328773 That's\n0.5256179916328773 an error\n",
    "jobid": "b8a15dad",
    "rc": 0
  }
]

Ejecutar algo que no existe, o algo para lo que no tenemos permisos, y comprobar que da error

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo /bin/doesntexist |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched"}

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo cat /dev/mem |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched"}

$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  ...
  {
    "pid": 1185,
    "starttime": "2024-08-01 11:29:40.174879+0000",
    "script": "/bin/doesntexist\n",
    "client": true,
    "status": "finished",
    "stdout": "",
    "stderr": "/bin/sh: 1: /bin/doesntexist: not found\n",
    "jobid": "ffdd1732",
    "rc": 127
  },
  {
    "pid": 1190,
    "starttime": "2024-08-01 11:29:45.716868+0000",
    "script": "cat /dev/mem\n",
    "client": true,
    "status": "finished",
    "stdout": "",
    "stderr": "cat: /dev/mem: Permission denied\n",
    "jobid": "c52b733d",
    "rc": 1
  }
]

Ejecutar el printer.py y terminarlo

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"false"}' https://$CLI_IP:8000/opengnsys/script
{"op": "launched", "jobid": "67ba74d1"}

$ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"jobid":"67ba74d1"}' https://$CLI_IP:8000/opengnsys/terminatescript
{}

$ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .
{}
[
  ...
  {
    "pid": 1215,
    "starttime": "2024-08-01 11:33:52.676085+0000",
    "script": "python3 /home/nati/printer.py\n",
    "client": false,
    "status": "finished",
    "stdout": "0.2649740185442616 Hello\n0.2649740185442616 World!\n0.2649740185442616 Actually,\n",
    "stderr": "0.2649740185442616 That's\n",
    "jobid": "726ed6ee",
    "rc": -15
  }
]

El "rc": -15 indica SIGTERM, es lo que tiene que salir. También tenemos stdout y stderr incompletos, lo cual indica que el proceso no terminó.

Limpieza

  • unset CLI_IP TOKEN
  • parar el ogserver de mentira
Historia de usuario: https://ognproject.evlt.uma.es/redmine/issues/494 ### Preparar el entorno Ejecutar una VM linux, windows o macos (por ejemplo con virtualbox, docukrr o qemu). Crear paquete de ogagent-1.3.4 ([wiki redmine](https://ognproject.evlt.uma.es/redmine/projects/opengnsys-dev/wiki/Creaci%C3%B3n_de_paquetes_del_agente)). Subirlo a la VM e instalarlo. Configurar IP del ogserver de mentira (usar la IP del portátil de uno): ``` # sed -i -e '/^remote=/s/192.168.2.10/192.168.1.249/' /usr/share/OGAgent/cfg/ogagent.cfg ``` Ejecutar ogserver de mentira: ``` $ sudo perl -MDateTime -Mojo -E 'a("/*foo" => sub { printf "%s %s %s %s\n", DateTime->now->strftime ("%Y%m%d %H%M%S%z"), $_->tx->remote_address, $_->param("foo"), $_->req->body; $_->render (text => "{}") })->start ("daemon", "-l", "https://192.168.1.249:443")' ``` Reiniciar la VM y comprobar que el agente arranca. Subir a la VM un script que tarde un tiempo en ejecutarse. Llamarlo `printer.py`. Por ejemplo este: ``` import time import sys import random r=random.random() def _out(txt,zzz=3): sys.stdout.write("{} {}".format(r,txt)) sys.stdout.flush() time.sleep(zzz) def _err(txt,zzz=3): sys.stderr.write("{} {}".format(r,txt)) sys.stderr.flush() time.sleep(zzz) _out ("Hello\n") _out ("World!\n") _err ("That's\n") _out ("Actually,\n") _err ("an error\n") _out ("I'm fine\n") ``` Exportar un par de variables para que el PR sea más fácil de seguir: ``` $ export CLI_IP=192.168.1.43 $ export TOKEN=elquesea ``` ### Ejecutar `pwd` en el lado de sistema ``` $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo pwd |base64 -w 0)'","client":"false"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched", "jobid": "cc7707b2"} ``` Para obtener la lista de trabajos hay que llamar a `preparescripts`, esperar un poco y llamar a `getscripts`: ``` $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ { "pid": 1144, "starttime": "2024-08-01 11:22:38.967481+0000", "script": "pwd\n", "client": false, "status": "finished", "stdout": "/\n", "stderr": "", "jobid": "cc7707b2", "rc": 0 } ] ``` ### Ejecutar `pwd` en el lado de usuario ``` $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo pwd |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched"} ``` Debido a la arquitectura asíncrona del agente, el jobid no está inmediatamente disponible y no sale en la salida de la llamada a `/script`. Consultar jobs: ``` $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ { "pid": 1144, "starttime": "2024-08-01 11:22:38.967481+0000", "script": "pwd\n", "client": false, "status": "finished", "stdout": "/\n", "stderr": "", "jobid": "cc7707b2", "rc": 0 }, { "pid": 1150, "starttime": "2024-08-01 11:23:37.751604+0000", "script": "pwd\n", "client": true, "status": "finished", "stdout": "/usr/share/OGAgent\n", "stderr": "", "jobid": "80618024", "rc": 0 } ] ``` ### Ejecutar dos instancias del `printer.py` en paralelo ``` $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched"} $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched"} ``` Consultarlos varias veces y ver el progreso: - stdout y stderr se van actualizando - cuando cada trabajo termina, `status` cambia a `finished` y aparece el `rc` ``` $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ ... { "pid": 1156, "starttime": "2024-08-01 11:24:57.236169+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "running", "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n", "stderr": "0.5838482699704048 That's\n", "jobid": "ef59b617" }, { "pid": 1161, "starttime": "2024-08-01 11:25:03.105171+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "running", "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n", "stderr": "", "jobid": "b8a15dad" } ] ``` ``` $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ ... { "pid": 1156, "starttime": "2024-08-01 11:24:57.236169+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "finished", "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n0.5838482699704048 I'm fine\n", "stderr": "0.5838482699704048 That's\n0.5838482699704048 an error\n", "jobid": "ef59b617", "rc": 0 }, { "pid": 1161, "starttime": "2024-08-01 11:25:03.105171+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "running", "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n0.5256179916328773 Actually,\n0.5256179916328773 I'm fine\n", "stderr": "0.5256179916328773 That's\n0.5256179916328773 an error\n", "jobid": "b8a15dad" } ] ``` ``` $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ ... { "pid": 1156, "starttime": "2024-08-01 11:24:57.236169+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "finished", "stdout": "0.5838482699704048 Hello\n0.5838482699704048 World!\n0.5838482699704048 Actually,\n0.5838482699704048 I'm fine\n", "stderr": "0.5838482699704048 That's\n0.5838482699704048 an error\n", "jobid": "ef59b617", "rc": 0 }, { "pid": 1161, "starttime": "2024-08-01 11:25:03.105171+0000", "script": "python3 /home/nati/printer.py\n", "client": true, "status": "finished", "stdout": "0.5256179916328773 Hello\n0.5256179916328773 World!\n0.5256179916328773 Actually,\n0.5256179916328773 I'm fine\n", "stderr": "0.5256179916328773 That's\n0.5256179916328773 an error\n", "jobid": "b8a15dad", "rc": 0 } ] ``` ### Ejecutar algo que no existe, o algo para lo que no tenemos permisos, y comprobar que da error ``` $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo /bin/doesntexist |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched"} $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo cat /dev/mem |base64 -w 0)'","client":"true"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched"} $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ ... { "pid": 1185, "starttime": "2024-08-01 11:29:40.174879+0000", "script": "/bin/doesntexist\n", "client": true, "status": "finished", "stdout": "", "stderr": "/bin/sh: 1: /bin/doesntexist: not found\n", "jobid": "ffdd1732", "rc": 127 }, { "pid": 1190, "starttime": "2024-08-01 11:29:45.716868+0000", "script": "cat /dev/mem\n", "client": true, "status": "finished", "stdout": "", "stderr": "cat: /dev/mem: Permission denied\n", "jobid": "c52b733d", "rc": 1 } ] ``` ### Ejecutar el `printer.py` y terminarlo ``` $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"script":"'$(echo python3 /home/nati/printer.py |base64 -w 0)'","client":"false"}' https://$CLI_IP:8000/opengnsys/script {"op": "launched", "jobid": "67ba74d1"} $ curl --insecure -X POST --header "Authorization: $TOKEN" --data '{"jobid":"67ba74d1"}' https://$CLI_IP:8000/opengnsys/terminatescript {} $ curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 1.2; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq . {} [ ... { "pid": 1215, "starttime": "2024-08-01 11:33:52.676085+0000", "script": "python3 /home/nati/printer.py\n", "client": false, "status": "finished", "stdout": "0.2649740185442616 Hello\n0.2649740185442616 World!\n0.2649740185442616 Actually,\n", "stderr": "0.2649740185442616 That's\n", "jobid": "726ed6ee", "rc": -15 } ] ``` El `"rc": -15` indica SIGTERM, es lo que tiene que salir. También tenemos stdout y stderr incompletos, lo cual indica que el proceso no terminó. ### Limpieza * `unset CLI_IP TOKEN` * parar el ogserver de mentira
nserrano added 4 commits 2024-08-01 12:54:04 +02:00
nserrano requested review from lgromero 2024-08-01 14:20:27 +02:00
nserrano requested review from vtroshchinskiy 2024-08-01 14:20:27 +02:00
nserrano requested review from aguerrero 2024-08-01 14:20:27 +02:00
nserrano requested review from arantuna 2024-08-01 14:20:27 +02:00
nserrano added 1 commit 2024-08-02 10:47:24 +02:00
nserrano added 1 commit 2024-08-05 15:33:04 +02:00
Collaborator

Funciona todo según lo esperado.

El id del job, ya aparece con 12 caracteres.

Lo único que he notado, es que he tenido que subir el sleep a 3 segundos, no me funcionaba con 1.2s ni con 2s

curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 3; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .

Validado también en MacOS, todo OK.

Funciona todo según lo esperado. El id del job, ya aparece con 12 caracteres. Lo único que he notado, es que he tenido que subir el sleep a 3 segundos, no me funcionaba con 1.2s ni con 2s ` curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/preparescripts; sleep 3; curl --insecure -X GET --header "Authorization: $TOKEN" https://$CLI_IP:8000/opengnsys/getscripts |jq .` Validado también en MacOS, todo OK.
nserrano merged commit 4b5193105c into main 2024-08-07 14:03:24 +02:00
Sign in to join this conversation.
No Label
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: opengnsys/ogagent#5
There is no content yet.