Compare commits

..

25 Commits

Author SHA1 Message Date
Juan Manuel Bardallo 2bc768fcc3 Merge branch 'devel'
ogrepository/pipeline/head This commit looks good Details
ogrepository/pipeline/tag This commit looks good Details
2026-09-17 12:31:25 +02:00
Juan Manuel Bardallo 4e4abdccaf docs: add release notes for version 0.10.7
ogrepository/pipeline/head This commit looks good Details
2026-09-17 12:30:38 +02:00
Juan Manuel Bardallo 253660d628 fix: adjust seek offset for small files in createTorrentSum 2026-09-17 12:21:58 +02:00
Juan Manuel Bardallo 063edb2207 Merge pull request 'devel 0.10.6' (#48) from devel into main
ogrepository/pipeline/head This commit looks good Details
ogrepository/pipeline/tag This commit looks good Details
Reviewed-on: #48
2026-07-31 09:50:38 +02:00
Juan Manuel Bardallo 6215f3d01b Docs: update CHANGELOG for 2MB target pieces fix
ogrepository/pipeline/head This commit looks good Details
2026-07-29 09:31:32 +02:00
Juan Manuel Bardallo 050a69ed65 Perf(ogrepository): increase target_pieces for torrent creation to cap piece size at 2MB for images up to 400GB
ogrepository/pipeline/head This commit looks good Details
2026-07-29 09:31:08 +02:00
Juan Manuel Bardallo 2a49259f72 Fix(ogrepository): make git repo deletion idempotent
ogrepository/pipeline/head This commit looks good Details
2026-07-28 12:48:52 +02:00
Juan Manuel Bardallo 4957129463 Docs: update CHANGELOG for git repository deletion API
ogrepository/pipeline/head This commit looks good Details
2026-07-28 12:32:41 +02:00
Juan Manuel Bardallo e6644191c1 Feat(ogrepository): implement forgejo git repository deletion endpoint
ogrepository/pipeline/head This commit looks good Details
2026-07-28 10:47:23 +02:00
Juan Manuel Bardallo 0267354899 Optimize mktorrent piece size calculation
ogrepository/pipeline/head This commit looks good Details
2026-07-27 13:40:34 +02:00
Nicolas Arenas 3139235b07 Update Jenkinsfile
ogrepository/pipeline/head This commit looks good Details
2026-05-14 18:07:47 +02:00
Nicolas Arenas ffc00f07d2 Fix permissions for update package it not oggit was installed previously
ogrepository/pipeline/head This commit looks good Details
2026-04-14 20:27:01 +02:00
Nicolas Arenas e755273275 Add .venv and .vscode to .gitignore
ogrepository/pipeline/head This commit looks good Details
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 14:46:27 +01:00
Natalia Serrano b2066d3e5d Merge pull request 'refs #2831 set safe.directory just once' (#47) from safe_dir into main
ogrepository/pipeline/head This commit looks good Details
ogrepository/pipeline/tag This commit looks good Details
Reviewed-on: #47
2025-09-30 12:38:34 +02:00
Natalia Serrano eace7142ae refs #2831 set safe.directory just once 2025-09-30 12:37:57 +02:00
Luis Gerardo Romero Garcia 8bd472fd6b Merge pull request 'refs #2789 Adds necessary ogboot commands to sudoers' (#46) from sudoers-ogboot into main
ogrepository/pipeline/head This commit looks good Details
Reviewed-on: #46
2025-09-22 13:12:34 +02:00
Luis Gerardo Romero Garcia 6ea7699ca5 refs #2789 Adds necessary ogboot commands to sudoers 2025-09-22 13:11:13 +02:00
Natalia Serrano c9c8952741 Merge pull request 'refs #2830 create tags via a query to the forgejo API' (#45) from forgejo-tags into main
ogrepository/pipeline/head This commit looks good Details
ogrepository/pipeline/tag This commit looks good Details
Reviewed-on: #45
2025-09-18 13:53:09 +02:00
Natalia Serrano e0b9a036ca refs #2830 create tags via a query to the forgejo API 2025-09-18 13:52:34 +02:00
Vadim vtroshchinskiy 206853bd23 Merge pull request 'refs #2509 Add missing Git swagger endpoints' (#44) from complete-git-swaggger-endpoints into main
ogrepository/pipeline/head This commit looks good Details
Reviewed-on: #44
2025-09-03 15:58:14 +02:00
Nicolas Arenas 1761ec4378 refs #2741 Test auto launch puublish
ogrepository/pipeline/head This commit looks good Details
2025-09-03 10:25:44 +02:00
Nicolas Arenas d8f3eb0a19 refs #2741 Test auto launch puublish
ogrepository/pipeline/head This commit looks good Details
2025-09-03 10:01:46 +02:00
Vadim vtroshchinskiy 4447126891 refs #2509 Add missing Git swagger endpoints 2025-09-03 00:18:09 +02:00
Nicolas Arenas 49f72febf8 Merge pull request 'fix_checked_file' (#43) from fix_checked_file into main
ogrepository/pipeline/head This commit looks good Details
Reviewed-on: #43
2025-09-01 16:48:04 +02:00
Nicolas Arenas f6617aee77 Merge pull request 'fix_checked_file' (#42) from fix_checked_file into main
ogrepository/pipeline/head This commit looks good Details
Reviewed-on: #42
2025-09-01 12:20:22 +02:00
9 changed files with 432 additions and 84 deletions

2
.gitignore vendored
View File

@ -7,3 +7,5 @@ debian/files
debian/*.debhelper debian/*.debhelper
debian/*.debhelper.log debian/*.debhelper.log
debian/debhelper-build-stamp debian/debhelper-build-stamp
.venv/
.vscode/

View File

@ -1,11 +1,49 @@
# Changelog # Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.10.7] - 2026-09-17
### Fixed
- **createTorrentSum**: Corregido error (`OSError: [Errno 22] Invalid argument`) al calcular la suma de verificación MD5 de los ficheros de imagen cuando estos ocupan menos de 1 MB, ajustando dinámicamente el `seek_offset` al tamaño real del fichero.
## [0.10.6] - 2026-07-27
### Added
- API de Repositorios Git (`repo_api.py`): Implementado el endpoint `DELETE /ogrepository/v1/git/repositories/<repo>` que sustituye al antiguo código "Not implemented". Ahora se encarga de eliminar el repositorio correctamente de la base de datos de Forgejo y de purgar el directorio `.git` del disco duro.
### Changed
- **createTorrentSum**: Optimizada la creación de ficheros torrent (`mktorrent`) calculando el tamaño de pieza (`-l`) de forma dinámica según el tamaño de la imagen, evitando archivos `.torrent` con cientos de miles de piezas que saturaban el tracker en imágenes muy grandes. Se ha incrementado `target_pieces` a 200.000 para garantizar que las piezas no superen los 2 MB en imágenes de hasta 400 GB.
## [0.10.5] - 2026-04-14
### Fixed
- Force 755 permissions on `/opt/opengnsys` in `preinst` to prevent `oggit` user from being locked out when `useradd` creates the directory with a restrictive umask (e.g. 027), which caused `opengnsys-forgejo` to fail to start during installation before `postinst` could apply the correct permissions
- Ensure forgejo working directory (`var/lib/forgejo/work`) is created in `postinst` before setting ownership, and restart `opengnsys-forgejo` after fixing permissions so it recovers from a failed state caused by the ordering between dependency and package configuration
## [0.10.4] - 2025-09-30
### Fixed
- Set safe.directory just once
## [0.10.3] - 2025-09-18
### Changed
- Create tags via a query to the forgejo API
## [0.10.2] - 2025-09-01 ## [0.10.2] - 2025-09-01
## Fixed ### Fixed
- Corregida la logica en el endpoint create torrent sum - Corregida la logica en el endpoint create torrent sum
- Mejoras en los mensajes de logs - Mejoras en los mensajes de logs
- Mejoras en la validación de los archivos info e info.checked - Mejoras en la validación de los archivos info e info.checked
## [0.10.1] - 2025-08-25 ## [0.10.1] - 2025-08-25
@ -21,13 +59,13 @@
- OgGit functionality (#2371, #2363, #2363, #2317) - OgGit functionality (#2371, #2363, #2363, #2317)
### Removed ### Removed
- Removed unused BitTorrent-related packages and logic - Removed unused BitTorrent-related packages and logic
## [0.9.0] - 2025-06-25 ## [0.9.0] - 2025-06-25
## Added ### Added
- Changed old tools for tools non dependant of Pyhton2 in repo - Changed old tools for tools non dependant of Pyhton2 in repo
- mktorrent to handle creation of torrent files - mktorrent to handle creation of torrent files

View File

@ -62,6 +62,19 @@ pipeline {
} }
} }
stage('Generate Changelog (Devel)') {
when {
branch 'devel'
}
steps {
script {
def devName = params.DEV_NAME ? params.DEV_NAME : env.DEFAULT_DEV_NAME
def devEmail = params.DEV_EMAIL ? params.DEV_EMAIL : env.DEFAULT_DEV_EMAIL
generateDebianChangelog(env.BUILD_DIR, devName, devEmail, "devel")
}
}
}
stage('Build') { stage('Build') {
steps { steps {
script { script {
@ -97,17 +110,44 @@ pipeline {
} }
} }
} }
stage ('Publish to Debian Repository (Devel)') {
when {
branch 'devel'
}
agent { label 'debian-repo' }
steps {
script {
def versionPattern = "-${env.BUILD_NUMBER}~devel"
publicarEnAptly('/var/tmp/opengnsys/debian-repo/ogrepository', 'devel', versionPattern)
}
}
}
} }
post { post {
success {
script {
// Solo lanzar cuando el build sea exitoso y en la rama main
if (env.BRANCH_NAME == 'main') {
build job: 'Aptly publish nightly repository',
wait: false,
parameters: [
string(name: 'TRIGGERED_BY', value: "${env.JOB_NAME}-${env.BUILD_NUMBER}")
]
}
if (env.BRANCH_NAME == 'devel') {
build job: 'Aptly publish develop repository',
wait: false,
parameters: [
string(name: 'TRIGGERED_BY', value: "${env.JOB_NAME}-${env.BUILD_NUMBER}")
]
}
}
}
always { always {
notifyBuildStatus('narenas@qindel.com') notifyBuildStatus('opengnsys@qindel.com')
} }
} }
} }
// stage ('Publish to Debian Repository') {
// agent { label 'debian-repo' }
// steps {
// sh "aptly repo add opengnsys-devel /var/tmp/opengnsys/debian-repo/*.deb"
// }
// }

View File

@ -14,15 +14,13 @@ Librerías Python requeridas: - flask (se puede instalar con "sudo apt install p
- flasgger (se puede instalar con "sudo apt install python3-flasgger") - flasgger (se puede instalar con "sudo apt install python3-flasgger")
""" """
# --------------------------------------------------------------------------------------------
# IMPORTS
# --------------------------------------------------------------------------------------------
from flask import Flask, jsonify, request from flask import Flask, jsonify, request
import os import os
import subprocess import subprocess
import json import json
from time import sleep import time
import sys
import git
import paramiko import paramiko
import logging import logging
import threading import threading
@ -34,13 +32,8 @@ from systemd import journal
# Imports para Swagger: # Imports para Swagger:
from flasgger import Swagger from flasgger import Swagger
import yaml import yaml
import time
# --------------------------------------------------------------------------------------------
# VARIABLES
# --------------------------------------------------------------------------------------------
repo_path = '/opt/opengnsys/ogrepository/images/' # No borrar la barra final repo_path = '/opt/opengnsys/ogrepository/images/' # No borrar la barra final
vm_path = '/opt/opengnsys/ogrepository/images_virtual/' # No borrar la barra final vm_path = '/opt/opengnsys/ogrepository/images_virtual/' # No borrar la barra final
script_path = '/opt/opengnsys/ogrepository/bin' script_path = '/opt/opengnsys/ogrepository/bin'
@ -48,21 +41,10 @@ repo_file = '/opt/opengnsys/ogrepository/etc/repoinfo.json'
trash_file = '/opt/opengnsys/ogrepository/etc/trashinfo.json' trash_file = '/opt/opengnsys/ogrepository/etc/trashinfo.json'
config_file = '/opt/opengnsys/ogrepository/etc/ogAdmRepo.cfg' config_file = '/opt/opengnsys/ogrepository/etc/ogAdmRepo.cfg'
# --------------------------------------------------------------------------------------------
# GIT
# --------------------------------------------------------------------------------------------
REPOSITORIES_BASE_PATH = "/opt/opengnsys/ogrepository/oggit/git/" REPOSITORIES_BASE_PATH = "/opt/opengnsys/ogrepository/oggit/git/"
OGGIT_USER = "oggit" OGGIT_USER = "oggit"
OGGIT_FORGEJO_PORT = 3100 OGGIT_FORGEJO_PORT = 3100
import sys
import git
import paramiko
# -------------------------------------------------------------------------------------------- # --------------------------------------------------------------------------------------------
# FUNCTIONS # FUNCTIONS
@ -273,7 +255,7 @@ def check_lock_local(image_file_path, job_id):
journal.send("Running function 'check_lock_local'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Running function 'check_lock_local'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 30 segundos, para dar tiempo a que se cree el archivo ".lock": # Esperamos 30 segundos, para dar tiempo a que se cree el archivo ".lock":
sleep(30) time.sleep(30)
# Creamos un bucle infinito: # Creamos un bucle infinito:
while True: while True:
@ -310,7 +292,7 @@ def check_lock_local(image_file_path, job_id):
journal.send("Task in process (.lock file exists)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Task in process (.lock file exists)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 1 minuto para volver a realizar la comprobación: # Esperamos 1 minuto para volver a realizar la comprobación:
sleep(60) time.sleep(60)
# --------------------------------------------------------- # ---------------------------------------------------------
@ -335,7 +317,7 @@ def check_remote_backup(image_name, remote_ip, remote_user, remote_path, job_id)
sftp_client = ssh_client.open_sftp() sftp_client = ssh_client.open_sftp()
# Esperamos 30 segundos antes de empezar a realizar la comprobación: # Esperamos 30 segundos antes de empezar a realizar la comprobación:
sleep(30) time.sleep(30)
# Creamos una lista con las extensiones de los archivos asociados a la imagen (incluyendo ninguna extensión, que corresponde a la propia imagen): # Creamos una lista con las extensiones de los archivos asociados a la imagen (incluyendo ninguna extensión, que corresponde a la propia imagen):
extensions = ['', '.size', '.sum', '.full.sum', '.info'] extensions = ['', '.size', '.sum', '.full.sum', '.info']
@ -424,7 +406,7 @@ def check_remote_backup(image_name, remote_ip, remote_user, remote_path, job_id)
journal.send("Task in process (backup incomplete)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Task in process (backup incomplete)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 1 minuto para volver a realizar la comprobación: # Esperamos 1 minuto para volver a realizar la comprobación:
sleep(60) time.sleep(60)
# Ya fuera del bucle, cerramos el cliente SSH y el cliente SFTP: # Ya fuera del bucle, cerramos el cliente SSH y el cliente SFTP:
ssh_client.close() ssh_client.close()
@ -490,7 +472,7 @@ def check_aux_files(image_file_path, job_id):
recall_ogcore(data) recall_ogcore(data)
break break
# Esperamos 10 segundos para volver a realizar la comprobación: # Esperamos 10 segundos para volver a realizar la comprobación:
sleep(10) time.sleep(10)
# --------------------------------------------------------- # ---------------------------------------------------------
@ -506,7 +488,7 @@ def check_virtual_image_conversion(image_name, job_id):
journal.send("Running function 'check_virtual_image_conversion'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Running function 'check_virtual_image_conversion'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 30 segundos, para dar tiempo a que se cree algún archivo: # Esperamos 30 segundos, para dar tiempo a que se cree algún archivo:
sleep(30) time.sleep(30)
# Construimos la ruta de la imagen (una vez convertida): # Construimos la ruta de la imagen (una vez convertida):
image_file_path = f"{repo_path}{image_name}.img" image_file_path = f"{repo_path}{image_name}.img"
@ -547,7 +529,7 @@ def check_virtual_image_conversion(image_name, job_id):
journal.send("Task in process (Conversion not finalized)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Task in process (Conversion not finalized)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 1 minuto para volver a realizar la comprobación: # Esperamos 1 minuto para volver a realizar la comprobación:
sleep(60) time.sleep(60)
# --------------------------------------------------------- # ---------------------------------------------------------
@ -562,7 +544,7 @@ def check_virtual_image_reconversion(image_name, vm_extension, job_id):
journal.send("Running function 'check_virtual_image_reconversion'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Running function 'check_virtual_image_reconversion'...", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 30 segundos, para dar tiempo a que se cree algún archivo: # Esperamos 30 segundos, para dar tiempo a que se cree algún archivo:
sleep(30) time.sleep(30)
# Construimos la ruta de la imagen virtual (una vez exportada), y la ruta de exportación: # Construimos la ruta de la imagen virtual (una vez exportada), y la ruta de exportación:
virtual_image_file_path = f"{vm_path}export/{image_name}.{vm_extension}" virtual_image_file_path = f"{vm_path}export/{image_name}.{vm_extension}"
@ -604,7 +586,7 @@ def check_virtual_image_reconversion(image_name, vm_extension, job_id):
journal.send("Task in process (Conversion not finalized)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send("Task in process (Conversion not finalized)", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
# Esperamos 1 minuto para volver a realizar la comprobación: # Esperamos 1 minuto para volver a realizar la comprobación:
sleep(60) time.sleep(60)
# --------------------------------------------------------- # ---------------------------------------------------------
@ -1504,7 +1486,7 @@ def send_udpcast():
result = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='UTF8') result = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='UTF8')
# Comprobamos si está corriendo el proceso correspondiente de "udp-sender" (esperando 5 segundos para darle tiempo a iniciarse): # Comprobamos si está corriendo el proceso correspondiente de "udp-sender" (esperando 5 segundos para darle tiempo a iniciarse):
sleep(5) time.sleep(5)
process_running = search_process('udp-sender', f"{param_dict['name']}.{param_dict['extension']}") process_running = search_process('udp-sender', f"{param_dict['name']}.{param_dict['extension']}")
# Evaluamos el resultado de la ejecución, y devolvemos una respuesta: # Evaluamos el resultado de la ejecución, y devolvemos una respuesta:
@ -1577,7 +1559,7 @@ def send_uftp():
result = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='UTF8') result = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='UTF8')
# Comprobamos si está corriendo el proceso correspondiente de "uftp" (esperando 5 segundos para darle tiempo a iniciarse): # Comprobamos si está corriendo el proceso correspondiente de "uftp" (esperando 5 segundos para darle tiempo a iniciarse):
sleep(5) time.sleep(5)
process_running = search_process('uftp', f"{param_dict['name']}.{param_dict['extension']}") process_running = search_process('uftp', f"{param_dict['name']}.{param_dict['extension']}")
# Evaluamos el resultado de la ejecución, y devolvemos una respuesta: # Evaluamos el resultado de la ejecución, y devolvemos una respuesta:
@ -1659,7 +1641,7 @@ def send_p2p():
# Comprobamos si el tracker y el seeder están corriendo, y si apuntan al directorio que le hemos pasado # Comprobamos si el tracker y el seeder están corriendo, y si apuntan al directorio que le hemos pasado
# (esperamos 10 segundos antes de hacerlo, porque los procesos no se inician inmediatamente): # (esperamos 10 segundos antes de hacerlo, porque los procesos no se inician inmediatamente):
sleep(10) time.sleep(10)
seeder_running = search_process('aria2c', f"{param_dict['name']}.img.torrent") # El seeder se ejecuta con "aria2c" y el nombre de la imagen como parámetro seeder_running = search_process('aria2c', f"{param_dict['name']}.img.torrent") # El seeder se ejecuta con "aria2c" y el nombre de la imagen como parámetro
@ -2267,6 +2249,11 @@ def rename_image():
# #
# ----------------------------------------------------------- # -----------------------------------------------------------
def repo_set_safe_directory (repo, path):
try:
repo.git.config ('--get', 'safe-directory')
except:
repo.git.config ('--global', '--add', 'safe.directory', path)
def git_forgejo_post_command(url, json_data): def git_forgejo_post_command(url, json_data):
token = "" token = ""
@ -2301,6 +2288,37 @@ def git_forgejo_create_repo(repo):
"private" : False "private" : False
}) })
def git_forgejo_delete_command(url):
token = ""
with open("/opt/opengnsys/etc/ogGitApiToken.cfg", "r", encoding='utf-8') as token_file:
token = token_file.read().strip()
if url[0] == "/":
url = url[1:]
url = f"http://localhost:{OGGIT_FORGEJO_PORT}/{url}"
journal.send(f"Executing forgejo DELETE to {url}...", PRIORITY=journal.LOG_DEBUG, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
r = requests.delete(
url,
headers = { 'Authorization' : f"token {token}" },
timeout = 60
)
journal.send(f"Request status was {r.status_code}, content {r.content}", PRIORITY=journal.LOG_DEBUG, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return r.status_code, r.content.decode('utf-8')
def git_forgejo_delete_repo(repo):
return git_forgejo_delete_command(f"/api/v1/repos/oggit/{repo}")
def git_forgejo_create_tag(repo,tag,msg,target):
return git_forgejo_post_command(f"/api/v1/repos/oggit/{repo}/tags", {
"message": msg,
"tag_name": tag,
"target": target
})
def git_forgejo_add_sshkey(pubkey, description = ""): def git_forgejo_add_sshkey(pubkey, description = ""):
return git_forgejo_post_command("/api/v1/user/keys", return git_forgejo_post_command("/api/v1/user/keys",
@ -2315,7 +2333,7 @@ def git_compact_repository_task(repo, job_id):
git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git") git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git")
git_repo = git.Repo(git_repo_path) git_repo = git.Repo(git_repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', git_repo_path) repo_set_safe_directory (git_repo, git_repo_path)
git_repo.git.gc() git_repo.git.gc()
@ -2332,7 +2350,7 @@ def git_sync_repository_task(repo, remote_repository, job_id):
git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git") git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git")
git_repo = git.Repo(git_repo_path) git_repo = git.Repo(git_repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', git_repo_path) repo_set_safe_directory (git_repo, git_repo_path)
# Recreate the remote every time, it might change # Recreate the remote every time, it might change
if "backup" in git_repo.remotes: if "backup" in git_repo.remotes:
@ -2360,7 +2378,7 @@ def git_backup_repository_task(repo, params, job_id):
git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git") git_repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git")
git_repo = git.Repo(git_repo_path) git_repo = git.Repo(git_repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', git_repo_path) repo_set_safe_directory (git_repo, git_repo_path)
ssh = paramiko.SSHClient() ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
@ -2460,9 +2478,23 @@ def git_create_repository():
return jsonify({"status": "Repository created"}), 201 return jsonify({"status": "Repository created"}), 201
@app.route("/ogrepository/v1/git/repositories", methods=['DELETE']) @app.route("/ogrepository/v1/git/repositories/<string:repo>", methods=['DELETE'])
def git_delete_repository(): def git_delete_repository(repo):
return jsonify({"error" : "Not implemented"}), 500 repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git")
status, content = git_forgejo_delete_repo(repo)
if status not in (204, 404):
journal.send(f"Can't delete repository {repo} from forgejo: {content}", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"error" : f"Failed to delete from forgejo: {content}"}), 500
if os.path.isdir(repo_path):
import shutil
shutil.rmtree(repo_path, ignore_errors=True)
journal.send(f"Repository {repo} deleted", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"status": "Repository deleted"}), 200
else:
journal.send(f"Repository {repo} already deleted or not found at {repo_path}", PRIORITY=journal.LOG_INFO, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"status": "Repository already deleted"}), 200
@app.route("/ogrepository/v1/git/repositories/<string:repo>/sync", methods=['POST']) @app.route("/ogrepository/v1/git/repositories/<string:repo>/sync", methods=['POST'])
def git_sync_repository(repo): def git_sync_repository(repo):
@ -2575,7 +2607,7 @@ def git_get_branches(repo):
return jsonify({"error": "Repository not found"}), 404 return jsonify({"error": "Repository not found"}), 404
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
branches = [] branches = []
for branch in git_repo.branches: for branch in git_repo.branches:
@ -2611,7 +2643,7 @@ def git_get_commits(repo, branch):
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
if not branch in git_repo.branches: if not branch in git_repo.branches:
@ -2701,7 +2733,7 @@ def git_create_branch(repo):
return jsonify({"error": "Repository not found"}), 404 return jsonify({"error": "Repository not found"}), 404
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
data = request.json data = request.json
if data is None: if data is None:
@ -2746,7 +2778,7 @@ def git_delete_branch(repo, branch):
return {"error": "Repository not found"}, 404 return {"error": "Repository not found"}, 404
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
if not branch in git_repo.branches: if not branch in git_repo.branches:
@ -2778,7 +2810,7 @@ def git_list_tags(repo):
return jsonify({"error": "Repository not found"}), 404 return jsonify({"error": "Repository not found"}), 404
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
tags = [] tags = []
@ -2818,14 +2850,6 @@ def git_create_tag(repo):
- 409: A JSON object with an "error" key containing the message "Tag already exists" - 409: A JSON object with an "error" key containing the message "Tag already exists"
""" """
repo_path = os.path.join(REPOSITORIES_BASE_PATH, OGGIT_USER, repo + ".git")
if not os.path.isdir(repo_path):
journal.send(f"Can't create tag. Repository storage at {REPOSITORIES_BASE_PATH} not found", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"error": "Repository not found"}), 404
git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path)
data = request.json data = request.json
if data is None: if data is None:
journal.send(f"Can't create tag. JSON post data missing", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send(f"Can't create tag. JSON post data missing", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
@ -2845,21 +2869,16 @@ def git_create_tag(repo):
if "message" in data: if "message" in data:
commit_message = data["message"] commit_message = data["message"]
if tag in git_repo.tags:
journal.send(f"Can't create tag. Already found in repository {repo}", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"error": "Tag already exists"}), 409
try: try:
git_repo.create_tag(tag, ref = data["commit"], message = commit_message) res = git_forgejo_create_tag(repo,tag,commit_message,data["commit"])
except git.exc.GitCommandError as ge: except Exception as e:
if "not a valid tag name" in ge.stderr: journal.send (str(e), PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
journal.send(f"Tag name {tag} is invalid", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") return jsonify({"error" : f"git_forgejo_create_tag failed: {e}"}), 500
return jsonify({"error" : "Invalid tag name"}), 400
else:
journal.send(f"Git error {ge}", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"error" : "Error when performing git command"}), 500
if 200 != res[0]:
j = json.loads(res[1])
journal.send (j["message"], PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"error" : j["message"]}), 409
journal.send(f"Tag {tag} created in repo {repo}", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG") journal.send(f"Tag {tag} created in repo {repo}", PRIORITY=journal.LOG_ERR, SYSLOG_IDENTIFIER="ogrepo-api_DEBUG")
return jsonify({"status": "created"}), 200 return jsonify({"status": "created"}), 200
@ -2883,7 +2902,7 @@ def git_delete_tag(repo, tag):
return {"error": "Repository not found"}, 404 return {"error": "Repository not found"}, 404
git_repo = git.Repo(repo_path) git_repo = git.Repo(repo_path)
git_repo.git.config('--global', '--add', 'safe.directory', repo_path) repo_set_safe_directory (git_repo, repo_path)
if not tag in git_repo.tags: if not tag in git_repo.tags:

View File

@ -2087,6 +2087,203 @@ paths:
error: error:
type: string type: string
example: "Parameters missing" example: "Parameters missing"
/ogrepository/v1/git/repositories/{repository}:
delete:
summary: "Eliminar repositorio"
description: |
Elimina un repositorio de Git
tags:
- "Git"
parameters:
- name: repository
in: path
required: true
type: string
description: "Nombre de repositorio"
responses:
"200":
description: "Repositorio eliminado"
schema:
type: object
properties:
status:
type: string
example: deleted
"404":
description: "Repositorio no encontrado"
schema:
type: object
properties:
status:
type: string
example: not found
"500":
description: "Excepción"
schema:
type: object
properties:
success:
type: boolean
example: false
exception:
type: string
example: "(Exception description)"
/ogrepository/v1/git/repositories/{repository}/sync:
post:
summary: "Sincronizar con repositorio remoto"
description: |
Sincroniza un repositorio de Git con uno remoto.
tags:
- "Git"
parameters:
- name: repository
in: path
required: true
type: string
description: "Nombre de repositorio"
- name: JSON
in: body
required: true
description: |
* **remote_repository** - Reposition remoto al que hacer push
responses:
"200":
description: "Repositorio en proceso de sincronización"
schema:
type: object
properties:
success:
type: boolean
example: True
output:
type: string
example: "Synchronizing..."
job_id:
type: string
example: "GitSync_02345"
"404":
description: "Repositorio no encontrado"
schema:
type: object
properties:
status:
type: string
example: not found
"500":
description: "Excepción"
schema:
type: object
properties:
success:
type: boolean
example: false
exception:
type: string
example: "(Exception description)"
/ogrepository/v1/git/repositories/{repository}/backup:
post:
summary: "Sincronizar con repositorio remoto"
description: |
Sincroniza un repositorio de Git con uno remoto.
tags:
- "Git"
parameters:
- name: repository
in: path
required: true
type: string
description: "Nombre de repositorio"
- name: JSON
in: body
required: true
description: |
* **ssh_server** - Servidor SSH al que enviar el backup
* **ssh_port** - Puerto del servidor SSH al que enviar el backup
* **ssh_user** - Usuario del servidor SSH al que enviar el backup
* **filename** - Nombre de archivo del backup
responses:
"200":
description: "Repositorio en proceso de sincronización"
schema:
type: object
properties:
success:
type: boolean
example: True
output:
type: string
example: "Backing up..."
job_id:
type: string
example: "GitBackup_02345"
"404":
description: "Repositorio no encontrado"
schema:
type: object
properties:
status:
type: string
example: not found
"500":
description: "Excepción"
schema:
type: object
properties:
success:
type: boolean
example: false
exception:
type: string
example: "(Exception description)"
/ogrepository/v1/git/repositories/{repository}/compact:
post:
summary: "Compactar repositorio"
description: |
Compacta un repositorio de Git para optimizar el rendimiento y liberar espacio
tags:
- "Git"
parameters:
- name: repository
in: path
required: true
type: string
description: "Nombre de repositorio"
responses:
"200":
description: "Repositorio en proceso de compactación"
schema:
type: object
properties:
success:
type: boolean
example: True
output:
type: string
example: "Compacting..."
job_id:
type: string
example: "GitGC_02345"
"404":
description: "Repositorio no encontrado"
schema:
type: object
properties:
status:
type: string
example: not found
"500":
description: "Excepción"
schema:
type: object
properties:
success:
type: boolean
example: false
exception:
type: string
example: "(Exception description)"
/ogrepository/v1/git/repositories/{repository}/tags: /ogrepository/v1/git/repositories/{repository}/tags:
get: get:
summary: "Obtener lista de tags" summary: "Obtener lista de tags"

View File

@ -96,8 +96,13 @@ def get_md5_sum(file_path, megabytes=1):
Se utiliza para crear el archivo ".sum" (para transferencias Unicast y Multicast). Se utiliza para crear el archivo ".sum" (para transferencias Unicast y Multicast).
""" """
hash_md5 = hashlib.md5() hash_md5 = hashlib.md5()
file_size = os.path.getsize(file_path)
seek_offset = -megabytes * 1024 * 1024
if file_size < (-seek_offset):
seek_offset = -file_size
with open(file_path, "rb") as f: with open(file_path, "rb") as f:
f.seek(-megabytes * 1024 * 1024, os.SEEK_END) f.seek(seek_offset, os.SEEK_END)
data = f.read(megabytes * 1024 * 1024) data = f.read(megabytes * 1024 * 1024)
hash_md5.update(data) hash_md5.update(data)
return hash_md5.hexdigest() return hash_md5.hexdigest()
@ -134,8 +139,18 @@ def create_torrent(file_path, torrent_file, datafullsum):
repo_ip = get_IPlocal() repo_ip = get_IPlocal()
tracker_url = f"http://{repo_ip}:6969/announce" tracker_url = f"http://{repo_ip}:6969/announce"
import math
file_size = os.path.getsize(file_path)
target_pieces = 200000
piece_size_target = file_size / target_pieces
if piece_size_target <= 262144:
l_val = 18
else:
l_val = math.ceil(math.log(piece_size_target, 2))
l_val = max(18, min(26, l_val))
# Creamos una litas para ejecutar el comando mktorrent para crear el archivo torrent # Creamos una litas para ejecutar el comando mktorrent para crear el archivo torrent
splitted_cmd = f"nice -n 0 mktorrent -a {tracker_url} -c {datafullsum} -o {torrent_file} {file_path}".split() splitted_cmd = f"nice -n 0 mktorrent -l {l_val} -a {tracker_url} -c {datafullsum} -o {torrent_file} {file_path}".split()
print(f"Sending command: {' '.join(splitted_cmd)}") print(f"Sending command: {' '.join(splitted_cmd)}")
# Ejecutamos el comando en el sistema, e imprimimos el resultado: # Ejecutamos el comando en el sistema, e imprimimos el resultado:

View File

@ -110,9 +110,13 @@ chmod 755 /opt/opengnsys/ogrepository/bin/*
# oggit -- these are from opengnsys-forgejo, and we're overwriting their # oggit -- these are from opengnsys-forgejo, and we're overwriting their
# permissions above. # permissions above.
# #
# Check if the paths exist just in case the opengnsys-forgejo package # Ensure the forgejo working directory exists and has the right ownership
# stops being a dependency. # before the service tries to start. This is needed because opengnsys-forgejo
if [ -d "/opt/opengnsys/ogrepository/var/lib/forgejo/" ] ; then # is configured (and starts its service) before ogrepository's postinst runs,
# so without this the service fails with CHDIR/Permission denied.
if getent passwd oggit > /dev/null 2>&1; then
mkdir -p /opt/opengnsys/ogrepository/var/lib/forgejo/work
mkdir -p /opt/opengnsys/ogrepository/var/lib/forgejo/data
chown -R oggit:oggit /opt/opengnsys/ogrepository/var/lib/forgejo/ chown -R oggit:oggit /opt/opengnsys/ogrepository/var/lib/forgejo/
fi fi
@ -120,6 +124,13 @@ if [ -d "/opt/opengnsys/ogrepository/oggit/" ] ; then
chown -R oggit:oggit /opt/opengnsys/ogrepository/oggit/ chown -R oggit:oggit /opt/opengnsys/ogrepository/oggit/
fi fi
# Restart forgejo now that permissions are correct (it may have failed during
# opengnsys-forgejo's own postinst due to the chown above overwriting its dirs)
if getent passwd oggit > /dev/null 2>&1; then
systemctl reset-failed opengnsys-forgejo 2>/dev/null || true
systemctl restart opengnsys-forgejo 2>/dev/null || true
fi
# Install http server stuff # Install http server stuff
# Reiniciar servicios si es necesario # Reiniciar servicios si es necesario
# systemctl restart nombre_del_servicio # systemctl restart nombre_del_servicio

View File

@ -24,6 +24,13 @@ else
echo "Creando el usuario $USER con home en $HOME_DIR." echo "Creando el usuario $USER con home en $HOME_DIR."
useradd -m -d "$HOME_DIR" -s /bin/bash "$USER" useradd -m -d "$HOME_DIR" -s /bin/bash "$USER"
fi fi
# Forzar 755 en /opt/opengnsys independientemente del umask del sistema.
# useradd -m puede crear el directorio con 750 si el umask es 027, lo que
# impide que el usuario oggit (de opengnsys-forgejo) atraviese el árbol
# antes de que el postinst de ogrepository aplique su propio chmod.
mkdir -p "$HOME_DIR"
chmod 755 "$HOME_DIR"
echo "Añadiendo el repositorio de Debian Buster en $REPO_FILE..." echo "Añadiendo el repositorio de Debian Buster en $REPO_FILE..."
# Crear el directorio si no existe # Crear el directorio si no existe

View File

@ -10,6 +10,25 @@ Cmnd_Alias MOUNT_RECOVERY = \
/usr/bin/umount /mnt/recovery/sys, \ /usr/bin/umount /mnt/recovery/sys, \
/usr/bin/umount -l /mnt/recovery /usr/bin/umount -l /mnt/recovery
Cmnd_Alias OGBOOT = \
/opt/bin/oglivecli, \
/usr/bin/chmod, \
/usr/bin/chown, \
/usr/bin/md5sum, \
/usr/bin/smbpasswd, \
/usr/bin/cat, \
/usr/bin/tee, \
/usr/bin/sed, \
/usr/bin/gzip, \
/usr/bin/lz4, \
/usr/bin/cpio, \
/usr/bin/find, \
/bin/tee, \
/usr/bin/dd, \
/usr/bin/mkfs.ext4, \
/usr/bin/rsync, \
/opt/opengnsys/ogboot/lib/*.iso /mnt
Cmnd_Alias CHROOT_GRUB = \ Cmnd_Alias CHROOT_GRUB = \
/usr/sbin/chroot /mnt/recovery /usr/sbin/grub-install --target=i386-pc *, \ /usr/sbin/chroot /mnt/recovery /usr/sbin/grub-install --target=i386-pc *, \
/usr/sbin/chroot /mnt/recovery /usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg /usr/sbin/chroot /mnt/recovery /usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg
@ -28,4 +47,4 @@ Cmnd_Alias KILL_BT = \
Cmnd_Alias PYTHON_OGREPO = /usr/bin/python3 /opt/opengnsys/ogrepository/bin/* Cmnd_Alias PYTHON_OGREPO = /usr/bin/python3 /opt/opengnsys/ogrepository/bin/*
# Permitir al usuario opengnsys ejecutar estos comandos sin contraseña # Permitir al usuario opengnsys ejecutar estos comandos sin contraseña
opengnsys ALL=(root) NOPASSWD: MOUNT_RECOVERY, CHROOT_GRUB, LOOP_KPARTX, KILL_BT, PYTHON_OGREPO opengnsys ALL=(root) NOPASSWD: MOUNT_RECOVERY, CHROOT_GRUB, LOOP_KPARTX, OGBOOT, KILL_BT, PYTHON_OGREPO