oggit/gitlib/README-en.md

GitLib

The gitlib.py is a Python library also usable as a command-line program for testing purposes.

It contains functions for managing git, and the command-line interface allows executing them without needing to write a program that uses the library.

Requirements

Gitlib is designed to work within an existing OpenGnsys environment. It invokes some OpenGnsys commands internally and reads the parameters passed to the kernel in oglive.

Therefore, it will not work correctly outside of an oglive environment.

Installing Python dependencies

The code conversion to Python 3 currently requires the packages specified in requirements.txt.

The venv module (https://docs.python.org/3/library/venv.html) is used to install Python dependencies, creating an environment isolated from the system.

Note: Ubuntu 24.04 includes most of the required dependencies as packages, but there is no blkid package, so it must be installed using pip within a virtual environment.

Run the following commands:

sudo apt install -y python3 libarchive-dev libblkid-dev pkg-config libacl1-dev
python3 -m venv venvog
. venvog/bin/activate
python3 -m pip install --upgrade pip
pip3 install -r requirements.txt

Usage

Run with:

# . venvog/bin/activate
# ./gitlib.py

In command-line mode, help can be displayed with:

./gitlib.py --help

Note: Execute as the root user, as sudo clears the environment variable changes made by venv. This will likely result in a Python module not found error or program failure due to outdated dependencies.

Note: Commands starting with --test exist for internal testing. They are temporary and meant to test specific parts of the code. These may require specific conditions to work and will be removed upon completion of development.

Initialize a repository:

./gitlib.py --init-repo-from /dev/sda2 --repo linux

This initializes the 'linux' repository with the content of /mnt/sda2.

--repo specifies the name of one of the repositories configured during the git installation (see git installer).

The repository is uploaded to the ogrepository, obtained from the boot parameter passed to the kernel.

Clone a repository:

./gitlib.py --clone-repo-to /dev/sda2 --boot-device /dev/sda --repo linux

This clones a repository from the ogrepository. The target is a physical device that will be formatted with the necessary file system.

--boot-device specifies the boot device where the bootloader (GRUB or similar) will be installed.

--repo is the repository name contained in ogrepository.

Special Considerations for Windows

Cloning

• Windows must be completely shut down, not hibernated. See: https://learn.microsoft.com/en-us/troubleshoot/windows-client/setup-upgrade-and-drivers/disable-and-re-enable-hibernation
• Windows must be cleanly shut down using "Shut Down". Gitlib may fail to mount a disk from an improperly shut down system. If so, boot Windows again and shut it down properly.
• Disk encryption (BitLocker) cannot be used.

Restoration

Windows uses a structure called BCD (https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/bcd-system-store-settings-for-uefi?view=windows-11) to store boot configuration.

This structure can vary depending on the machine where it is deployed. For this reason, gitlib supports storing multiple versions of the BCD internally and selecting the one corresponding to a specific machine.

Documentation

Python documentation can be generated using utilities such as pdoc3 (other alternatives are also possible):

# Install pdoc3
pip install --user pdoc3

# Generate documentation
pdoc3 --force --html opengnsys_git_installer.py

Functionality

Metadata

Git cannot store data about extended attributes, sockets, or other special file types. Gitlib stores these in .opengnsys-metadata at the root of the repository.

The data is saved in jsonl files, a structure with one JSON object per line. This facilitates partial applications by applying only the necessary lines.

The following files are included:

• acls.jsonl: ACLs
• empty_directories.jsonl: Empty directories, as Git cannot store them
• filesystems.json: Information about file systems: types, sizes, UUIDs
• gitignores.jsonl: List of .gitignore files (renamed to avoid interfering with Git)
• metadata.json: General metadata about the repository
• special_files.jsonl: Special files like sockets
• xattrs.jsonl: Extended attributes
• renamed.jsonl: Files renamed to avoid interfering with Git
• unix_permissions.jsonl: UNIX permissions (not precisely stored by Git)
• ntfs_secaudit.txt: NTFS security data
• efi_data: Copy of the EFI (ESP) partition
• efi_data.(id): EFI partition copy corresponding to a specific machine
• efi_data.(name): EFI partition copy corresponding to a name specified by the administrator.