Compare commits
5 Commits
main
...
7705731dd5
| Author | SHA1 | Date | |
|---|---|---|---|
|
7705731dd5
|
|||
|
e639c12c20
|
|||
|
c10077392e
|
|||
|
7dd7b57e1f
|
|||
|
b10ed69d59
|
1
.gitignore
vendored
1
.gitignore
vendored
@@ -1,2 +1,3 @@
|
||||
poetry.lock
|
||||
__pycache__
|
||||
docs/build
|
||||
|
||||
20
docs/Makefile
Normal file
20
docs/Makefile
Normal file
@@ -0,0 +1,20 @@
|
||||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
SOURCEDIR = source
|
||||
BUILDDIR = build
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
35
docs/make.bat
Normal file
35
docs/make.bat
Normal file
@@ -0,0 +1,35 @@
|
||||
@ECHO OFF
|
||||
|
||||
pushd %~dp0
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set SOURCEDIR=source
|
||||
set BUILDDIR=build
|
||||
|
||||
%SPHINXBUILD% >NUL 2>NUL
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.https://www.sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
goto end
|
||||
|
||||
:help
|
||||
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
|
||||
|
||||
:end
|
||||
popd
|
||||
36
docs/source/conf.py
Normal file
36
docs/source/conf.py
Normal file
@@ -0,0 +1,36 @@
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# For the full list of built-in configuration values, see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
|
||||
|
||||
project = 'unisync'
|
||||
copyright = '2026, Paul Retourné'
|
||||
author = 'Paul Retourné'
|
||||
release = '0.1.0'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
|
||||
|
||||
extensions = [
|
||||
'sphinx.ext.autodoc',
|
||||
'sphinx.ext.viewcode',
|
||||
'sphinx.ext.napoleon',
|
||||
'sphinx.ext.todo'
|
||||
]
|
||||
|
||||
templates_path = ['_templates']
|
||||
exclude_patterns = []
|
||||
|
||||
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
|
||||
|
||||
#html_theme = 'alabaster'
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_static_path = ['_static']
|
||||
|
||||
autodoc_docstring_signature = True
|
||||
24
docs/source/example.rst
Normal file
24
docs/source/example.rst
Normal file
@@ -0,0 +1,24 @@
|
||||
.. _example_how_it_works:
|
||||
|
||||
Example of how unisync works
|
||||
============================
|
||||
|
||||
Let's say you have the following structure::
|
||||
|
||||
$ tree .
|
||||
.
|
||||
├── big_file
|
||||
└── folder
|
||||
├── file
|
||||
└── other_file
|
||||
|
||||
If you only want to synchronise `folder` and its content on your laptop the following will be automatically generated::
|
||||
|
||||
$ tree .
|
||||
.
|
||||
├── big_file -> ../.data/big_file
|
||||
└── folder
|
||||
├── file
|
||||
└── other_file
|
||||
|
||||
`big_file` is now a symbolic link and by mounting the remote directory you can still seemlessly access `big_file` through the network.
|
||||
31
docs/source/index.rst
Normal file
31
docs/source/index.rst
Normal file
@@ -0,0 +1,31 @@
|
||||
.. unisync documentation master file, created by
|
||||
sphinx-quickstart on Sun Jan 4 15:02:58 2026.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Documentation for unisync
|
||||
=========================
|
||||
|
||||
Unisync is a data synchronising tool built around `unison`_ and expending on it.
|
||||
|
||||
Unisync tries to solve two problems that are often solved separately but never together :
|
||||
|
||||
* Keeping your data synchronised between multiple machines (through a central server), examples of this are rsync and of course unison.
|
||||
* Being able to access and edit files stored on your server without having to download them, the gui interface of nextcloud for example.
|
||||
* And of course I want to be able to do all of this without ever having to leave my terminal.
|
||||
|
||||
Unisync solves this problem by placing each file on your local machine but with only the selected files and folders being physically present on your drive,
|
||||
the others are replaced by symbolic links pointing to a directory that is mounted from your server.
|
||||
|
||||
See this
|
||||
:ref:`example_how_it_works`.
|
||||
|
||||
.. _unison: https://github.com/bcpierce00/unison
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents:
|
||||
|
||||
example
|
||||
modules
|
||||
|
||||
7
docs/source/modules.rst
Normal file
7
docs/source/modules.rst
Normal file
@@ -0,0 +1,7 @@
|
||||
unisync
|
||||
=======
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
unisync
|
||||
77
docs/source/unisync.rst
Normal file
77
docs/source/unisync.rst
Normal file
@@ -0,0 +1,77 @@
|
||||
unisync package
|
||||
===============
|
||||
|
||||
Submodules
|
||||
----------
|
||||
|
||||
unisync.argparser module
|
||||
------------------------
|
||||
|
||||
.. automodule:: unisync.argparser
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.config module
|
||||
---------------------
|
||||
|
||||
.. automodule:: unisync.config
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.defaults module
|
||||
-----------------------
|
||||
|
||||
.. automodule:: unisync.defaults
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.errors module
|
||||
---------------------
|
||||
|
||||
.. automodule:: unisync.errors
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.main module
|
||||
-------------------
|
||||
|
||||
.. automodule:: unisync.main
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.paths module
|
||||
--------------------
|
||||
|
||||
.. automodule:: unisync.paths
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.runners module
|
||||
----------------------
|
||||
|
||||
.. automodule:: unisync.runners
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
unisync.synchroniser module
|
||||
---------------------------
|
||||
|
||||
.. automodule:: unisync.synchroniser
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
|
||||
Module contents
|
||||
---------------
|
||||
|
||||
.. automodule:: unisync
|
||||
:members:
|
||||
:show-inheritance:
|
||||
:undoc-members:
|
||||
@@ -20,3 +20,10 @@ packages = [{include = "unisync", from = "src"}]
|
||||
[build-system]
|
||||
requires = ["poetry-core>=2.0.0,<3.0.0"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
|
||||
[dependency-groups]
|
||||
docs = [
|
||||
"sphinx (>=9.1.0,<10.0.0)",
|
||||
"sphinx-rtd-theme (>=3.0.2,<4.0.0)",
|
||||
]
|
||||
|
||||
|
||||
@@ -15,4 +15,4 @@ DEFAULT_SERVER_PORT: int = 22
|
||||
DEFAULT_ROOTS_LOCAL: str = str(Path("~/files").expanduser())
|
||||
# DEFAULT_ROOTS_REMOTE: str = ""
|
||||
|
||||
DEFAULT_MISC_CACHE_DIR_PATH: Path = Path("~/.unisync").expanduser()
|
||||
DEFAULT_MISC_CACHE_DIR_PATH: str = "~/.unisync"
|
||||
|
||||
@@ -21,11 +21,11 @@ def main():
|
||||
elif os.path.isfile(config_path):
|
||||
config = load_config(config_path)
|
||||
else:
|
||||
# TODO replace the next line with something to do if no config file is found
|
||||
# TODO: replace the next line with something to do if no config file is found
|
||||
config = load_config(config_path)
|
||||
pass
|
||||
|
||||
# TODO make the command line arguments work and override the config options
|
||||
# TODO: make the command line arguments work and override the config options
|
||||
|
||||
synchroniser = Synchroniser(
|
||||
config.roots.remote,
|
||||
|
||||
@@ -14,7 +14,7 @@ def unisync_sync(synchroniser:Synchroniser, paths_manager:PathsManager):
|
||||
synchroniser.sync_files(paths_manager.get_paths_to_sync())
|
||||
synchroniser.sync_links(paths_manager.get_paths_to_sync())
|
||||
|
||||
# TODO check the config options and do or don't do the following
|
||||
# TODO: check the config options and do or don't do the following
|
||||
synchroniser.update_links()
|
||||
#synchroniser.mount_remote_dir()
|
||||
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
# Copyright (C) 2025-2026 Paul Retourné
|
||||
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
"""Exports the Synchroniser class.
|
||||
|
||||
This class is used to perform all the actions that require a connection to
|
||||
the remote.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import os
|
||||
import sys
|
||||
@@ -14,9 +20,37 @@ from unisync.errors import RemoteMountedError, InvalidMountError
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class Synchroniser:
|
||||
"""Synchroniser used to synchronise with a server.
|
||||
|
||||
It is used to perform every action needing a connection to the remote.
|
||||
Create an ssh connection.
|
||||
Perform the various synchronisation steps (files, links).
|
||||
Update the links on the remote.
|
||||
Mount the remote directory.
|
||||
Close the ssh connection.
|
||||
|
||||
Attributes:
|
||||
remote: The directory to synchronise to on the remote.
|
||||
local: The directory to synchronise from locally.
|
||||
user: The user on the remote server.
|
||||
ip: The ip of the remote server.
|
||||
port: The ssh port on the remote.
|
||||
args_bool:
|
||||
A list of boolean arguments for unison.
|
||||
They will be passed directly to unison when calling it.
|
||||
For example : auto will be passed as -auto
|
||||
args_value:
|
||||
Same as args_bool but for key value arguments.
|
||||
Will be passed to unison as "-key value".
|
||||
ssh_settings:
|
||||
Settings to pass to the underlying ssh connection.
|
||||
Currently unused.
|
||||
"""
|
||||
|
||||
def __init__(self, remote:str, local:str, user:str, ip:str,
|
||||
port:int=22, args_bool:list=[], args_value:dict={}, ssh_settings:dict={}):
|
||||
"""Initialises an instance of Synchroniser.
|
||||
"""
|
||||
self.remote_dir:str = remote
|
||||
self.local:str = local
|
||||
self.args_bool:list[str] = args_bool
|
||||
@@ -27,14 +61,20 @@ class Synchroniser:
|
||||
self.remote_port:int = port
|
||||
|
||||
def create_ssh_master_connection(self, control_path:str="~/.ssh/control_%C", connection_timeout:int=60) -> int:
|
||||
"""
|
||||
Creates an ssh master connection so the user only has to authenticate once to the remote server.
|
||||
The subsequent connections will be made through this master connection which speeds up connecting.
|
||||
@control_path: Set the location of the ssh control socket
|
||||
@connection_timeout:
|
||||
"""Creates an ssh master connection.
|
||||
|
||||
It is used so the user only has to authenticate once to the remote server.
|
||||
The subsequent connections will be made through this master connection
|
||||
which speeds up connnection.
|
||||
The users only have to enter their password once per synchronisation.
|
||||
|
||||
Args:
|
||||
control_path: Set the location of the ssh control socket
|
||||
connection_timeout:
|
||||
Time given to the user to authenticate to the remote server.
|
||||
On slow connections one might want to increase this.
|
||||
Returns 0 on success.
|
||||
Returns:
|
||||
An error code (0 success, 1 TimeoutExpired, 2 KeyboardInterrupt).
|
||||
"""
|
||||
self.control_path = os.path.expanduser(control_path)
|
||||
command = [
|
||||
@@ -46,6 +86,7 @@ class Synchroniser:
|
||||
"-p", str(self.remote_port)
|
||||
]
|
||||
master_ssh = subprocess.Popen(command)
|
||||
# TODO: Raise an exception instead of changing the return value
|
||||
try:
|
||||
ret_code = master_ssh.wait(timeout=connection_timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
@@ -61,8 +102,10 @@ class Synchroniser:
|
||||
|
||||
|
||||
def close_ssh_master_connection(self) -> int:
|
||||
"""
|
||||
Close the ssh master connection.
|
||||
"""Closes the ssh master connection.
|
||||
|
||||
Returns:
|
||||
The return code of the ssh call.
|
||||
"""
|
||||
command = [
|
||||
"/usr/bin/ssh",
|
||||
@@ -75,8 +118,14 @@ class Synchroniser:
|
||||
return close.wait()
|
||||
|
||||
def sync_files(self, paths:list, force:bool=False) -> int:
|
||||
"""
|
||||
Synchronises the files.
|
||||
"""Synchronises the files.
|
||||
|
||||
Args:
|
||||
paths: List of paths to synchronise.
|
||||
force: Force the changes from remote to local.
|
||||
|
||||
Returns:
|
||||
The return code of sync.
|
||||
"""
|
||||
return self.sync(
|
||||
f"ssh://{self.remote_user}@{self.remote_ip}/{self.remote_dir}/.data",
|
||||
@@ -86,8 +135,13 @@ class Synchroniser:
|
||||
)
|
||||
|
||||
def sync_links(self, ignore:list) -> int:
|
||||
"""
|
||||
Synchronises the links, they must exist already.
|
||||
"""Synchronises the links, they must exist already.
|
||||
|
||||
Args:
|
||||
ignore: List of paths to ignore.
|
||||
|
||||
Returns:
|
||||
The return code of sync.
|
||||
"""
|
||||
return self.sync(
|
||||
f"ssh://{self.remote_user}@{self.remote_ip}/{self.remote_dir}/links",
|
||||
@@ -97,17 +151,20 @@ class Synchroniser:
|
||||
|
||||
def sync(self, remote_root:str, local_root:str,
|
||||
paths:list=[], ignore:list=[], force:bool=False) -> int:
|
||||
"""
|
||||
Perform the synchronisation by calling unison.
|
||||
@remote_root: The remote root, must be a full root usable by unison.
|
||||
@local_root: The local root, must be a full root usable by unison.
|
||||
@paths: List of paths to synchronise
|
||||
@ignore: List of paths to ignore
|
||||
"""Performs the synchronisation by calling unison.
|
||||
|
||||
Args:
|
||||
remote_root: The remote root, must be a full root usable by unison.
|
||||
local_root: The local root, must be a full root usable by unison.
|
||||
paths: List of paths to synchronise
|
||||
ignore: List of paths to ignore
|
||||
The paths and everything under them will be ignored.
|
||||
If you need to ignore some specific files use the arguments.
|
||||
@force: Force all changes from remote to local.
|
||||
force: Force all changes from remote to local.
|
||||
Used mostly when replacing a link by the file.
|
||||
Returns: the unison return code see section 6.11 of the documentation
|
||||
|
||||
Returns:
|
||||
the unison return code see section 6.11 of the documentation
|
||||
"""
|
||||
command = [ "/usr/bin/unison", "-root", remote_root, "-root", local_root ]
|
||||
for arg in self.args_bool:
|
||||
@@ -140,12 +197,13 @@ class Synchroniser:
|
||||
return ret_code
|
||||
|
||||
def update_links(self, background:bool=True):
|
||||
"""
|
||||
Update the links on the remote.
|
||||
"""Updates the links on the remote.
|
||||
|
||||
First calls cleanlinks to remove deadlinks and empty directories.
|
||||
Then calls lndir to create the new links.
|
||||
|
||||
Args:
|
||||
- background: controls if the update is done in the background or waited for
|
||||
background: controls if the update is done in the background or waited for.
|
||||
"""
|
||||
|
||||
link_update_script = (f"cd {self.remote_dir}/links && "
|
||||
@@ -173,13 +231,14 @@ class Synchroniser:
|
||||
print("Done")
|
||||
|
||||
def mount_remote_dir(self):
|
||||
"""
|
||||
Mount the remote directory to make the local links work.
|
||||
This is achieved using sshfs.
|
||||
Raise:
|
||||
- RemoteMountedError: The .data directory is already a mount point
|
||||
- InvalidMountError: .data is either not a directory or not empty
|
||||
- subprocess.CalledProcessError: An error occured with sshfs
|
||||
"""Mounts the remote directory to make the local links work.
|
||||
|
||||
This is achieved using sshfs which may fail.
|
||||
|
||||
Raises:
|
||||
RemoteMountedError: The .data directory is already a mount point.
|
||||
InvalidMountError: .data is either not a directory or not empty.
|
||||
subprocess.CalledProcessError: An error occured with sshfs.
|
||||
"""
|
||||
# Get the absolute path to the correct .data directory resolving symlinks
|
||||
path_to_mount:Path = Path(f"{self.local}/../.data").resolve()
|
||||
|
||||
Reference in New Issue
Block a user