Compare commits

..

20 Commits

Author SHA1 Message Date
ec8030fc81 config : fix cache_dir_path value parsing error
Configparser's config.get returns a string and we want a Path. for the
moment convert it to Path directly.
2026-01-03 18:05:43 +01:00
f050dcc94f runners : fix sync runner not synchronising links
The wrong function was call in the sync runner (update_links instead of
sync_links) which mean the links were updated remotly but never
synchronised with the local.
Call sync_links instead.
We keep the call to update_links but set it to be in background.
2026-01-03 18:02:20 +01:00
f40a5c9276 Merge branch 'abstract_defaults'
Abstract the defaults into a seperate file
2026-01-03 17:20:18 +01:00
0e80ba0b0d config : use the defaults from defaults.py
Remove the defaults from the dataclasses as they are redundent with the
fallbacks of configparser.
Use the values in defaults.py as the fallbacks instead of hardcoded
values.
2026-01-03 17:18:19 +01:00
a223f04909 config : take cache_dir_path into account
cache_dir_path and all of the OtherConfig was ignored and the default
value was loaded, read its value from the config file instead.
2026-01-03 17:15:22 +01:00
e42ae71862 defaults : Create defaults.py
Creates the file defaults.py this is used to store the defaults and
easily include them into the config.
Changing defaults is thus possible without touching the code leaving
less room for errors.
2026-01-03 17:10:06 +01:00
58c7f7d1be Add main as a script with poetry 2026-01-03 16:39:00 +01:00
eefb21faff Mark local imports as such.
Prefix local imports with "unisync." so they are not mistaken with
external modules imports
2026-01-03 16:24:58 +01:00
941c467fc2 Bump copyright year and add missing file headers 2026-01-02 10:58:27 +01:00
4dcab777ca Merge branch 'dev'
Got to a state that seems stable enough to go into main
2026-01-02 10:45:00 +01:00
a169890351 main : adds subcommands, move to Path and improve
Multiple changes to the main file, after this unisync becomes kind of
usable.
Add subcommands : this uses the 2 previous commits to add the
subcommands to unisync it is now possible to sync, add and mount.
pathlib : move from PosixPath to Path
Remove unused imports
Rename base_namespace to cli_args
Add some comments and TODOs
2026-01-02 10:44:59 +01:00
b70070ba1a argparser : adds subcommands to the argparser
this adds subcommands to the argparser using subparsers, we also set a
default value for func depending on which of the subcommands is
selected.
Also change the formatting of the epilog so it is on two lines.
2026-01-02 10:44:58 +01:00
bd72d740e6 runners : Create runners file and basic runnners
This adds runners.py, it contains a set of functions that peform all the
various task that unisync can do (sync, add and mount for now).
They are simple function that put together all the rest.
2026-01-02 10:44:57 +01:00
e43c16adb3 paths : fixes write_new_paths writing of the file
I was writing the file using 'w' instead of 'a' so the old paths were
deleted use 'a'.
2026-01-02 10:44:56 +01:00
10200fceb9 config : fallback to port 22 instead of None
The configparser fallback option for port was None set it to use 22
instead as None doesn't make sense
2026-01-02 10:44:55 +01:00
27924013d9 Bug fixes and small improvements
Fix :
- paths : true instead of True
- paths : Path has no len convert to str first to get the number of
  characters

Improvements :
- Replace all PosixPath by Path
2026-01-01 17:24:46 +01:00
48179034a7 Adds prerequisites to README 2025-12-31 00:03:59 +01:00
f9001ecb9d Adds usage of the paths manager 2025-12-31 00:03:17 +01:00
86a6c8acce Adds error handling for the paths 2025-12-30 17:56:03 +01:00
4f6f48247d Add classes for error handling 2025-12-30 17:54:47 +01:00
10 changed files with 161 additions and 50 deletions

View File

@@ -1,6 +1,19 @@
Unisync is a data synchronisation tool written in python and based on [unison](https://github.com/bcpierce00/unison). Unisync is a data synchronisation tool written in python and based on [unison](https://github.com/bcpierce00/unison).
I couldn't find a tool to fulfill the requirements I had for a synchronisation tool so I am creating my own as a wrapper around unison. I couldn't find a tool to fulfill the requirements I had for a synchronisation tool so I am creating my own as a wrapper around unison.
# Prerequisite
You need to have the following tools installed.
Locally :
- unison
- sshfs
- nnn
Remotely :
- unison
- cleanlinks and lndir (Should be in `xutils-dev` when using apt)
# Goal # Goal
Unisync purpose is to keep personal data synchronised between multiple machines without needing to have all the data present an all the machines at the same time. For example you might not need to have your movies on your laptop but still want them on your desktop at home or you might want to keep your old pictures only on a server. Unisync purpose is to keep personal data synchronised between multiple machines without needing to have all the data present an all the machines at the same time. For example you might not need to have your movies on your laptop but still want them on your desktop at home or you might want to keep your old pictures only on a server.

View File

@@ -10,6 +10,9 @@ requires-python = ">=3.13"
dependencies = [ dependencies = [
] ]
[project.scripts]
unisync = "unisync.main:main"
[tool.poetry] [tool.poetry]
packages = [{include = "unisync", from = "src"}] packages = [{include = "unisync", from = "src"}]

View File

@@ -1,22 +1,36 @@
# Copyright (C) 2025 Paul Retourné # Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
import argparse import argparse
def create_argparser() -> argparse.ArgumentParser: def create_argparser(sync_function, add_function, mount_function) -> argparse.ArgumentParser:
"""
Creates an argument parser to parse the command line arguments.
We use subparsers and set a default function for each to perform the correct action.
"""
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
prog='unisync', prog='unisync',
description='File synchronisation application', description='File synchronisation application',
epilog=""" epilog="Copyright © 2025-2026 Paul Retourné.\n"
Copyright © 2025 Paul Retourné. "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.",
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.""" formatter_class=argparse.RawDescriptionHelpFormatter
) )
parser.add_argument("local", nargs="?") parser.add_argument("local", nargs="?")
parser.add_argument("remote", nargs="?") parser.add_argument("remote", nargs="?")
parser.set_defaults(func=sync_function)
remote_addr_group = parser.add_mutually_exclusive_group() remote_addr_group = parser.add_mutually_exclusive_group()
remote_addr_group.add_argument("--ip") remote_addr_group.add_argument("--ip")
remote_addr_group.add_argument("--hostname") remote_addr_group.add_argument("--hostname")
parser.add_argument("--config", help="Path to the configuration file", metavar="path_to_config") parser.add_argument("--config", help="Path to the configuration file", metavar="path_to_config")
subparsers = parser.add_subparsers(help='Actions other than synchronisation')
parser_add = subparsers.add_parser('add', help='Add files to be synchronised.')
parser_add.set_defaults(func=add_function)
parser_mount = subparsers.add_parser('mount', help='Mount the remote.')
parser_mount.set_defaults(func=mount_function)
return parser return parser

View File

@@ -1,22 +1,24 @@
# Copyright (C) 2025 Paul Retourné # Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
from configparser import UNNAMED_SECTION from configparser import UNNAMED_SECTION
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path, PosixPath from pathlib import Path
import ipaddress import ipaddress
import configparser import configparser
from unisync.defaults import *
@dataclass @dataclass
class ServerConfig: class ServerConfig:
""" """
Dataclass keeping the config for connecting to the server Dataclass keeping the config for connecting to the server
""" """
user: str user: str
sshargs: list[str] | None = field(default_factory=list) sshargs: str
hostname: str = "" hostname: str
ip: str = "" ip: str
port: int = 22 port: int
def __post_init__(self): def __post_init__(self):
""" """
@@ -44,15 +46,15 @@ class UnisonConfig:
""" """
Dataclass keeping unison specific configurations Dataclass keeping unison specific configurations
""" """
bools: list = field(default_factory=list) bools: list
values: dict = field(default_factory=dict) values: dict
@dataclass @dataclass
class OtherConfig: class OtherConfig:
""" """
Dataclass keeping miscellanous configuration options Dataclass keeping miscellanous configuration options
""" """
cache_dir_path: PosixPath = Path("~/.unisync").expanduser() cache_dir_path: Path
@dataclass @dataclass
class Config: class Config:
@@ -62,7 +64,7 @@ class Config:
server: ServerConfig server: ServerConfig
roots: RootsConfig roots: RootsConfig
unison: UnisonConfig unison: UnisonConfig
other: OtherConfig = field(default_factory=OtherConfig) other: OtherConfig
def load_config(config_path:str) -> Config: def load_config(config_path:str) -> Config:
@@ -79,18 +81,22 @@ def load_config(config_path:str) -> Config:
# Check if sections are provided # Check if sections are provided
server_section = "Server" if "Server" in config.sections() else UNNAMED_SECTION server_section = "Server" if "Server" in config.sections() else UNNAMED_SECTION
roots_section = "Roots" if "Roots" in config.sections() else UNNAMED_SECTION roots_section = "Roots" if "Roots" in config.sections() else UNNAMED_SECTION
other_section = "Other" if "Other" in config.sections() else UNNAMED_SECTION
server_config = ServerConfig( server_config = ServerConfig(
config.get(server_section, "user"), config.get(server_section, "user"),
config.get(server_section, "sshargs", fallback=None), config.get(server_section, "sshargs", fallback=DEFAULT_SERVER_SSHARGS),
config.get(server_section, "hostname", fallback=None), config.get(server_section, "hostname", fallback=DEFAULT_SERVER_HOSTNAME),
config.get(server_section, "ip", fallback=None), config.get(server_section, "ip", fallback=DEFAULT_SERVER_IP),
config.getint(server_section, "port", fallback=None) config.getint(server_section, "port", fallback=DEFAULT_SERVER_PORT)
) )
roots_config = RootsConfig( roots_config = RootsConfig(
config.get(roots_section, "local"), config.get(roots_section, "local", fallback=DEFAULT_ROOTS_LOCAL),
config.get(roots_section, "remote") config.get(roots_section, "remote")
) )
other_config = OtherConfig(
Path(config.get(other_section, "cache_dir_path", fallback=DEFAULT_MISC_CACHE_DIR_PATH)).expanduser()
)
args_bool = list() args_bool = list()
args_val = dict() args_val = dict()
@@ -104,4 +110,4 @@ def load_config(config_path:str) -> Config:
args_val[key] = val args_val[key] = val
unison_config = UnisonConfig(args_bool, args_val) unison_config = UnisonConfig(args_bool, args_val)
return Config(server_config, roots_config, unison_config) return Config(server_config, roots_config, unison_config, other_config)

18
src/unisync/defaults.py Normal file
View File

@@ -0,0 +1,18 @@
# copyright (c) 2026 paul retourné
# spdx-license-identifier: gpl-3.0-or-later
from pathlib import Path
# Commented out values are part of the config but are required so there is no defaults.
# This allows this file to be a list of all the config options.
# DEFAULT_SERVER_USER: str = ""
DEFAULT_SERVER_SSHARGS: str = ""
DEFAULT_SERVER_HOSTNAME: str = ""
DEFAULT_SERVER_IP: str = ""
DEFAULT_SERVER_PORT: int = 22
DEFAULT_ROOTS_LOCAL: str = str(Path("~/files").expanduser())
# DEFAULT_ROOTS_REMOTE: str = ""
DEFAULT_MISC_CACHE_DIR_PATH: Path = Path("~/.unisync").expanduser()

8
src/unisync/errors.py Normal file
View File

@@ -0,0 +1,8 @@
# Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later
class RemoteMountedError(BaseException):
pass
class InvalidMountError(BaseException):
pass

View File

@@ -1,24 +1,32 @@
# Copyright (C) 2025 Paul Retourné # Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
import os import os
from argparser import create_argparser from pathlib import Path
from config import RootsConfig, ServerConfig, Config, load_config
from synchroniser import Synchroniser from unisync.argparser import create_argparser
from unisync.runners import unisync_sync, unisync_add, unisync_mount
from unisync.config import load_config
from unisync.synchroniser import Synchroniser
from unisync.paths import *
def main(): def main():
parser = create_argparser() parser = create_argparser(unisync_sync, unisync_add, unisync_mount)
base_namespace = parser.parse_args() cli_args = parser.parse_args()
config_path = os.path.expanduser("~/.config/unisync/config.ini") config_path = os.path.expanduser("~/.config/unisync/config.ini")
if base_namespace.config != None and os.path.isfile(base_namespace.config): # Check if --config is set
config = load_config(base_namespace.config) if cli_args.config != None and os.path.isfile(cli_args.config):
config = load_config(cli_args.config)
elif os.path.isfile(config_path): elif os.path.isfile(config_path):
config = load_config(config_path) config = load_config(config_path)
else: else:
# TODO make the command line arguments work and override the config options # TODO replace the next line with something to do if no config file is found
config = load_config(config_path)
pass pass
# TODO make the command line arguments work and override the config options
synchroniser = Synchroniser( synchroniser = Synchroniser(
config.roots.remote, config.roots.remote,
config.roots.local, config.roots.local,
@@ -29,15 +37,9 @@ def main():
config.unison.values config.unison.values
) )
if synchroniser.create_ssh_master_connection() != 0: paths_manager = PathsManager(Path(config.roots.local), config.other.cache_dir_path)
print("Connection failed quitting")
return 1
print("Connected to the remote.")
synchroniser.sync_files(["salut"])
synchroniser.close_ssh_master_connection()
cli_args.func(synchroniser, paths_manager)
if __name__ == "__main__": if __name__ == "__main__":

View File

@@ -1,15 +1,16 @@
# Copyright (C) 2025 Paul Retourné # Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
import os.path import os.path
import subprocess import subprocess
import sys
from pathlib import Path, PosixPath from pathlib import Path
class PathsManager: class PathsManager:
def __init__(self, local_dir:PosixPath, cache_dir:PosixPath): def __init__(self, local_dir:Path, cache_dir:Path):
""" """
Creates a PathsManager with the necessary data Creates a PathsManager with the necessary data
Args: Args:
@@ -24,7 +25,7 @@ class PathsManager:
raise ValueError("Invalid cache directory") raise ValueError("Invalid cache directory")
self.cache_dir = cache_dir self.cache_dir = cache_dir
self.paths_file:PosixPath = self.cache_dir / "paths" self.paths_file:Path = self.cache_dir / "paths"
if not self.paths_file.is_file(): if not self.paths_file.is_file():
raise ValueError("The paths file does not exist") raise ValueError("The paths file does not exist")
@@ -48,7 +49,7 @@ class PathsManager:
"-p", "-", "-p", "-",
self.local_dir self.local_dir
] ]
nnn_process = subprocess.Popen(command, stdout=subprocess.PIPE) nnn_process:subprocess.Popen = subprocess.Popen(command, stdout=subprocess.PIPE)
try: try:
ret_code = nnn_process.wait(timeout=choice_timeout) ret_code = nnn_process.wait(timeout=choice_timeout)
except subprocess.TimeoutExpired as e: except subprocess.TimeoutExpired as e:
@@ -57,18 +58,18 @@ class PathsManager:
if ret_code != 0: if ret_code != 0:
print("File selection failed", file=sys.stderr) print("File selection failed", file=sys.stderr)
raise subprocess.CalledProcessError("File selection failed") raise subprocess.CalledProcessError(1, "File selection failed")
paths_list:list[str] = [] paths_list:list[str] = []
while (next_path := nnn_process.stdout.readline()) != b'': while (next_path := nnn_process.stdout.readline()) != b'':
next_path = next_path.decode().strip() next_path = next_path.decode().strip()
# Make the path relative to the top directory # Make the path relative to the top directory
next_path = next_path[len(self.local_dir):].lstrip("/") next_path = next_path[len(str(self.local_dir)):].lstrip("/")
paths_list.append(next_path) paths_list.append(next_path)
return paths_list return paths_list
def add_files_to_sync(self): def add_files_to_sync(self):
while true: while True:
try: try:
paths = self.user_select_files() paths = self.user_select_files()
break break
@@ -105,7 +106,7 @@ class PathsManager:
if not is_contained and new_path not in paths_to_add: if not is_contained and new_path not in paths_to_add:
paths_to_add.append(new_path) paths_to_add.append(new_path)
with self.paths_file.open("w") as f: with self.paths_file.open("a") as f:
for p in paths_to_add: for p in paths_to_add:
f.write(p + "\n") f.write(p + "\n")

36
src/unisync/runners.py Normal file
View File

@@ -0,0 +1,36 @@
# Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later
from unisync.synchroniser import Synchroniser
from unisync.paths import PathsManager
def unisync_sync(synchroniser:Synchroniser, paths_manager:PathsManager):
if synchroniser.create_ssh_master_connection() != 0:
print("Connection failed quitting")
return 1
print("Connected to the remote.")
synchroniser.sync_files(paths_manager.get_paths_to_sync())
synchroniser.sync_links(paths_manager.get_paths_to_sync())
# TODO check the config options and do or don't do the following
synchroniser.update_links()
#synchroniser.mount_remote_dir()
synchroniser.close_ssh_master_connection()
def unisync_add(synchroniser:Synchroniser, paths_manager:PathsManager):
if synchroniser.create_ssh_master_connection() != 0:
print("Connection failed quitting")
return 1
print("Connected to the remote.")
paths_manager.add_files_to_sync()
synchroniser.close_ssh_master_connection()
def unisync_mount(synchroniser:Synchroniser, paths_manager:PathsManager):
synchroniser.mount_remote_dir()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2025 Paul Retourné # Copyright (C) 2025-2026 Paul Retourné
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
import subprocess import subprocess
@@ -9,6 +9,8 @@ import logging
from pathlib import Path from pathlib import Path
from unisync.errors import RemoteMountedError, InvalidMountError
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class Synchroniser: class Synchroniser:
@@ -175,16 +177,24 @@ class Synchroniser:
Mount the remote directory to make the local links work. Mount the remote directory to make the local links work.
This is achieved using sshfs. This is achieved using sshfs.
Raise: Raise:
- RemoteMountedError: The .data directory is already a mount point
- InvalidMountError: .data is either not a directory or not empty
- subprocess.CalledProcessError: An error occured with sshfs - subprocess.CalledProcessError: An error occured with sshfs
""" """
# Get the absolute path to the correct .data directory resolving symlinks
path_to_mount:Path = Path(f"{self.local}/../.data").resolve()
if path_to_mount.is_mount():
raise RemoteMountedError
# Check if it is an empty directory
if not path_to_mount.is_dir() or any(path_to_mount.iterdir()):
raise InvalidMountError
command = [ command = [
"/usr/bin/sshfs", "/usr/bin/sshfs",
"-o", "ControlPath={self.control_path}", "-o", "ControlPath={self.control_path}",
"-o", "ServerAliveInterval=15", "-o", "ServerAliveInterval=15",
"-p", str(self.remote_port), "-p", str(self.remote_port),
f"{self.remote_user}@{self.remote_ip}:{self.remote_dir}/.data", f"{self.remote_user}@{self.remote_ip}:{self.remote_dir}/.data",
# Get the absolute path to the correct .data directory resolving symlinks str(path_to_mount)
str(Path(f"{self.local}/../.data").resolve())
] ]
completed_process = subprocess.run(command) completed_process = subprocess.run(command)
completed_process.check_returncode() completed_process.check_returncode()