epymorph.cache

epymorph's file caching utilities.

CACHE_PATH module-attribute

CACHE_PATH = _cache_path()

FileTree module-attribute

FileTree = Directory | File

Nodes in a file tree are either directories or files.

FileError

Bases: Exception

Error during a file operation.

FileMissingError

Bases: FileError

Error loading a file, as it does not exist.

FileWriteError

Bases: FileError

Error writing a file.

FileReadError

Bases: FileError

Error loading a file.

FileVersionError

Bases: FileError

Error loading a file due to unmet version requirements.

CacheMissError

Bases: FileError

Raised on a cache-miss (for any reason) during a load-from-cache operation.

CacheWarning

Bases: Warning

Warning issued when we are unable to interact with the file cache but in a situation where program execution can continue, even if less optimally. For example: if we successfully load data from an external source but are unable to cache it for later, this is a warning because we assume the data is valid and that it could always be loaded again from the same source at a later time. The warning is issued to give the user the opportunity to fix it for next time.

Directory

Bases: NamedTuple

A directory.

name instance-attribute

name: str

The directory name.

size instance-attribute

size: int

The combined size of all of this directory's children.

children instance-attribute

children: Sequence[FileTree]

The directory's children, which may be files or nested directories.

File

Bases: NamedTuple

A file.

name instance-attribute

name: str

The file name.

size instance-attribute

size: int

The file size.

module_cache_path

module_cache_path(name: str) -> Path

When epymorph modules need to store files in the cache, they should use a subdirectory tree within the application's cache path. This tree should correspond to the module's path within epymorph. e.g.: module epymorph.adrio.acs5 will store files at $CACHE_PATH/adrio/acs5. (The returned value is a relative path since the cache functions require that.)

Usage example:

_TIGER_CACHE_PATH = module_cache_path(__name__)

save_file

save_file(
    to_path: str | PathLike[str], file: BytesIO
) -> None

Save a single file. to_path can be absolute or relative; relative paths will be resolved against the current working directory. Folders in the path which do not exist will be created automatically.

load_file

load_file(from_path: str | PathLike[str]) -> BytesIO

Load a single file. An Exception is raised if the file cannot be loaded for any reason. On success, returns the bytes of the file.

save_bundle

save_bundle(
    to_path: str | PathLike[str],
    version: int,
    files: dict[str, BytesIO],
) -> None

Save a bundle of files in our tar format with an associated version number. to_path can be absolute or relative; relative paths will be resolved against the current working directory. Folders in the path which do not exist will be created automatically.

load_bundle

load_bundle(
    from_path: str | PathLike[str],
    version_at_least: int = -1,
) -> dict[str, BytesIO]

Load a bundle of files in our tar format, optionally enforcing a minimum version. An Exception is raised if the file cannot be loaded for any reason, or if its version is incorrect. On success, returns a dictionary of the contained files, mapping the file name to the bytes of the file.

check_file_in_cache

check_file_in_cache(cache_path: Path) -> bool

Returns True if a file is currently in the cache.

save_file_to_cache

save_file_to_cache(
    to_path: str | PathLike[str], file: BytesIO
) -> None

Save a single file to the cache (overwriting the existing file, if any). This is a low-level building block.

load_file_from_cache

load_file_from_cache(
    from_path: str | PathLike[str],
) -> BytesIO

Load a single file from the cache. This is a low-level building block.

load_or_fetch

load_or_fetch(
    cache_path: Path, fetch: Callable[[], BytesIO]
) -> BytesIO

Attempts to load a file from the cache. If it doesn't exist, uses the provided fetch method to load the file, then attempts to save the file to the cache for next time. (This is a higher-level but still generic building block.) Any exceptions raised by fetch will not be caught in this method.

load_or_fetch_url

load_or_fetch_url(url: str, cache_path: Path) -> BytesIO

Attempts to load a file from the cache. If it doesn't exist, fetches the file contents from the given URL, then attempts to save the file to the cache for next time.

save_bundle_to_cache

save_bundle_to_cache(
    to_path: str | PathLike[str],
    version: int,
    files: dict[str, BytesIO],
) -> None

Save a tar bundle of files to the cache (overwriting the existing file, if any). The tar includes the sha256 checksums of every content file, and a version file indicating which application version was responsible for writing the file (thus allowing the application to decide if a cached file is still valid when reading it).

load_bundle_from_cache

load_bundle_from_cache(
    from_path: str | PathLike[str],
    version_at_least: int = -1,
) -> dict[str, BytesIO]

Load a tar bundle of files from the cache. from_path must be a relative path. version_at_least optionally specifies a version number that must be met or beat by the cached file in order for the file to be considered valid. If the cached file was written against a version less than this, it will be considered a cache miss (raises CacheMiss).

format_file_size

format_file_size(size: int) -> str

Format a file size given in bytes in 1024-based-unit representation.

cache_inventory

cache_inventory() -> Directory

Lists the contents of epymorph's cache as a FileTree.

cache_remove_confirmation

cache_remove_confirmation(
    path: str | PathLike[str],
) -> tuple[Path, Callable[[], None]]

Creates a function which removes a directory or file from the cache. Also returns the resolved path to the thing that will be removed; this allows the application to confirm the removal.

cache_remove

cache_remove(path: str | PathLike[str]) -> None

Removes a directory or file from the cache.