Skip to content

epymorph.cache

epymorph's file caching utilities.

CACHE_PATH module-attribute 

CACHE_PATH = _cache_path()

The root directory for epymorph's cached files.

FileTree module-attribute 

FileTree = Directory | File

Nodes in a file tree are either directories or files.

FileError

Bases: Exception

Error during a file operation.

FileMissingError

Bases: FileError

Error loading a file, as it does not exist.

FileWriteError

Bases: FileError

Error writing a file.

FileReadError

Bases: FileError

Error loading a file.

FileVersionError

Bases: FileError

Error loading a file due to unmet version requirements.

CacheMissError

Bases: FileError

Raised on a cache-miss (for any reason) during a load-from-cache operation.

CacheWarning

Bases: Warning

Warning issued when we are unable to interact with the file cache but in a situation where program execution can continue, even if less optimally. For example: if we successfully load data from an external source but are unable to cache it for later, this is a warning because we assume the data is valid and that it could always be loaded again from the same source at a later time. The warning is issued to give the user the opportunity to fix it for next time.

Directory

Bases: NamedTuple

A directory.

name instance-attribute 

name: str

The directory name.

size instance-attribute 

size: int

The combined size of all of this directory's children.

children instance-attribute 

children: Sequence[FileTree]

The directory's children, which may be files or nested directories.

File

Bases: NamedTuple

A file.

name instance-attribute 

name: str

The file name.

size instance-attribute 

size: int

The file size.

module_cache_path 

module_cache_path(name: str) -> Path

Return the cache directory to use for the given module.

When epymorph modules need to store files in the cache, they should use a subdirectory within the application's cache path. The path should correspond to the module's path within epymorph. e.g.: module epymorph.adrio.acs5 will store files in $CACHE_PATH/adrio/acs5.

Parameters:

  • name (str) –

    The name of the module, typically given as __name__.

Returns:

  • Path –

    The cache directory to use, as a relative path since the cache functions require that.

Examples:

1
2
3
>>> # if we're in module `epymorph.adrio.acs5`...
>>> module_cache_path(__name__)
PosixPath('adrio/acs5')

save_file 

save_file(
    to_path: str | PathLike[str], file: BytesIO
) -> None

Save a single file.

Parameters:

  • to_path (str | PathLike[str]) –

    An absolute or relative path to store the file. Relative paths will be resolved against the current working directory. Folders in the path which do not exist will be created automatically.

  • file (BytesIO) –

    The file to store.

load_file 

load_file(from_path: str | PathLike[str]) -> BytesIO

Load a single file.

Parameters:

Returns:

  • BytesIO –

    The bytes of the file.

Raises:

save_bundle 

save_bundle(
    to_path: str | PathLike[str],
    version: int,
    files: dict[str, BytesIO],
) -> None

Save a bundle of files in our tar format with an associated version number.

Parameters:

  • to_path (str | PathLike[str]) –

    An absolute or relative path to store the file. Relative paths will be resolved against the current working directory. Folders in the path which do not exist will be created automatically.

  • version (int) –

    The version number for the archive.

  • files (dict[str, BytesIO]) –

    The files to store, with a file name for each (its name within the archive).

load_bundle 

load_bundle(
    from_path: str | PathLike[str],
    version_at_least: int = -1,
) -> dict[str, BytesIO]

Load a bundle of files in our tar format, optionally enforcing a minimum version.

Parameters:

  • from_path (str | PathLike[str]) –

    The path of the bundle.

  • version_at_least (int, default: -1 ) –

    The minimally accepted version number to load. -1 to accept any version.

Returns:

  • dict[str, BytesIO] –

    A dictionary of the contained files, mapping the internal file names to the bytes of the file.

Raises:

check_file_in_cache 

check_file_in_cache(cache_path: Path) -> bool

Check if a file is currently in the cache.

Returns:

  • bool –

    True if so.

save_file_to_cache 

save_file_to_cache(
    to_path: str | PathLike[str], file: BytesIO
) -> None

Save a single file to the cache (overwriting the existing file, if any).

This is a low-level building block.

Parameters:

  • to_path (str | PathLike[str]) –

    A path to store the file, relative to the cache folder.

  • file (BytesIO) –

    The file bytes to store.

Raises:

load_file_from_cache 

load_file_from_cache(
    from_path: str | PathLike[str],
) -> BytesIO

Load a single file from the cache.

This is a low-level building block.

Parameters:

  • from_path (str | PathLike[str]) –

    The path to the file, relative to the cache folder.

Returns:

load_or_fetch 

load_or_fetch(
    cache_path: Path, fetch: Callable[[], BytesIO]
) -> BytesIO

Attempt to load a file from the cache.

If the file isn't already in the cache, the provided fetch method is used to load the file and then it is cached for next time. Any exceptions raised by fetch will not be caught in this method. If the file was successfully loaded but failed to save to the cache, a warning is raised.

This is a higher-level but still fairly generic building block.

Parameters:

  • cache_path (Path) –

    The path to the file, relative to the cache folder.

  • fetch (Callable[[], BytesIO]) –

    The function to load the file if it's not in the cache.

Returns:

load_or_fetch_url 

load_or_fetch_url(url: str, cache_path: Path) -> BytesIO

Attempt to load a file from the cache.

If the file isn't already in the cache, the provided url is used to load the file and then it is cached for next time. Any exceptions raised by fetch will not be caught in this method. If the file was successfully loaded but failed to save to the cache, a warning is raised. URLs must use the http or https protocol.

This is a higher-level but still fairly generic building block.

Parameters:

  • url (str) –

    The url to locate the file if it's not in the cache.

  • cache_path (Path) –

    The path to the file, relative to the cache folder.

Returns:

save_bundle_to_cache 

save_bundle_to_cache(
    to_path: str | PathLike[str],
    version: int,
    files: dict[str, BytesIO],
) -> None

Save a tar bundle of files to the cache (overwriting the existing file, if any).

The tar includes the sha256 checksums of every content file, and a version file indicating which application version was responsible for writing the file (thus allowing the application to decide if a cached file is still valid when reading it).

Parameters:

  • to_path (str | PathLike[str]) –

    The path to store the bundle, relative to the cache folder.

  • version (int) –

    The version number for the archive.

  • files (dict[str, BytesIO]) –

    The files to store, with a file name for each (its name within the archive).

load_bundle_from_cache 

load_bundle_from_cache(
    from_path: str | PathLike[str],
    version_at_least: int = -1,
) -> dict[str, BytesIO]

Load a tar bundle of files from the cache.

Parameters:

  • from_path (str | PathLike[str]) –

    The path of the bundle, relative to the cache folder.

  • version_at_least (int, default: -1 ) –

    The minimally accepted version number to load. -1 to accept any version.

Raises:

format_file_size 

format_file_size(size: int) -> str

Format a file size given in bytes in 1024-based-unit representation.

Parameters:

  • size (int) –

    The size in bytes.

Returns:

  • str –

    The file size in human-readable format.

cache_inventory 

cache_inventory() -> Directory

List the contents of epymorph's cache as a FileTree.

cache_remove_confirmation 

cache_remove_confirmation(
    path: str | PathLike[str],
) -> tuple[Path, Callable[[], None]]

Create a function which removes a directory or file from the cache.

Also returns the resolved path to the thing that will be removed; this allows the application to confirm the removal.

Parameters:

  • path (str | PathLike[str]) –

    The path to the file, relative to the cache folder.

Returns:

  • tuple[Path, Callable[[], None]] –

    A tuple of the absolute file path and a function to remove it.

cache_remove 

cache_remove(path: str | PathLike[str]) -> None

Remove a directory or file from the cache.

Parameters: