datazen package

datazen package#

Subpackages#

Submodules#

datazen.app module#

datazen - This package’s command-line entry-point application.

datazen.app.add_app_args(parser: ArgumentParser) → None[source]#: Add application-specific arguments to the command-line parser.

datazen.app.entry(args: Namespace) → int[source]#: Execute the requested task.

datazen.compile module#

datazen - An interface for turning a dictionary into various serialized forms.

datazen.compile.get_compile_output(entry: Dict[str, Any], default_type: str = 'yaml') → Tuple[str, str][source]#: Determine the output path and type of a compile target, from the target’s data.

datazen.compile.str_compile(configs: ~typing.Dict[str, ~typing.Any], data_type: str, logger: ~logging.Logger = <Logger datazen.compile (WARNING)>, **kwargs) → str[source]#: Serialize dictionary data into the String-form of a specific, serializeable type.

datazen.compile.write_dir(directory: Path | str | None, data: Dict[str, Any], out_type: str = 'json', **kwargs) → None[source]#: Write dictionary data to the file-system.

datazen.configs module#

datazen - Top-level APIs for loading and interacting with configuration data.

datazen.configs.load(directories: Iterable[Path | str | None], variable_data: Dict[str, Any] = None, loads: LoadedFiles = (None, None)) → LoadResult[source]#: Load configuration data from a list of directories.

datazen.entry module#

This package’s command-line entry-point (boilerplate).

datazen.entry.main(argv: List[str] = None) → int[source]#: Program entry-point.

datazen.fingerprinting module#

datazen - Reading and writing a known header signature to output files.

datazen.fingerprinting.apply_barriers(comment_lines: List[str], char: str) → None[source]#: Optionally add decoration to the comment lines by adding a barrier at the beginning and end.

datazen.fingerprinting.build_fingerprint(file_data: str, file_ext: str, char: str = '=', dynamic: bool = True, newline: str = '\n') → str[source]#: Build a String that should be prepended to the final file output for user awareness and automated interpretation.

datazen.fingerprinting.get_comment_data(file_data: str, dynamic: bool = True) → List[Tuple[str, str]][source]#: Get tuples (key-value pairs) of data to write into the file fingerprint.

datazen.fingerprinting.resolve_encapsulation(comment_lines: List[str], file_ext: str, newline: str = '\n') → str[source]#: Turn the requested line data into something that can be embedded in the target file type.

datazen.load module#

datazen - APIs for loading data from directory trees.

class datazen.load.LoadedFiles(files: List[str] | None = None, file_data: Dict[str, Dict[str, Any]] | None = None)[source]#

Bases: NamedTuple

A collection of data for files loaded at runtime (or, a continuation of this information loaded from a cache).

file_data: Dict[str, Dict[str, Any]] | None#: Alias for field number 1

files: List[str] | None#: Alias for field number 0

datazen.load.data_added(key: Any, value: Any, data: Dict[str, Any] = None) → Iterator[Dict[str, Any]][source]#: Inject a key-value pair into a dictionary, in a context.

datazen.load.load_dir(path: ~pathlib.Path | str | None, existing_data: ~typing.Dict[str, ~typing.Any], variables: ~typing.Dict[str, ~typing.Any] = None, loads: ~datazen.load.LoadedFiles = (None, None), expect_overwrite: bool = False, are_templates: bool = True, logger: ~logging.Logger = <Logger datazen.load (WARNING)>) → LoadResult[source]#: Load a directory tree into a dictionary, optionally meld.

datazen.load.load_dir_only(path: ~pathlib.Path | str | None, expect_overwrite: bool = False, are_templates: bool = True, logger: ~logging.Logger = <Logger datazen.load (WARNING)>) → LoadResult[source]#: A convenient wrapper for loading just directory data from a path without worrying about melding data, resolving variables, enforcing schemas, etc.

datazen.load.load_files(file_paths: List[Path | str | None], root: str, meld_data: Tuple[Dict[str, Any], Dict[str, Any], bool], hashes: Dict[str, Dict[str, Any]] = None, expect_overwrite: bool = False, are_templates: bool = True) → Tuple[List[str], int][source]#: Load files into a dictionary and return a list of the files that are new or had hash mismatches.

datazen.load.meld_and_resolve(path: Path | str | None, existing_data: Dict[str, Any], variables: Dict[str, Any], globals_added: bool = False, expect_overwrite: bool = False, is_template: bool = True, **kwargs) → bool[source]#: Meld dictionary data from a file into an existing dictionary, assume existing data is a template and attempt to resolve variables.

datazen.parsing module#

datazen - APIs for loading raw data from files.

datazen.parsing.dedup_dict_lists(data: Dict[Any, Any]) → Dict[Any, Any][source]#: Finds list elements in a dictionary and removes duplicate entries, mutates the original list.

datazen.parsing.load(path: ~pathlib.Path | str | None, variables: ~typing.Dict[str, ~typing.Any], dict_to_update: ~typing.Dict[str, ~typing.Any], expect_overwrite: bool = False, is_template: bool = True, logger: ~logging.Logger = <Logger datazen.parsing (WARNING)>, **kwargs) → LoadResult[source]#: Load raw file data and meld it into an existing dictionary. Update the result as if it’s a template using the provided variables.

datazen.parsing.set_file_hash(hashes: Dict[str, Any], path: Path | str | None, set_new: bool = True) → bool[source]#: Evaluate a hash dictionary and update it on a miss.

datazen.parsing.template_preprocessor_factory(variables: Dict[str, Any], is_template: bool, stack: ExitStack) → Callable[[TextIO | StringIO], TextIO | StringIO][source]#: Create a stream-processing function for data decoding.

datazen.paths module#

datazen - APIs for working with file paths.

datazen.paths.advance_dict_by_path(path_list: List[str], data: Dict[str, Any]) → Dict[str, Any][source]#: Given a dictionary and a list of directory names, return the child dictionary advanced by each key, in order, from the provided data.

datazen.paths.format_resolve_delims(value: str, fmt_data: Dict[str, Any], delim: str = '.', delim_replace: str = '_') → str[source]#: Attempt to resolve a format String with data, but handle replacements with custom delimeters correctly.

datazen.paths.get_path_list(root: Path | str | None, current: Path | str | None) → List[str][source]#: From a root directory and a child path, compute the list of directories, in order, mapping the root to the child.

datazen.paths.resolve_dir(data: str, rel_base: str = '') → str[source]#: Turn directory data into an absolute path, optionally from a relative base.

datazen.paths.unflatten_dict(data: Dict[str, Any], delim: str = '.') → Dict[str, Any][source]#: Attempt to unflatten dictionary data based on String keys and a delimeter.

datazen.paths.walk_with_excludes(path: Path | str | None, excludes: List[str] = None) → Iterator[Tuple[str, List[str], List[str]]][source]#: Behaves like os.walk but attempts to skip iterations that would enter directories we want to exclude (and thus not traverse).

datazen.schemas module#

datazen - Top-level APIs for loading and interacting with schema definitions.

datazen.schemas.add_global_schemas(schema_data: ~typing.Dict[str, ~typing.Dict[str, ~typing.Any]], logger: ~logging.Logger = <Logger datazen.schemas (WARNING)>) → None[source]#: Add schema-type registrations, globally.

datazen.schemas.inject_custom_schemas(schema_data: ~typing.Dict[str, ~typing.Dict[str, ~typing.Any]], should_inject: bool = True, logger: ~logging.Logger = <Logger datazen.schemas (WARNING)>) → Iterator[None][source]#: Allow the user to more easily control adding and removing global schema definitions.

datazen.schemas.load(directories: ~typing.Iterable[~pathlib.Path | str | None], require_all: bool = True, loads: ~datazen.load.LoadedFiles = (None, None), cls: ~typing.Type[~vcorelib.schemas.base.SchemaMap] = <class 'vcorelib.schemas.CerberusSchemaMap'>) → SchemaMap[source]#: Load schemas from a list of directories.

datazen.schemas.load_types(directories: List[Path | str | None], loads: LoadedFiles = (None, None)) → Dict[str, Dict[str, Any]][source]#: Load schema types and optionally register them.

datazen.schemas.remove_global_schemas(schema_data: ~typing.Dict[str, ~typing.Dict[str, ~typing.Any]], logger: ~logging.Logger = <Logger datazen.schemas (WARNING)>) → None[source]#: Remove schema-type registrations by key name.

datazen.schemas.validate(schema_data: ~typing.Dict[str, ~vcorelib.schemas.base.Schema], data: ~typing.Dict[str, ~typing.Any], logger: ~logging.Logger = <Logger datazen.schemas (WARNING)>) → bool[source]#: For every top-level key in the schema data, attempt to validate the provided data.

datazen.targets module#

datazen - An interface for parsing and matching targets.

datazen.targets.parse_targets(targets: List[Dict[str, Any]]) → Tuple[Dict[str, Dict[str, Any]], Dict[str, Dict[str, Any]]][source]#: From a list of target structures, parse them into a dictionary with keys as target names and data initialization to support future interaction.

datazen.targets.resolve_dep_data(entry: Dict[str, Any], data: Dict[str, Any]) → Dict[str, Any][source]#: Implements the business logic for applying match data to manifest entries.

datazen.targets.resolve_target_data(target_data: Dict[str, Any], match_data: Dict[str, str | int]) → Dict[str, Any][source]#: Resolve matched-target data into a target’s data from a manifest.

datazen.targets.resolve_target_list(target_list: List[Any], match_data: Dict[str, str | int]) → List[Any][source]#: Resovle matched-target data into a list form of target data from a manifest.

datazen.templates module#

datazen - Top-level APIs for loading and interacting with templates.

datazen.templates.environment(auto_reload: bool = False, autoescape_kwargs: ~typing.Dict[str, ~typing.Any] = None, lstrip_blocks: bool = True, trim_blocks: bool = True, undefined: ~typing.Type[~jinja2.runtime.Undefined] = <class 'jinja2.runtime.StrictUndefined'>, **kwargs) → Environment[source]#: Create a jinja environment with some sane defaults.

datazen.templates.load(template_dirs: Iterable[Path | str | None], loads: LoadedFiles = (None, None)) → Dict[str, Template][source]#: Load jinja2 templates from a list of directories where templates can be found.

datazen.templates.update_cache_primitives(dir_path: str, loads: LoadedFiles) → None[source]#: From a directory path, update the ‘loaded_list’ and ‘hashes’ primitives that belong to an upstream cache.

datazen.variables module#

datazen - Top-level APIs for loading and interacting with variables.

datazen.variables.load(directories: Iterable[Path | str | None], loads: LoadedFiles = (None, None)) → LoadResult[source]#: Load variable data from a list of directories.

Module contents#

Useful defaults and other package metadata.

datazen.to_private(input_str: str) → str[source]#: Convert some input to a ‘private’ String.