Core API

Datasets

class compresso_recsys.SplitBundle(train, val, test)

Parameters:

• train (DataFrame)

• val (DataFrame)

• test (DataFrame)

class compresso_recsys.RecSysDataset(data_dir='data')

Thin base class for interaction datasets used in example pipelines.

Canonical interactions schema: - user_id: str - item_id: str - value: float - timestamp: int | float | None

Parameters:

data_dir (str | Path)

static preprocess_interactions_for_recsys(df, *, min_value_to_keep=4.0, user_min_support=5, item_min_support=1, set_all_values_to=1.0, max_steps=0)

Paper-style preprocessing: threshold, binarize, iterative pruning, categorical cleanup.

Return type:

DataFrame

Parameters:

• df (DataFrame)

• min_value_to_keep (float | None)

• user_min_support (int)

• item_min_support (int)

• set_all_values_to (float | None)

• max_steps (int)

to_hf_dataset(df=None)

Convert interactions to HuggingFace Dataset.

Import is optional so core library does not hard-depend on datasets.

Parameters:

df (DataFrame | None)

static to_sparse_matrix(df)

Return (X, user_ids, item_ids) where X is user x item CSR.

Parameters:

df (DataFrame)

class compresso_recsys.Goodbooks(data_dir='data', *, metadata_text_fields=None, min_entity_text_words=0)

Parameters:

• data_dir (str)

• metadata_text_fields (Iterable[str] | None)

• min_entity_text_words (int)

class compresso_recsys.MovieLens1M(data_dir='data', *, metadata_text_fields=None, min_entity_text_words=0)

Parameters:

• data_dir (str)

• metadata_text_fields (Iterable[str] | None)

• min_entity_text_words (int)

class compresso_recsys.MovieLens20M(data_dir='data', *, metadata_text_fields=None, min_entity_text_words=0)

Parameters:

• data_dir (str)

• metadata_text_fields (Iterable[str] | None)

• min_entity_text_words (int)

class compresso_recsys.AmazonReviews2023(data_dir='data', *, category='Toys_and_Games', metadata_text_fields=None, min_entity_text_words=0, include_image_urls=False, show_progress=True)

Amazon Reviews 2023 category dataset loaded from McAuley’s files.

The recommender pipeline only needs compact rating-only interactions plus item metadata. Reviews are intentionally not downloaded.

Parameters:

• data_dir (str)

• category (str)

• metadata_text_fields (Iterable[str] | None)

• min_entity_text_words (int)

• include_image_urls (bool)

• show_progress (bool)

load_source_dataframe(config, *, split='full')

Load a McAuley Amazon 2023 config into a DataFrame.

Kept under its historical name for compatibility with tests and subclasses, but this no longer uses Hugging Face datasets. Recent datasets releases reject repositories that still expose loading scripts, so we read direct Hugging Face/McAuley data files instead.

Return type:

DataFrame

Parameters:

• config (str)

• split (str)

load_timestamp_splits_with_history()

Load McAuley’s timestamp split with per-row history fields.

Return type:

dict[str, DataFrame]

Checkpoint Helpers

compresso_recsys.build_recsys_checkpoint(*, dataset, data_dir='data', checkpoint_path=None, seed=None, val_users=None, test_users=None, min_user_support=None, item_min_support=None, min_value_to_keep=None, set_all_values_to=None, eval_draws=1, eval_holdout_frac=0.2, split_mode='user_split', val_items=None, test_items=None, item_val_frac=0.05, item_test_frac=0.1, temporal_test_frac=None, temporal_period_hours=None, min_source_items=1, min_target_items=1, amazon_category='Toys_and_Games', metadata_text_fields=None, min_entity_text_words=None, include_image_urls=False, annotation_source='genres', annotation_min_count=100, show_progress=True, multimodal_features=None)

Build a recommender-system split checkpoint and return its path.

temporal_period_hours=None uses 720 hours for Gowalla and 8136 for other datasets. An explicit positive period overrides that default.

Return type:

Path

Parameters:

• dataset (str)

• data_dir (str)

• checkpoint_path (str | None)

• seed (int | None)

• val_users (int | None)

• test_users (int | None)

• min_user_support (int | None)

• item_min_support (int | None)

• min_value_to_keep (float | None)

• set_all_values_to (float | None)

• eval_draws (int)

• eval_holdout_frac (float)

• split_mode (str)

• val_items (int | None)

• test_items (int | None)

• item_val_frac (float)

• item_test_frac (float)

• temporal_test_frac (float | None)

• temporal_period_hours (float | None)

• min_source_items (int)

• min_target_items (int)

• amazon_category (str)

• metadata_text_fields (str | list[str] | tuple[str, ...] | None)

• min_entity_text_words (int | None)

• include_image_urls (bool)

• annotation_source (str)

• annotation_min_count (int)

• show_progress (bool)

• multimodal_features (str | list[str] | None)

compresso_recsys.update_checkpoint(path)

Extract a zip checkpoint to a temp dir, let caller edit it, then rewrite it.

Return type:

Iterator[Path]

Parameters:

path (str | Path)

compresso_recsys.read_checkpoint(path)

Extract a zip checkpoint to a read-only temp workspace.

Return type:

Iterator[Path]

Parameters:

path (str | Path)

compresso_recsys.load_manifest(root)

Return type:

dict[str, Any]

Parameters:

root (str | Path)

compresso_recsys.save_manifest(root, manifest)

Return type:

None

Parameters:

• root (str | Path)

• manifest (dict[str, Any])

compresso_recsys.update_stage_manifest(root, stage, metadata)

Return type:

None

Parameters:

• root (str | Path)

• stage (str)

• metadata (dict[str, Any])

compresso_recsys.save_json(root, relpath, data)

Return type:

Path

Parameters:

• root (str | Path)

• relpath (str)

• data (dict[str, Any])

compresso_recsys.load_json(root, relpath)

Return type:

dict[str, Any]

Parameters:

• root (str | Path)

• relpath (str)

compresso_recsys.save_recsys_split(root, *, item_ids, x_train, train_item_ids=None, val_item_ids=None, test_item_ids=None, val_source_indices, val_target_indices, test_source_indices, test_target_indices, train_source_matrix=None, train_target_matrix=None, val_source_matrix=None, val_target_matrix=None, test_source_matrix=None, test_target_matrix=None, train_user_ids=None, val_user_ids=None, test_user_ids=None, val_eval_user_ids=None, test_eval_user_ids=None, warm_item_indices=None, val_cold_item_indices=None, test_cold_item_indices=None, x_train_sequences=None, train_source_sequences=None, val_source_sequences=None, test_source_sequences=None, entity_tag_matrix=None, tag_names=None, entity_metadata=None, metadata=None)

Write the split stage of a checkpoint.

Return type:

None

Parameters:

• root (str | Path)

• item_ids (ndarray)

• x_train (csr_matrix)

• train_item_ids (ndarray | list[str] | None)

• val_item_ids (ndarray | list[str] | None)

• test_item_ids (ndarray | list[str] | None)

• val_source_indices (list[ndarray])

• val_target_indices (list[ndarray])

• test_source_indices (list[ndarray])

• test_target_indices (list[ndarray])

• train_source_matrix (csr_matrix | None)

• train_target_matrix (csr_matrix | None)

• val_source_matrix (csr_matrix | None)

• val_target_matrix (csr_matrix | None)

• test_source_matrix (csr_matrix | None)

• test_target_matrix (csr_matrix | None)

• train_user_ids (ndarray | list[str] | None)

• val_user_ids (ndarray | list[str] | None)

• test_user_ids (ndarray | list[str] | None)

• val_eval_user_ids (ndarray | list[str] | None)

• test_eval_user_ids (ndarray | list[str] | None)

• warm_item_indices (ndarray | None)

• val_cold_item_indices (ndarray | None)

• test_cold_item_indices (ndarray | None)

• x_train_sequences (ItemSequences | None)

• train_source_sequences (ItemSequences | None)

• val_source_sequences (ItemSequences | None)

• test_source_sequences (ItemSequences | None)

• entity_tag_matrix (csr_matrix | None)

• tag_names (ndarray | list[str] | None)

• entity_metadata (DataFrame | None)

• metadata (dict[str, Any] | None)

Training matrices

Three keys describe the same training data, and the relationship between them is fixed:

x_train = train_source_matrix ∪ train_target_matrix

x_train is what a symmetric model trains on — an autoencoder reconstructs the whole window. The pair is what an asymmetric model trains on, mapping source to target. They must agree, and this function refuses a checkpoint where they do not.

How the training data is partitioned follows each split mode’s protocol, and only the chronological modes have one to follow:

• temporal: by time. Source is everything before the first target window, target is the events inside it.

• leave_last_out: by position. Target is the last interaction of the training window, source is everything earlier.

• user_split and item_split: no partition. Both keys equal x_train, and the invariant holds trivially.

The last case is deliberate rather than a gap. A non-chronological split has no boundary to divide on, so any per-user division would be an arbitrary choice invented here rather than a property of the protocol. A model wanting asymmetric training on those modes can partition x_train itself, under its own seed, and own that choice. The same absence of an ordering is why sequences exist only for the chronological modes.

Sequence views

x_train_sequences and {stage}_source_sequences carry the same events as their matrix counterparts, in chronological order and with duplicates kept. A matrix row is a set; a sequence row is a history. Targets have no sequence view because a ranking target is a set — order is irrelevant to every metric — so {stage}_target_matrix serves both model families.

They are written only when the split mode produced them, which means the chronological modes. user_split and item_split have no ordering to preserve, and the same absence that makes their training partition arbitrary (above) makes a sequence meaningless.

Loading a checkpoint without them yields None rather than an error. A checkpoint that predates sequences, or comes from a non-chronological mode, is still complete for every matrix model, so refusing it would break working setups over a field they never touch. A sequential model fails later, where the message can name the split mode that would have produced them.

A sequence whose n_items disagrees with its own stage’s item IDs is refused: the two views must share a column space or a model scores one item and is credited for another. Per stage rather than globally, because temporal windows each have their own catalog — it grows window by window — which is the same allowance the matrix check above makes.

Item partitions

warm_item_indices, val_cold_item_indices and test_cold_item_indices are positions into item_ids naming the items that phase introduces, not the items it may score. Together they partition the catalog by first appearance: the warm partition is exactly the columns present in x_train, and each cold partition holds the items that become observable only at that stage.

They are named for what they hold rather than for their phase because the older {phase}_item_indices spelling promised a relationship to {phase}_item_ids that does not exist. The two answer different questions: *_item_ids is the column space a phase lives in, while these are a partition by first appearance. The two agree only by coincidence, and only under temporal and user_split, where the catalogs already encode the partition; under leave_last_out and item_split all three phases share one catalog and the partition is observed, so it cannot be recovered from the catalogs at all.

• user_split: training spans every item and the later phases introduce none, so the train partition is the full range and val/test are empty.

• item_split: three disjoint partitions, the val/test ones being the cold items held out of training.

• leave_last_out: nothing is held out of the catalog. An item lands in the val or test partition only when every one of its occurrences falls in a held-out tail, so on dense data both partitions are empty and on sparse data they hold the genuinely new items.

• temporal: each phase introduces the items first seen in its window, so the partitions are consecutive ranges of the growing catalog.

An empty partition therefore means “this phase introduces no new items”, which is not the same as “this phase has no candidates”. The candidate space of a phase is {phase}_item_ids, which defaults to item_ids when not given. Callers that select feature or metadata rows for a phase should index with that phase’s *_item_ids (or the union of partitions up to it), because mirroring warm_item_indices into a later phase silently yields an empty selection for splits that hold no items out.

Passing None for a partition omits its file, and load_recsys_split() then falls back to the whole catalog for the warm partition and to an empty array for the cold ones. Prefer writing all three explicitly, since those defaults turn an omission into a confident wrong answer rather than an error.

compresso_recsys.load_recsys_split(root)

Read the split stage of a checkpoint.

See save_recsys_split() for what *_item_indices mean: they are the items each phase introduces, so they are empty for phases that hold no items out, while *_item_ids give the candidate space and default to item_ids.

For checkpoints written before every partition was stored explicitly, a missing warm_item_indices.npy loads as the full catalog range and missing cold files load as empty arrays. Checkpoints written before the rename are read under their old names first, because those defaults would otherwise turn a missing file into a confident wrong answer.

Return type:

dict[str, Any]

Parameters:

root (str | Path)

compresso_recsys.save_cluster_graph_stage(root, graph, *, stage_dir='clustering', metadata=None)

Return type:

Path

Parameters:

• root (str | Path)

• graph (SparseClusterSet)

• stage_dir (str)

• metadata (dict[str, Any] | None)

compresso_recsys.load_cluster_graph_stage(root, *, stage_dir='clustering')

Return type:

SparseClusterSet

Parameters:

• root (str | Path)

• stage_dir (str)