Getting Started

Compresso Recsys provides recommender-system experiments and pipeline utilities around Compresso sparse representation learning.

The public API is centered on:

  • dataset loaders for GoodBooks, MovieLens, Amazon Reviews 2023, Steam, Netflix Prize, MSD Taste Profile, Gowalla, DBbook, and Last.fm-2K

  • ZIP checkpoint helpers for storing splits, embeddings, sparse embeddings, and metrics

  • retrieval holdout builders for user split, item cold-start split, leave-last-out, and temporal evaluation

Basic Python Usage

import compresso_recsys as cr

dataset = cr.MovieLens1M(data_dir="data")
interactions = dataset.get_interactions()

print(interactions.head())

Datasets expose interactions with a canonical schema:

  • user_id as a string

  • item_id as a string

  • value as a float

  • timestamp when available

Checkpoint Workflow

Use compresso_recsys.build_recsys_checkpoint() to create the ZIP checkpoint used as the exchange format between stages:

import compresso_recsys as cr

checkpoint_path = cr.build_recsys_checkpoint(
    dataset="ml1m",
    checkpoint_path="artifacts/ml1m/exp001.zip",
    annotation_source="genres",
)

Checkpoint helpers can then read or update that checkpoint:

import compresso_recsys as cr

with cr.update_checkpoint(checkpoint_path) as root:
    split = cr.load_recsys_split(root)

Downstream scripts and adapters can add embeddings, sparse representations, metrics, or cluster graphs to the same checkpoint.

What Goes Into a Checkpoint

The dataset builder writes one portable ZIP file. The core split contains:

  • item_ids: global catalog order used by metadata and embedding stages

  • train_item_ids, val_item_ids, and test_item_ids: column order for each split’s source and target matrices; these equal item_ids for legacy single-catalog splits

  • train_source_matrix / train_target_matrix: sparse matrices defining the training input and target

  • val_source_matrix / val_target_matrix and test_source_matrix / test_target_matrix: sparse matrices defining validation/test retrieval inputs and targets

  • val_source_indices / val_target_indices and test_source_indices / test_target_indices: list-of-index views of the same validation/test holdouts

  • train_user_ids and val_eval_user_ids / test_eval_user_ids when user ids are meaningful for the split

  • warm_item_indices / val_cold_item_indices / test_cold_item_indices when the protocol partitions items, especially for cold-item experiments

  • optional entity_metadata, entity_tag_matrix, and tag_names

For ordinary reconstruction splits, x_train loaded by compresso_recsys.load_recsys_split() equals train_source_matrix. For temporal, it is the boolean union of the train source and target, so models that consume one interaction matrix can use every training-period event.

Additional experiment stages can append their own directories to the same checkpoint. Each stage can save embeddings, sparse representations, model files, and metrics.

Split Modes

user_split

Holds out validation/test users and builds source/target folds from those users. The checkpoint stores train_user_ids, val_user_ids, and test_user_ids. It is not a future-blind temporal protocol.

item_split

Holds out cold validation/test items. Downstream embedding stages should fit only on warm_item_indices and then transform all items before evaluation. The checkpoint stores item partitions rather than user partitions.

leave_last_out

Uses each user’s latest timestamped interaction as the target and earlier interactions as source. This requires timestamps and respects order within each user, but it is not globally future-blind: interactions from other users may occur after a given user’s held-out target.

temporal

Builds three equal target windows at the end of the timestamp range. The default window is 30 days for Gowalla and 339 days otherwise; change it with temporal_period_hours. Histories expand through time, and each split has a cumulative catalog containing warm items plus newly supported cold items. Source and target share a column order within a split, but train, validation, and test may have different widths.

Retrieval Metrics

Evaluation functions return calibrated_recall@K and ndcg@K for the single K requested. Examples commonly call them for K = 20, 50, 100 and store the six common metrics:

  • calibrated_recall@20

  • ndcg@20

  • calibrated_recall@50

  • ndcg@50

  • calibrated_recall@100

  • ndcg@100