Retrieval API

Holdout Builders

compresso_recsys.retrieval.build_eval_holdout(*, train_item_ids, eval_interactions, min_user_support=5, random_state=42, eval_draws=1, eval_holdout_frac=0.2)[source]

Build a fixed source/target holdout for strongly generalized evaluation.

Each held-out user’s items are split into a fold-in history the model sees and a held-out share it is scored against, following Liang et al. (2018). eval_holdout_frac is the scored share; their description sets it to 0.2.

eval_draws defaults to one split per user. Increasing it repeats that split independently, stacking one row per user per draw. The ELSA line of papers uses five. More draws sharpen each user’s score by averaging over which items happened to be held out; they do not add independent observations, so paired comparison groups the rows back together by user.

Return type:

dict[str, object]

Parameters:

  • train_item_ids (Index | ndarray)

  • eval_interactions (DataFrame)

  • min_user_support (int)

  • random_state (int)

  • eval_draws (int)

  • eval_holdout_frac (float)

compresso_recsys.retrieval.build_item_cold_holdout(*, item_ids, interactions, source_item_ids, target_item_ids, min_source_items=1, min_target_items=1)[source]

Build source=train-item and target=cold-item holdout for overlapping users.

Return type:

dict[str, object]

Parameters:

  • item_ids (Index | ndarray)

  • interactions (DataFrame)

  • source_item_ids (set[str] | list[str] | ndarray)

  • target_item_ids (set[str] | list[str] | ndarray)

  • min_source_items (int)

  • min_target_items (int)

compresso_recsys.retrieval.build_leave_last_out_holdout(*, item_ids, interactions, stage='test', min_history=4)[source]

Build one stage of the leave-last-out holdout.

Each user’s chronologically last interaction is the test target, the one before it the validation target, and the one before that the training target. Sources are the corresponding prefixes, so each stage’s source is the previous stage’s source plus the previous stage’s target.

Nothing is removed from the catalog. An item is absent from training only when every one of its occurrences happens to fall in a held-out tail, which is a property of the data rather than something this function imposes.

Return type:

dict[str, object]

Parameters:

  • item_ids (Index | ndarray)

  • interactions (DataFrame)

  • stage (str)

  • min_history (int)

compresso_recsys.retrieval.build_temporal_holdout(*, item_ids, interactions, test_frac=0.1, min_source_items=1, min_target_items=1)[source]

Build source/target using a global timestamp cutoff.

Return type:

dict[str, object]

Parameters:

  • item_ids (Index | ndarray)

  • interactions (DataFrame)

  • test_frac (float)

  • min_source_items (int)

  • min_target_items (int)

Embedding Evaluation

compresso_recsys.retrieval.evaluate_item_embeddings(*, train_item_ids, item_embeddings, eval_interactions, k=100, eval_holdout_frac=0.2, min_user_support=5, random_state=42, eval_draws=1, score_batch_size=512, metrics=None, debug=False, debug_users=5, show_progress=False)[source]

Evaluate item embeddings with torch top-k retrieval.

  • User profile: sum of source-item embeddings.

  • Scores: dot(profile, item_embedding).

  • Seen source items are masked.

Return type:

dict[str, Any]

Parameters:

  • train_item_ids (Index)

  • item_embeddings (ndarray)

  • eval_interactions (DataFrame)

  • k (int)

  • eval_holdout_frac (float)

  • min_user_support (int)

  • random_state (int)

  • eval_draws (int)

  • score_batch_size (int)

  • metrics (Sequence[RankingMetric] | None)

  • debug (bool)

  • debug_users (int)

  • show_progress (bool)

compresso_recsys.retrieval.evaluate_item_embeddings_with_holdout(*, item_embeddings, source_indices, target_indices, k=100, score_batch_size=512, metrics=None, debug=False, debug_users=5, show_progress=False)[source]

Evaluate item embeddings against a precomputed source/target holdout.

Predictions are generated and evaluated one batch at a time. Supplying metrics allows multiple cutoffs to reuse the same ranked predictions and target-hit tensor.

Return type:

dict[str, Any]

Parameters:

  • item_embeddings (ndarray)

  • source_indices (list[ndarray])

  • target_indices (list[ndarray])

  • k (int)

  • score_batch_size (int)

  • metrics (Sequence[RankingMetric] | None)

  • debug (bool)

  • debug_users (int)

  • show_progress (bool)