Welcome to Compresso’s documentation!

Compresso is a small PyTorch library for fixed-k sparse representations, sparse autoencoders, sparse parameters, and sparse-representation clustering.

The project distribution is named compresso-pytorch and the Python package is imported as compresso.

import numpy as np
from compresso import TopKSAEConfig, TopKSAETrainer

embeddings = np.random.randn(10_000, 512).astype("float32")
srp = TopKSAETrainer(TopKSAEConfig(hidden_dim=4096, k=32, epochs=50)).fit_transform(embeddings)
print(srp)   # SRPTensor(shape=(10000, 4096), k=32, ...)

New here? Read Getting Started for the concepts, then A First Example: Seeing What an SAE Learns to see what a sparse autoencoder learns.

Getting Started

• Installation
  • Install from PyPI
  • Install from Source
  • Development Install
  • Build the Documentation Locally
• Getting Started
  • Why sparse representations?
  • The top-k sparse autoencoder
  • Key objects
  • Your first run
  • Fit and transform separately
  • What’s next

User Guide

• A First Example: Seeing What an SAE Learns
  • Treat each image as a dense vector
  • Train a top-k SAE
  • The decoder is a dictionary
  • A code is a recipe
  • Reading a sparse code
  • Where to go next
• Advanced Usage
  • The raw model: TopKSAE
  • The stats dictionary
  • Encoder, decoder, and tied weights
  • Controlling sparsification
  • Post-sparsification hooks
  • Full config reference
  • Sparse parameters and pruning
• Input and Output
  • Input data
  • The SRPTensor format
  • Building one directly
  • Converting to other formats
  • Saving and loading
  • Using codes in another run or project
• Clustering Sparse Codes
  • The idea
  • Step 1 — from text to sparse codes
  • Step 2 — build a clustering pipeline
  • Step 3 — the entity-tag matrix (optional labels)
  • What you get
  • Inspecting clusters
  • Saving cluster graphs
  • Other building blocks
• Clustering Example
  • Build checkpoint & encode item descriptions
  • Train sparse autoencoder and obtain sparse codes
  • Clustering
  • Labeling
  • Visualization
  • Using clustering pipelines

API Reference

• API Reference
  • Core API
  • Clustering API

Indices and Tables

• Index

• Module Index

• Search Page