{
"cells": [
{
"cell_type": "markdown",
"id": "e5fb7eea-a8ac-46c6-9e33-27a647b5f414",
"metadata": {},
"source": [
"# Clustering Example\n",
"\n",
"This example assumes that the `compresso-pytorch` and `compresso-recsys` packages are installed:\n",
"\n",
"```bash\n",
"pip install \"compresso-pytorch@git+https://github.com/zombak79/compresso.git\"\n",
"pip install \"compresso-recsys@git+https://github.com/zombak79/compresso-recsys.git\"\n",
"```\n",
"\n",
"This notebook demonstrates a basic clustering workflow. We first import Compresso and its recommender-system extension, then download Amazon metadata and build a checkpoint for the experiment.\n"
]
},
{
"cell_type": "markdown",
"id": "a6577858-9606-4507-9da4-0fe17410aa45",
"metadata": {},
"source": [
"## Build checkpoint & encode item descriptions"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "169f0da5-1b87-4266-9302-65415fd9ea8e",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "24f594fa87f946a3b48813417c18c719",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Building checkpoint: 0%| | 0/6 [00:00, ?step/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import compresso\n",
"import compresso_recsys as cr\n",
"import pandas as pd\n",
"\n",
"from compresso import clustering as cc\n",
"from compresso import TopKSAETrainer, TopKSAEConfig, L1Normalize\n",
"\n",
"checkpoint_path = cr.build_recsys_checkpoint(\n",
" dataset=\"amazon2023\",\n",
" amazon_category=\"Office_Products\",\n",
" checkpoint_path=\"artifacts/amazon_office.zip\",\n",
" metadata_text_fields=[\"title\", \"features\", \"description\", \"categories\"],\n",
" split_mode=\"item_split\",\n",
" min_entity_text_words=20,\n",
" val_items=500,\n",
" test_items=1000,\n",
" min_user_support=10,\n",
" item_min_support=10,\n",
" min_value_to_keep=1.0,\n",
" set_all_values_to=1.0,\n",
" min_source_items=1,\n",
" min_target_items=1,\n",
" annotation_source=\"none\",\n",
" show_progress=True,\n",
" include_image_urls=True,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "1ddc00d2-7455-473f-81e3-9e03ff51d71f",
"metadata": {},
"source": [
"Now let's load the item metadata from the checkpoint and inspect a sample of the data.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f5476ab8-225f-43be-a5ee-c663a821b4dc",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" item_id | \n",
" title | \n",
" entity_text | \n",
" image_url | \n",
"
\n",
" \n",
" \n",
" \n",
" | 1448 | \n",
" B01BYKUI9C | \n",
" HP 65 Black Ink Cartridge | Works with HP AMP ... | \n",
" Title: HP 65 Black Ink Cartridge | Works with ... | \n",
" https://m.media-amazon.com/images/I/61GAiSy74W... | \n",
"
\n",
" \n",
" | 2115 | \n",
" B07F9SVWXW | \n",
" Quartet Dry Erase Markers, Whiteboard Markers,... | \n",
" Title: Quartet Dry Erase Markers, Whiteboard M... | \n",
" https://m.media-amazon.com/images/I/61b1qTIOLB... | \n",
"
\n",
" \n",
" | 1921 | \n",
" B078Z65VQY | \n",
" Canon PIXMA G4210 Wireless All-In-One Supertan... | \n",
" Title: Canon PIXMA G4210 Wireless All-In-One S... | \n",
" https://m.media-amazon.com/images/I/81sbe681tY... | \n",
"
\n",
" \n",
" | 4175 | \n",
" B0BXR5F7LS | \n",
" Westcott B-70 8ths Graph Beveled Ruler, 12 in | \n",
" Title: Westcott B-70 8ths Graph Beveled Ruler,... | \n",
" https://m.media-amazon.com/images/I/81e+DtAFL5... | \n",
"
\n",
" \n",
" | 134 | \n",
" B0003WN0DO | \n",
" Sharpie Fine Electro Pop Marker, Fine Point, A... | \n",
" Title: Sharpie Fine Electro Pop Marker, Fine P... | \n",
" https://m.media-amazon.com/images/I/81W3-nzwxi... | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" item_id title \\\n",
"1448 B01BYKUI9C HP 65 Black Ink Cartridge | Works with HP AMP ... \n",
"2115 B07F9SVWXW Quartet Dry Erase Markers, Whiteboard Markers,... \n",
"1921 B078Z65VQY Canon PIXMA G4210 Wireless All-In-One Supertan... \n",
"4175 B0BXR5F7LS Westcott B-70 8ths Graph Beveled Ruler, 12 in \n",
"134 B0003WN0DO Sharpie Fine Electro Pop Marker, Fine Point, A... \n",
"\n",
" entity_text \\\n",
"1448 Title: HP 65 Black Ink Cartridge | Works with ... \n",
"2115 Title: Quartet Dry Erase Markers, Whiteboard M... \n",
"1921 Title: Canon PIXMA G4210 Wireless All-In-One S... \n",
"4175 Title: Westcott B-70 8ths Graph Beveled Ruler,... \n",
"134 Title: Sharpie Fine Electro Pop Marker, Fine P... \n",
"\n",
" image_url \n",
"1448 https://m.media-amazon.com/images/I/61GAiSy74W... \n",
"2115 https://m.media-amazon.com/images/I/61b1qTIOLB... \n",
"1921 https://m.media-amazon.com/images/I/81sbe681tY... \n",
"4175 https://m.media-amazon.com/images/I/81e+DtAFL5... \n",
"134 https://m.media-amazon.com/images/I/81W3-nzwxi... "
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"with cr.read_checkpoint(checkpoint_path) as root:\n",
" split = cr.load_recsys_split(root)\n",
" meta = split[\"entity_metadata\"]\n",
"\n",
"meta[[\"item_id\", \"title\", \"entity_text\", \"image_url\"]].sample(5)"
]
},
{
"cell_type": "markdown",
"id": "f5b058d3-475d-4b8c-9ebf-e4818dffa8bf",
"metadata": {},
"source": [
"Next, we create semantic embeddings with the `sentence_transformers` library.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "1297191e-98c8-45c3-bf0d-7742b66a2458",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "044a7771f90e4f8e998f29d54de1254d",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Loading weights: 0%| | 0/103 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "7a9fd39cf9ce4bb6ae05d153a9c44826",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Batches: 0%| | 0/139 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"(4445, 384)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from sentence_transformers import SentenceTransformer\n",
"\n",
"sbert = SentenceTransformer(\"sentence-transformers/all-MiniLM-L6-v2\")\n",
"\n",
"item_embeddings = sbert.encode(\n",
" meta.entity_text,\n",
" convert_to_numpy=True,\n",
" normalize_embeddings=True,\n",
" show_progress_bar=True\n",
")\n",
"\n",
"item_embeddings.shape"
]
},
{
"cell_type": "markdown",
"id": "f477ea04-20a8-4b7e-abe4-b9cfb7c4056f",
"metadata": {},
"source": [
"Now let's create a sparse autoencoder, train it, and obtain sparse item embeddings.\n"
]
},
{
"cell_type": "markdown",
"id": "5f373415-0ce9-4345-86fa-697a8a6089c3",
"metadata": {},
"source": [
"## Train sparse autoencoder and obtain sparse codes"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "a9ca70e1-0e7d-4bfd-b0e4-99234fd6ccdf",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "9a761ef7ea15467abfe7c7af3037bded",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0/100 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "dcfc6eb9301941a8bb391e1c96435452",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0/5 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/plain": [
"(4445, 4096)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sae = TopKSAETrainer(\n",
" TopKSAEConfig(\n",
" hidden_dim=4096,\n",
" k=32,\n",
" batch_size=1024,\n",
" epochs=100,\n",
" lr=1e-3,\n",
" decay=True,\n",
" post_sparsify=L1Normalize(),\n",
" sparsify_score_mode=\"abs\",\n",
" sparsify_ste_alpha=0.01,\n",
" device=\"cpu\",\n",
" )\n",
")\n",
"\n",
"sparse_embeddings = sae.fit_transform(item_embeddings)\n",
"\n",
"sparse_embeddings.shape"
]
},
{
"cell_type": "markdown",
"id": "e54a78df-73d3-47b7-abb4-a8f8bd7dd004",
"metadata": {},
"source": [
"## Clustering\n",
"\n",
"Next, we create clusters based on sparse-representation similarity."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "49ea3afa-b24d-4f74-abca-bd02262aee32",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "239f69ee808d45f59b8ae17cd6a834ad",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"build_srp_similarity_clusters: batches: 0%| | 0/139 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of discovered clusters: 28, active_clusters: 28, root clusters: 28\n"
]
}
],
"source": [
"cluster_graph = cc.SRPSimilarityClustering(\n",
" threshold=0.5, # minimum semantic similarity between items inside a cluster\n",
" top_k=None, # None = all pairs above threshold\n",
" min_cluster_size=20, # smaller clusters are discarded \n",
" normalize_rows=True, # centroids of the cluster will be normalized\n",
" min_local_density=None, # optional cleanup\n",
" centroid_top_k=8, # how many top features are included in centroid definition\n",
" batch_size=32,\n",
" show_progress=True,\n",
" )(sparse_embeddings)\n",
"\n",
"print(f\"Number of discovered clusters: {len(cluster_graph.clusters)}, active_clusters: {len(cluster_graph.active_clusters)}, root clusters: {len(cluster_graph.root_clusters)}\")"
]
},
{
"cell_type": "markdown",
"id": "d816de43-cee6-4878-b915-b89eec16a415",
"metadata": {},
"source": [
"## Labeling\n",
"\n",
"Next, we label the discovered clusters.\n",
"\n",
"For labeling, we need two functions:\n",
"\n",
"- a text extractor that receives a cluster and the item metadata dataframe, then returns item descriptions as a string\n",
"- a labeling function that receives those item descriptions and returns a concise cluster name\n",
"\n",
"This example uses a locally running `gpt-oss:20b` model through Ollama. The same pattern can be adapted to another local model or provider API.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4b4295ea-f9a7-482f-8173-510a722e6f77",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Items texts for the first cluster in the graph:\n",
"\n",
"Sharp EL-1901 Paperless Printing Calculator with Check and Correct, 12-Digit LCD Primary Display, Functions the Same as a Printing Calculator/Adding Machine with Scrolling LCD Display Instead of Paper\n",
"Texas Instruments TI-1795 SV Standard Function Calculator\n",
"Victor 1180-3A 12-Digit Standard Function Calculator, Battery and Solar Hybrid Powered Adjustable Angle LCD Display, Great for Home and Office Desks, Black\n",
"Sharp EL233SB Standard Function Calculator\n",
"Sharp El-1501 Compact Cordless Paperless Large 12-Digit Display Desktop Printing Calculator That Utilizes Printing Calculator Logic\n",
"Sharp EL-M335 10-Digit Extra Large Desktop Calculator with Currency Conversion Functions, Tax, Percent and Backspace Keys, and a Large Angled LCD Display, Perfect for Home or Office Use\n",
"Casio SL-100L Basic Solar Folding Compact Calculator, Multicolor\n",
"Sharp EL-1611V Handheld Portable Cordless 12 Digit Large LCD Display Two-Color Printing Calculator with Tax Functions, 191 x 99 x 42 mm\n",
"Casio SL-300VC Standard Function Calculator, Orange 2.75 x 4.63\n",
"Casio HS-8VA, Solar Powered Standard Function Calculator\n",
"\n",
"LLM label for the first cluster: Calculator Devices\n"
]
}
],
"source": [
"import ollama\n",
"\n",
"MODEL = \"gpt-oss:20b\"\n",
"\n",
"def label_cluster(items: str) -> str:\n",
" \n",
" prompt = f\"\"\"You are given a list of items, each with metadata.\n",
"\n",
" 1. Identify the most common concept that links all the items\n",
" 2. Convert that concept into a segment name suitable for a recommendation system.\n",
" \n",
" The segment name must be title-case, concise (2-5 words), and sound natural — like a content \n",
" category. Prefer distinctive labels over generic ones.\n",
" \n",
" Output: Only the final segment name, nothing else.\n",
" \n",
" Items:\n",
" {items}\n",
" \n",
" Segment name:\"\"\"\n",
"\n",
" response = ollama.chat(\n",
" model=MODEL,\n",
" messages=[\n",
" {\n",
" \"role\": \"system\",\n",
" \"content\": (\n",
" \"You are a strict classification model. \"\n",
" \"Return only one segment name. Never explain.\"\n",
" ),\n",
" },\n",
" {\n",
" \"role\": \"user\",\n",
" \"content\": prompt,\n",
" },\n",
" ],\n",
" options={\n",
" \"temperature\": 0.0,\n",
" \"num_predict\": 1000,\n",
" \"num_ctx\": 8192,\n",
" },\n",
" think=\"low\",\n",
" stream=False,\n",
" )\n",
"\n",
" content = response[\"message\"][\"content\"] if response[\"message\"][\"content\"] is not None else None\n",
" \n",
" return content\n",
"\n",
"def item_texts(cluster: compresso.clustering.types.SparseCluster, metadata: pd.DataFrame, limit: int = 50) -> str:\n",
" rows = metadata.iloc[cluster.entity_indices] # select rows with items that are members of the cluster\n",
" if len(rows) > limit: # optionally sample maximum allowed number of items to save tokens\n",
" rows = rows.sample(limit)\n",
" return \"\\n\".join(rows[\"title\"].fillna(\"\").tolist()) # just return item titles, one per row\n",
"\n",
"\n",
"print(\"Items texts for the first cluster in the graph:\\n\")\n",
"texts = item_texts(cluster_graph.clusters[0], meta, limit=10)\n",
"print(texts)\n",
"\n",
"print(\"\\nLLM label for the first cluster:\", label_cluster(texts))"
]
},
{
"cell_type": "markdown",
"id": "36ef2d7d-2414-4ac2-bfdf-eab3ad0aa58a",
"metadata": {},
"source": [
"Now we can apply the labeling function to the full cluster graph.\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "38f4d084-a492-4db7-b88d-13284f410493",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "711f0b9b34b04666a7dc4cc847d72ada",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"label_clusters: 0%| | 0/28 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"cluster_graph = cc.LabelClusters(\n",
" entity_metadata=meta,\n",
" text_extractor=item_texts,\n",
" label_fn=label_cluster,\n",
" cluster_scope=\"all\",\n",
" show_progress=True,\n",
" )(cluster_graph)"
]
},
{
"cell_type": "markdown",
"id": "759855df-45c0-46d0-83ca-a79ace45e46e",
"metadata": {},
"source": [
"## Visualization\n",
"\n",
"We can also visualize clusters alongside a sample of their items.\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "1747165d-cea8-4126-9f1d-c64f8fda3afc",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" | Cluster | \n",
" Items (sample) | \n",
"
\n",
" \n",
" \n",
" \n",
" | Desktop & Handheld Calculators | \n",
" | \n",
"
\n",
" \n",
" | Office Organization Essentials | \n",
" | \n",
"
\n",
" \n",
" | Office & Packaging Supplies | \n",
" | \n",
"
\n",
" \n",
" | Photo Printing Paper | \n",
" | \n",
"
\n",
" \n",
" | HP Printer Ink | \n",
" | \n",
"
\n",
" \n",
" | Paper Trimmers | \n",
" | \n",
"
\n",
" \n",
" | Dry Erase & Marker Supplies | \n",
" | \n",
"
\n",
" \n",
" | Binder Tab Dividers | \n",
" | \n",
"
\n",
" \n",
" | Notebook & Paper | \n",
" | \n",
"
\n",
" \n",
" | Pens & Pencils | \n",
" | \n",
"
\n",
" \n",
" | Pencil Sharpeners | \n",
" | \n",
"
\n",
" \n",
" | Premium Colored Pencils | \n",
" | \n",
"
\n",
" \n",
" | Label & File Organization | \n",
" | \n",
"
\n",
" \n",
" | Index Card Sets | \n",
" | \n",
"
\n",
" \n",
" | Staplers & Staples | \n",
" | \n",
"
\n",
" \n",
" | Envelopes\\n\\n | \n",
" | \n",
"
\n",
" \n",
" | Wireless All‑In‑One Printers | \n",
" | \n",
"
\n",
" \n",
" | Thermal Laminators & Pouches | \n",
" | \n",
"
\n",
" \n",
" | Office Printer Paper | \n",
" | \n",
"
\n",
" \n",
" | Ergonomic Gaming Desk Mat | \n",
" | \n",
"
\n",
" \n",
" | Cordless Home Phone Systems | \n",
" | \n",
"
\n",
" \n",
" | Epson All‑In‑One Printing Solutions | \n",
" | \n",
"
\n",
" \n",
" | Tombow Mono Stationery Essentials | \n",
" | \n",
"
\n",
" \n",
" | Multi‑Compartment Desk Caddy | \n",
" | \n",
"
\n",
" \n",
" | Electric Pencil Sharpeners | \n",
" | \n",
"
\n",
" \n",
" | Paper Shredders | \n",
" | \n",
"
\n",
" \n",
" | Planner & Calendar Collection | \n",
" | \n",
"
\n",
" \n",
" | Office Chair Mats | \n",
" | \n",
"
\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"from IPython.display import HTML\n",
"\n",
"# construct a dataframe with cluster label and 5 random sampled item images belonging to the cluster\n",
"def visualize_clusters(cluster_graph):\n",
" data = [\n",
" (\n",
" cluster_graph.clusters[i].label,\n",
" meta.iloc[cluster_graph.clusters[i].entity_indices].image_url.sample(5).to_list()\n",
" ) for i in range(len(cluster_graph.clusters))\n",
" ]\n",
" \n",
" df = pd.DataFrame(data, columns=[\"Cluster\", \"image_urls\"])\n",
"\n",
" def image_gallery(urls):\n",
" imgs = [\n",
" f'
'\n",
" for url in urls\n",
" ]\n",
" return \"\" + \"\".join(imgs) + \"
\"\n",
"\n",
" df_render = df.copy()\n",
" df_render[\"Items (sample)\"] = df_render[\"image_urls\"].map(image_gallery)\n",
" \n",
" html = df_render[[\"Cluster\", \"Items (sample)\"]].to_html(\n",
" escape=False,\n",
" index=False\n",
" )\n",
"\n",
" return HTML(html)\n",
"\n",
"visualize_clusters(cluster_graph)"
]
},
{
"cell_type": "markdown",
"id": "50be2702-7915-4aa4-b38f-9002a96c44a8",
"metadata": {},
"source": [
"## Using clustering pipelines\n",
"\n",
"The same steps can be combined with `ClusteringPipeline`. Results should be similar, although labels may vary because they are generated by an LLM.\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "3cb2c206-e44a-4b9b-a27e-12d34d86260c",
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "1c02e6daa51046cd9689773be30386d0",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"build_srp_similarity_clusters: batches: 0%| | 0/139 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "829004d2ecc847f7b7268f82cd9bc163",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"label_clusters: 0%| | 0/28 [00:00, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
" \n",
" \n",
" | Cluster | \n",
" Items (sample) | \n",
"
\n",
" \n",
" \n",
" \n",
" | Office & Home Standard Calculators | \n",
" | \n",
"
\n",
" \n",
" | Office Organization Essentials | \n",
" | \n",
"
\n",
" \n",
" | Office Stationery & Packing | \n",
" | \n",
"
\n",
" \n",
" | Photo Printing Paper | \n",
" | \n",
"
\n",
" \n",
" | HP Printer Ink | \n",
" | \n",
"
\n",
" \n",
" | Paper Trimmers | \n",
" | \n",
"
\n",
" \n",
" | Dry Erase & Whiteboard Supplies | \n",
" | \n",
"
\n",
" \n",
" | Binder Tab Organizers | \n",
" | \n",
"
\n",
" \n",
" | Notebook & Paper | \n",
" | \n",
"
\n",
" \n",
" | Pens & Pencils | \n",
" | \n",
"
\n",
" \n",
" | School & Office Pencil Sharpeners | \n",
" | \n",
"
\n",
" \n",
" | Premium Colored Pencils | \n",
" | \n",
"
\n",
" \n",
" | Labeling & Filing Essentials | \n",
" | \n",
"
\n",
" \n",
" | Index Card Essentials | \n",
" | \n",
"
\n",
" \n",
" | Staplers & Accessories | \n",
" | \n",
"
\n",
" \n",
" | Secure Mailing Envelopes | \n",
" | \n",
"
\n",
" \n",
" | Ink & Toner Supplies | \n",
" | \n",
"
\n",
" \n",
" | Thermal Laminators & Pouches | \n",
" | \n",
"
\n",
" \n",
" | Printing Paper | \n",
" | \n",
"
\n",
" \n",
" | Ergonomic Gaming Desk Mat | \n",
" | \n",
"
\n",
" \n",
" | Cordless Home Phone Systems | \n",
" | \n",
"
\n",
" \n",
" | Office All‑In‑One Printers | \n",
" | \n",
"
\n",
" \n",
" | Mono Stationery Essentials | \n",
" | \n",
"
\n",
" \n",
" | Desk Organizers | \n",
" | \n",
"
\n",
" \n",
" | Electric Pencil Sharpeners | \n",
" | \n",
"
\n",
" \n",
" | Office Paper Shredder | \n",
" | \n",
"
\n",
" \n",
" | Planner & Calendar Collection | \n",
" | \n",
"
\n",
" \n",
" | Office Chair Mats | \n",
" | \n",
"
\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"pipeline_cluster_graph = cc.ClusteringPipeline(\n",
" [\n",
" cc.SRPSimilarityClustering(\n",
" threshold=0.5, # minimum semantic similarity between items inside a cluster\n",
" top_k=None, # None = all pairs above threshold\n",
" min_cluster_size=20, # smaller clusters are discarded \n",
" normalize_rows=True, # centroids of the cluster will be normalized\n",
" min_local_density=None, # optional cleanup\n",
" centroid_top_k=8, # how many top features are included in centroid definition\n",
" batch_size=32,\n",
" show_progress=True,\n",
" ),\n",
" cc.LabelClusters(\n",
" entity_metadata=meta,\n",
" text_extractor=item_texts,\n",
" label_fn=label_cluster,\n",
" cluster_scope=\"all\",\n",
" show_progress=True,\n",
" ),\n",
" ]\n",
")(sparse_embeddings)\n",
" \n",
"visualize_clusters(pipeline_cluster_graph)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4ff0d86-4c42-40d2-968d-b6aef33e6d9a",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}