cellxgene-census – skill Dữ liệu & Khoa học cho agent AI

CZ CELLxGENE Census

Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes:

• 217+ million total cells and 125+ million unique cells in the 2025-11-08 stable LTS release
• 1,845 datasets in the 2025-11-08 stable LTS release
• Human, mouse, marmoset, rhesus macaque, and chimpanzee data in the current schema
• Standardized metadata (cell types, tissues, diseases, donors)
• Raw gene expression matrices and source H5AD lookup/download helpers
• Pre-calculated summary counts, embeddings, and spatial data
• Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools

When to Use This Skill

This skill should be used when:

• Querying single-cell expression data by cell type, tissue, or disease
• Exploring available single-cell datasets and metadata
• Training machine learning models on single-cell data
• Performing large-scale cross-dataset analyses
• Integrating Census data with scanpy or other analysis frameworks
• Computing statistics across millions of cells
• Accessing pre-calculated embeddings or model predictions

Installation and Setup

Install the Census API:

uv pip install "cellxgene-census==1.18.0"

For spatial workflows:

uv pip install "cellxgene-census[spatial]==1.18.0"

For PyTorch model training, use TileDB-SOMA-ML. The old cellxgene_census.experimental.ml loaders are absent from 1.18.0:

uv pip install "cellxgene-census==1.18.0" tiledbsoma-ml

Core Workflow Patterns

Eight patterns, each with code, are in

[references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. Opening the Census — always pin census_version so an analysis stays reproducible.

2. Exploring Census information — available datasets, cell counts, and summary tables.

3. Querying expression data — small to medium scale into an AnnData.

4. Large-scale queries — out-of-core processing when the slice will not fit in memory.

5. Machine learning with PyTorch — TileDB-SOMA-ML data loaders.

6. Spatial Census data — accessing spatial assays.

7. Integration with Scanpy — handing a Census slice to a standard Scanpy workflow.

8. Multi-dataset integration — combining datasets and handling batch effects.

Key Concepts and Best Practices

The examples pin the current LTS build 2025-11-08, verified through the live

release directory on 2026-09-30. The SDK and data release are separate versions.

Resolve stable once with get_census_version_description("stable")["release_build"]

and record that date; never silently switch builds midway through analysis.

Always Filter for Primary Data

Unless analyzing duplicates, always include is_primary_data == True in queries to avoid counting cells multiple times:

obs_value_filter="cell_type == 'B cell' and is_primary_data == True"

Specify Census Version for Reproducibility

Always specify the Census version in production analyses:

census = cellxgene_census.open_soma(census_version="2025-11-08")

Estimate Query Size Before Loading

For large queries, count the selected rows without loading every metadata column.

Cell count alone is not a memory estimate: gene count, sparsity, dtype, layers,

embeddings, and downstream dense copies also matter:

import tiledbsoma as soma
with census["census_data"]["homo_sapiens"].axis_query(
    measurement_name="RNA",
    obs_query=soma.AxisQuery(
        value_filter="tissue_general == 'brain' and is_primary_data == True"
    ),
) as query:
    print(f"Selected {query.n_obs:,} cells and {query.n_vars:,} genes")

# Query axes consume memory too; stream expression if the matrix will not fit.

Use tissue_general for Broader Groupings

The tissue_general field provides coarser categories than tissue, useful for cross-tissue analyses:

# Broader grouping
obs_value_filter="tissue_general == 'immune system'"

# Specific tissue
obs_value_filter="tissue == 'venous blood'"

Select Only Needed Columns

Minimize data transfer by specifying only required metadata columns:

obs_column_names=["cell_type", "tissue_general", "disease"]  # Not all columns

Check Dataset Presence for Gene-Specific Queries

When analyzing specific genes, verify which datasets measured them:

genes = cellxgene_census.get_var(
    census, "homo_sapiens",
    value_filter="feature_name in ['CD4', 'CD8A']",
    column_names=["soma_joinid", "feature_id", "feature_name"],
)
presence = cellxgene_census.get_presence_matrix(census, "homo_sapiens")
# Columns use Census join IDs, not positions in the filtered gene table.
gene_presence = presence[:, genes["soma_joinid"].to_numpy()]

Gene symbols are not necessarily unique in schema 2.4.0; keep feature_id as the

feature key, inspect all symbol matches, and never silently select the first match.

Presence rows are dataset soma_joinid values, not cell IDs; a zero means the

feature was not measured in that dataset, not that measured expression was zero.

The remote-query snippets are illustrative; verify the selected release and

returned schema before loading a large expression slice.

Two-Step Workflow: Explore Then Query

First explore metadata to understand available data, then query expression:

# Step 1: Explore what's available
metadata = cellxgene_census.get_obs(
    census, "homo_sapiens",
    value_filter="disease == 'COVID-19' and is_primary_data == True",
    column_names=["cell_type", "tissue_general"]
)
print(metadata.value_counts())

# Step 2: Query based on findings
adata = cellxgene_census.get_anndata(
    census=census,
    organism="Homo sapiens",
    obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True",
)

For complete disease cohorts, use the multi-value disease workflow in

[references/common_patterns.md](references/common_patterns.md). Exact equality in

the small examples selects only cells whose whole disease field equals that label.

Available Metadata Fields

Cell Metadata (obs)

Key fields for filtering:

• cell_type, cell_type_ontology_term_id
• tissue, tissue_general, tissue_ontology_term_id
• disease, disease_ontology_term_id
• assay, assay_ontology_term_id
• donor_id, sex, self_reported_ethnicity
• development_stage, development_stage_ontology_term_id
• dataset_id
• is_primary_data (Boolean: True = primary representation)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with list(census["census_data"].keys()).

Gene Metadata (var)

• feature_id (Ensembl gene ID, e.g., "ENSG00000161798")
• feature_name (Gene symbol, e.g., "FOXP2")
• feature_type (present in the verified 2025-11-08 build; inspect the selected schema)
• feature_length (Gene length in base pairs)
• nnz, n_measured_obs (availability summaries useful for checking sparsity and coverage)

Reference Documentation

This skill includes detailed reference documentation:

Sources, endpoint contracts, H5AD lookup, and embeddings are documented in

[references/api_access.md](references/api_access.md).

references/census_schema.md

Comprehensive documentation of:

• Census data structure and organization
• All available metadata fields
• Value filter syntax and operators
• SOMA object types
• Data inclusion criteria

When to read: When you need detailed schema information, full list of metadata fields, or complex filter syntax.

references/common_patterns.md

Examples and patterns for:

• Exploratory queries (metadata only)
• Small-to-medium queries (AnnData)
• Large queries (out-of-core processing)
• PyTorch integration
• Spatial Census access patterns
• Scanpy integration workflows
• Multi-dataset integration
• Best practices and common pitfalls

When to read: When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

Common Use Cases

Use Case 1: Explore Cell Types in a Tissue

with cellxgene_census.open_soma(census_version="2025-11-08") as census:
    cells = cellxgene_census.get_obs(
        census, "homo_sapiens",
        value_filter="tissue_general == 'lung' and is_primary_data == True",
        column_names=["cell_type"]
    )
    print(cells["cell_type"].value_counts())

Use Case 2: Query Marker Gene Expression

with cellxgene_census.open_soma(census_version="2025-11-08") as census:
    adata = cellxgene_census.get_anndata(
        census=census,
        organism="Homo sapiens",
        var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']",
        obs_value_filter="cell_type in 

… (nội dung đầy đủ có trong gói tải về)