Scaling MongoDB for Billions of Documents: Best Practices for Data Storage and Processing

MongoDB is used frequently due to its flexibility, JSON-like documents, and horizontal scaling. However, as your dataset goes beyond millions and into billions of records, the choices you make on the first page of creating a system, such as data modeling, indexing, sharding, etc., can decide whether your system can be scaled or falls over due to the weight of its own data.

This post takes a tour of battle-tested best practices when managing MongoDB at scale, including schema design, choice of sharding strategies, ingestion pipelines, and operational protections.

1. Data Modeling at Scale

MongoDB is schema-flexible, but that doesn’t mean “schema-less.” At scale, you need intentional structure.

• Design around queries, not data. A well-designed index should support every query you run regularly.

• Document size: Keep documents small, typically a few KB to tens of KB. MongoDB’s hard limit is 16 MB per document, but you should stay far below this. Large binary blobs (images, logs, PDFs) belong in GridFS or external object storage.

• Avoid unbounded arrays. Don’t let a single document contain arrays that grow indefinitely (e.g., all orders for a user). This leads to document rewrites and poor performance. Instead, use bucket collections (e.g., group items by day/week) or separate child collections.

• Time-series data: If you’re storing events, metrics, or logs, MongoDB’s time-series collections are highly optimized for billions of rows. They bucket data automatically, compress efficiently, and support automatic data expiry.

Example:

db.createCollection("metrics", {
    timeseries: {
        timeField: "ts",
        metaField: "tags",
        granularity: "seconds"
    },
    expireAfterSeconds: 31536000 // 1 year retention
});

2. Sharding: Scaling Horizontally

For billions of documents, sharding is inevitable. The earlier you design for it, the easier your life will be.

• Pick the right shard key. This is the most critical decision in a large MongoDB deployment.

  • Avoid monotonically increasing keys (like timestamp) for high-ingest workloads because they create hot shards. Instead, use a hashed shard key or compound key.

  • Shards are created by tenantId (possibly hashed) for multi-tenant applications, so each tenant’s data is distributed.

  • For regional workloads, combine shard keys with zone sharding so queries stay local to their region.

• Pre-split and zone data. If you already know your data distribution (e.g., country codes, tenant IDs), define shard zones in advance to avoid expensive rebalancing later.

Example:

sh.shardCollection(
    "app.events",
    {
        tenantId: "hashed",
        ts: 1
    }
);

3. Indexing: Balance Reads and Writes

Indexes make queries fast, but every index slows down writes and consumes memory. At a billion-row scale, indexing discipline is essential.

• Use compound indexes tailored to your query patterns. MongoDB follows “prefix rules,” so an index { a:1, b:1, c:1 } can serve queries on a, a+b, and a+b+c.

• Keep index count low. Each index must be updated on every write, so over-indexing kills throughput.

• TTL indexes automatically expire documents, which is perfect for logs or session data.

• Partial indexes help focus on hot subsets (e.g., “active” or “pending” orders).

• Test with hidden indexes before committing, to see if queries would benefit.

Example:

db.orders.createIndex(
    { status: 1, updatedAt: -1, buyerId: 1 },
    {
        partialFilterExpression: {
            status: { $in: ["OPEN", "PENDING"] }
        }
    }
);

4. High-Throughput Ingestion

With billions of rows, ingestion efficiency matters as much as query speed.

• Batch inserts. Insert documents in batches of 1–10k (depending on doc size). This reduces round-trips.

• Bulk writes. Use unordered bulk writes for resilience and speed.

• Keep _id small. Use ObjectId or UUIDv7 instead of large strings.

• Write concern. Tune based on durability needs. For extreme throughput, it w:1 is faster thanw:"majority", but less durable.

• Staging collections. Ingest raw data into staging, then transform and merge into production collections.

5. Query & Aggregation at Scale

For analytics and reporting:

• Push $match and $project to the start of pipelines to reduce scanned data early.

• Use $merge or $out for building materialized views instead of ad-hoc, expensive queries.

• Replace MapReduce with Aggregation Framework (faster and simpler).

Example daily aggregation:

db.app.events.aggregate(
    [
        {
            $match: {
                ts: { $gte: ISODate("2025-08-01") }
            }
        },
        {
            $project: {
                tenantId: 1,
                ts: 1,
                value: 1
            }
        },
        {
            $group: {
                _id: {
                    day: {
                        $dateTrunc: {
                            date: "$ts",
                            unit: "day"
                        }
                    },
                    tenantId: "$tenantId"
                },
                sum: { $sum: "$value" }
            }
        },
        {
            $merge: {
                into: "daily_summaries",
                on: ["_id"],
                whenMatched: "replace",
                whenNotMatched: "insert"
            }
        }
    ],
    {
        allowDiskUse: true
    }
);

6. Storage Engine & Resource Tuning

MongoDB uses WiredTiger by default. Tuning matters at scale:

• Hardware: Use NVMe SSDs and a fast network (25–40 GbE for clusters). Filesystem: XFS is recommended.

• Compression: Default snappy is balanced. Use zstd for cold data (better compression), or no compression for high-ingest workloads.

• RAM: Aim for your working set (active indexes + frequently read data) to fit in memory. MongoDB allocates 50% of RAM to WT cache by default.

• OS settings: Disable Transparent Huge Pages, set ulimit high, and minimize swapping.

7. Replication & Durability

For safety and uptime:

• Use 3+ members per replica set per shard, spread across independent zones.

• Size the oplog window to cover expected backup lag (hours to days).

• Journaling should stay enabled. It protects against crashes.

• Backups: use filesystem snapshots + oplog-based PITR. Test restores regularly.

8. Monitoring & Operations

With billions of documents, blind spots are fatal. Track:

• Query plans (explain() should show IXSCAN, not COLLSCAN).

• Cache usage (watch page faults and eviction in WiredTiger).

• Replication lag (especially during heavy ingestion).

• Balancer activity (avoid running during peak traffic).

• Oplog growth (to prevent replication failures).

Tools:

• mongostat, mongotop For quick checks.

• Atlas, Prometheus, or Ops Manager for dashboards.

9. Multi-Tenancy & Tiered Data

Not all data is equally hot:

• Separate hot and cold collections. Use indexes only where needed.

• Archive old data to dedicated collections with higher compression and fewer indexes.

• Zone sharding ensures tenant or region data stays in the right place.

10. Common Pitfalls

• Choosing a bad shard key (monotonic → hot shard).

• Allowing too many indexes on write-heavy collections.

• Using large arrays in documents.

• Running COLLSCANs on production workloads.

• Forgetting to size the oplog for peak ingest.

• Letting the balancer interfere with peak operations.

Final Thoughts

MongoDB can handle billions of documents only if designed and operated thoughtfully. The core ideas are:

• Model data for queries.

• Choose a shard key with care.

• Balance indexes against write speed.

• Monitor constantly.

• Automate data lifecycle (TTL, archiving, materialized views).

Done right, MongoDB becomes a scalable, cost-efficient backbone for everything from IoT and event logging to real-time analytics and multi-tenant SaaS platforms.

Read more: Which Databases are Best for Fast Delivery, and High-Traffic Applications

Tags: