GridFS

#What GridFS is

A MongoDB document cannot exceed 16 MB. GridFS is how MongoDB stores files bigger than that: the file is cut into chunks across two collections — <bucket>.files, one document of metadata per file, and <bucket>.chunks, one document per chunk of data.

If you have ever opened a MongoDB database and found a collection full of metadata next to a collection full of binary crumbs, that was a GridFS bucket that your client did not recognise.

#TableCore shows the bucket, not the halves

The explorer detects buckets from the collection names it already has, so recognising one costs no extra query. A bucket row replaces both collections in the tree, has its own icon, and offers file actions rather than collection actions — rename, indexes and reshape make no sense applied to half a bucket.

Two rules keep that honest:

• A bucket needs both halves. A lone .files or .chunks stays an ordinary collection. Without the metadata collection the driver cannot list files; without the chunks it cannot read bytes. A bucket row that opened a browser answering nothing would be worse than showing the collection.
• A bucket name may contain dots. images.thumbnails.files plus images.thumbnails.chunks is one bucket named images.thumbnails: the split is on the last suffix, not the first dot. A collection literally named files is a collection named files, and system.* is never a bucket.

#Browsing, downloading, uploading

Browsing a bucket is free. Moving bytes is not:

They are the same entitlements as export and import, applied to one file. Locking the bucket itself would put you back in front of two unexplained collections, which is the problem this feature solves.

Everything streams, through a 1 MiB buffer. This feature exists for files too big to be documents, so nothing holds a whole file, and progress is counted in bytes rather than chunks — chunks are an implementation detail of the bucket and say nothing to somebody watching a 2 GB video move.

#Same name means versions, not collisions

Files are listed sorted by name, newest upload first, and each carries its revision number — 0 is the current one — and the total number of revisions.

Uploading a file whose name already exists creates a new current version. It does not overwrite anything, and the dialog says that rather than warning you about an overwrite that is not going to happen.

#A failed transfer leaves nothing pretending to be whole

A cancelled or failed download deletes the file it had begun writing. A failed upload aborts: the entry in .files is only created when the stream closes, so everything before that is chunks nothing points at, and the abort removes them.

#Orphaned chunks are reported, not tidied away

Chunks with no entry in .files are a real and common state — an interrupted upload, or someone deleting from the collections by hand.

TableCore finds them with a single aggregation on the server, because the chunks are the file and pulling them to the client would read the whole bucket. The report stops at 100 groups and says that it is truncated.

#Details worth knowing

• A file's _id need not be an ObjectId. GridFS does not require it, and another tool may have written whatever it liked, so TableCore reads, deletes and downloads through a bucket typed for any BSON value. Uploads generate an ObjectId because the driver's upload path does.
• contentType at the top level of the file document is deprecated in the GridFS specification. TableCore writes it into metadata and reads it from both places, so files uploaded by an older tool still have a type.