Q: How do I pick chunk size and overlap in RAG?

There's no universal number, but there's a reliable starting point.

Start with chunks of roughly 500-1,000 characters and overlap of 10-20%, split on paragraph boundaries. That's a sane default for most document Q&A.

Then understand the trade-off you're tuning. Small chunks give precise matching (each chunk is about one thing, so retrieval finds exactly the right passage) but fragmented context (the model receives crumbs, not explanations). Large chunks give complete context but diluted meaning: an embedding of a chunk covering five topics matches none of them well, and big chunks fill your context window and your bill faster. Overlap exists so that an idea sitting on a boundary between two chunks appears intact in at least one of them.