UltraBERT Fits 32K Tokens in One Vector. RIP Chunking?

Every retrieval pipeline starts with the same call: how big is a chunk? Too small, and a chunk loses its context. Too big, and the embedding model cuts it off. DMIS Lab’s UltraBERT-Embed-150M moves that cutoff to 32K tokens. It’s a 151M-parameter dense embedding model built on UltraBERT, an English encoder trained for 32K-token inputs. Most documents fit in one pass, and you get back one 768-dimension vector. So is chunking dead? We tested it in Qdrant: one vector per document against 512-token chunks, the two together, and both compressed.