The Problem
My Enterprise RAG API ran on a free Render instance (the host has since suspended it) with 512 MB of memory. For weeks it was effectively down, while the dashboards said the service was healthy. From outside, it looked like a very slow cold start, which is exactly what the README said to expect, so I did not question it.
It was not slow. It was crashing and restarting. I peeled the problem apart in layers, and each layer I fixed was only part of the answer.
Layer One: the Wrong torch
The Dockerfile installed the full local requirements file, which pins plain torch. The default Linux wheel on PyPI bundles the CUDA runtime and is 526.6 MB, on an instance with 512 MB of memory. A file with the right dependencies had existed for the deployment all along, but Render builds from the Dockerfile and ignores the build command, so nothing used it.
Pinning the CPU build brought the wheel down to 191.8 MB. That was still too heavy.
Layer Two: the Model on Every Wake
The embedding model was downloaded from the Hugging Face Hub every time the container woke up. That was the real reason for the long cold start. I baked the model into the image instead.
Layer Three: Loading Before Listening
The app loaded the model at import time, before the web server bound its port. On 512 MB the process was killed during the load, so the port never opened, and the platform saw a dead service and restarted it. That cycle looks just like a slow start from the outside.
Layer Four: a Dependency Nobody Asked For
Even with those fixes, it did not fit. Following the imports showed why: langchain_groq pulls in langchain_core, which does an optional import of transformers for token counting, and transformers imports torch. Nothing in my request path needed torch. It arrived through a library I had installed for something else.
If transformers is not installed, langchain_core skips that import and langchain_groq still works. I checked that by importing it inside the built image. For the embeddings I moved to fastembed, which runs the same all-MiniLM-L6-v2 weights through ONNX Runtime. The vectors stay 384-dimensional, so the existing Qdrant collection needed no re-indexing.
The Result
- Import time went from about two minutes to 1.87 seconds.
- Peak memory, with the model loaded and a real batch embedded, went from 438 MiB to 259 MiB of the 512.
- The health endpoint answers in 0.14 seconds.
Two Mistakes of My Own
Both showed up only after I deployed.
An adapter that was not quite an Embeddings class. I wrote the ONNX adapter without subclassing LangChain's Embeddings base class, on the reasoning that two methods are the whole interface. The Qdrant vector store type-checks it and refused it with an invalid embeddings type error. The service connected to Qdrant, created the collection, and only then failed to build the store. It came up degraded, every retrieval returned 503, and the startup log looked healthy. The import I had been avoiding was cheap all along, because the expensive import lives somewhere else in langchain_core.
Picking the backend by the wrong package. I first chose the embedding backend by checking whether langchain_huggingface could be imported. That package is installed in the image and torch is not, so the check passed, the torch path was chosen and the embedding call failed. It now checks for torch, because torch is the dependency that decides the answer.
What I Took From It
- A service can report healthy and still be unusable. I only trusted the fix after I ran a real question through the deployed page in a browser while it was up.
- When something does not fit in memory, find out what is importing what. The heavy dependency was a transitive one.
- Measure before and after. The two figures that matter here, 1.87 seconds and 259 MiB, came from running the built image.
The project is on GitHub.