Dec 8, 2023
AbanteAI is excited to announce Mentat Auto-Context.
Auto-Context matches your messages with the most relevant chunks of your codebase automatically. This means you can ask questions or give instructions about your entire codebase without including lots of unnecessary context.
So far we’ve found it really especially useful for:
Q&A over large codebases. Ask targeted questions without specifying where to look. “How do I add my API key?”, “How does my uploader handle PDFs?”
Reasoning across several files. Ask about interaction between functions, variables, etc. “Walk me through the ‘get_code_message’ process step-by-step.” “How do I coordinate access to my database?”
Semantic find-and-replace. Change all instances of something, each based on its unique circumstances. “Use GitPython instead of subprocess for anything to do with branches.” “Add docstrings to all FeatureFilters’ init functions”.
Mentat exists to make it faster and easier for developers to use AI, and we’re 100% open source. We started by automating the code-edit process: Mentat proposes specific edits to your code, asks you to accept them, and then makes the edits automatically. With the addition of auto-context, we automate the context-selection process as well. Just ask or instruct Mentat, and then review its proposals.
Mentat Auto-Context works similar to other retrieval-augmented generation (RAG) systems in that it:
Generates embeddings for all files in your working directory, in chunks, and stores them in a local database.
Generates an embedding for your message to ChatGPT and stores it.
Sorts all the (active versions of) files in your directory by similarity to your message.
Starting with the most-similar, select chunks that fit until the limit is reached. The default limit is 8000 tokens and can be set with the ‘--auto-tokens’ flag.
Add selected chunks to user query for generating completion.
Auto-Context uses OpenAI’s `text-embedding-ada-002` embeddings by default. We currently fetch them via the API and store them in a local database. Support for other embedding models is on our product wishlist, and of course Mentat is open-source so feel free to implement it yourself!
Mentat also includes semantic search powered by embeddings. When you use the search command, e.g. `/search “functions that return a list of floats”`, you’ll see the same sorted list of chunks that Auto-Context uses, along with its similarity score.