Ecosystem overview

Four projects that together cover remembering, running, observing and measuring an AI application.

dynavec started as a vector database, and it's growing into a full agent stack that runs inside your own cloud account. The plan has four parts. Each one answers a question about an AI application: what does it know, what does it do, what happened when it ran, and how good were the results?

The four projects

How they fit together

Each project feeds the next one, and all of them keep their data in the same account:

• dynaflow runs the agent. Its retriever and memory steps read from and write to dynavec, and the state of each run is saved in DynamoDB so it can be resumed.
• Every step records what it did. dynalogs puts those records together into one view of the run that you can search and drill into.
• dynaevals scores the same runs, shows the scores next to the trace, and can fail a CI build when a score goes down.
• What the scores tell you goes back into the agent: its prompts, its steps and its retrieval settings.

   dynaflow  --- retrieve / remember -->  dynavec
 (build & run)                          (remember)
       |
       |  traces, logs
       v
   dynalogs  --- runs, traces ------->  dynaevals
   (observe)                        (measure & improve)
       ^                                    |
       +------- improve the agent ----------+

What you can use today

Some of the newer pieces have already landed in the dynavec package.

Chat models (dynaflow)

One interface for OpenAI, Anthropic and Amazon Bedrock. With Bedrock, the model calls stay in your AWS account like the rest of dynavec.

from dynavec.chat import BedrockChatModel, Message

model = BedrockChatModel(region_name="us-east-1")   # or OpenAIChatModel()
result = model.invoke([
    Message(role="system", content="Answer in one sentence."),
    Message(role="user", content="What is a vector database?"),
])
print(result.message.content)

Every model also has stream(), ainvoke() and astream(), and accepts tools= for function calling.

Evaluation (dynaevals)

Check how well search finds the documents you expect:

from dynavec.eval import compute_mrr, compute_ndcg_at_k, compute_recall_at_k

retrieved = ["d3", "d1", "d7", "d2"]   # ids returned by search, best first
relevant = {"d1", "d2"}                # ids you know are correct

compute_recall_at_k(retrieved, relevant, k=3)   # 0.5
compute_mrr(retrieved, relevant)                # 0.5
compute_ndcg_at_k(retrieved, relevant, k=3)     # 0.3869

And use an LLM as a judge to check whether an answer sticks to the retrieved context and actually answers the question:

from dynavec.eval import OpenAIJudge, evaluate_rag

result = evaluate_rag(
    query="What powers the cell?",
    context=["The mitochondria is the powerhouse of the cell."],
    answer="The mitochondria.",
    judge=OpenAIJudge(model="gpt-4o-mini"),   # or BedrockJudge, GeminiJudge, CustomJudge
)
print(result.faithfulness.score, result.answer_relevance.score)

Telemetry (dynalogs)

Attach a TelemetryRecorder to your client to capture latency, cache hits and scores for every search, then view them in the telemetry dashboard.

Roadmap

The proposed order of work, grouped by area. Each row links to its issues.

Want to help? Pick an unassigned issue, comment on it to get assigned, and follow CONTRIBUTING.md.