{"componentChunkName":"component---src-templates-project-template-js","path":"/projects/aqa-system-of-record-for-agentic-verification","result":{"data":{"project":{"title":"AQA — System of Record for Agentic Verification","slug":"aqa-system-of-record-for-agentic-verification","description":"A TestLink-style test-management and verification-memory platform built for the agentic coding loop, exposed over MCP (43 tools). It tracks plans, cases, dependencies, run history, and requirements-to-test traceability, with a claim/verify protocol (doer ≠ checker) and semantic regression memory: a failed run's root-cause reasoning is stored and recalled when a similar failure recurs, so agents get the prior diagnosis instead of re-deriving it. One service layer, surfaced three ways — REST API, MCP server, and CLI.","caseStudy":"## The problem\n\nWhen coding agents write and run tests, the verification itself becomes throwaway. Green CI\ndoesn't mean reliable software: a failure gets diagnosed, fixed, and the reasoning evaporates —\nso the next time a similar failure appears, an agent re-derives the root cause from scratch.\nAgents also tend to mark their own homework, and a flaky or cascading failure can hide a real\nregression behind a false green. AQA is built to make verification **institutional** — a\ndurable, attributable, queryable record of what was tested, what failed, and why.\n\n## The approach\n\nAQA is a TestLink-style test-management system designed for the agentic coding loop, reachable\nby agents over **MCP** (not just humans in a dashboard). It sits one layer above the test\nrunner and tracks plans, cases, dependencies, run history, and requirements-to-test\ntraceability. Its distinguishing feature is **verification memory**: a failed run's root-cause\nreasoning is stored and semantically recalled, so a recurrence comes back with its earlier\ndiagnosis instead of being re-investigated.\n\n## Architecture\n\nOne transport-agnostic service layer is exposed three ways — a **REST API** (FastAPI), an\n**MCP server** (FastMCP, 43 tools), and a **CLI** (Typer) — so humans and agents operate on the\nsame state. Data lives in **PostgreSQL with pgvector**; failure root-cause prose is embedded\nlocally with `all-MiniLM-L6-v2` for semantic similarity search, and artifacts (traces, logs,\nscreenshots) go to S3-compatible blob storage.\n\nFour capabilities define it:\n\n- **Regression memory** — `search_similar_failures` / `get_known_regressions` recall prior\n  diagnoses and cached fix-paths, and log the re-investigations they avoid.\n- **Blind-spot radar** — requirements → coverage links surface tests that don't exist yet.\n- **Doer ≠ checker** — a claim/verify protocol with identity enforcement, so work isn't\n  self-certified.\n- **Dependency gating** — a dependency-aware run manifest and cascade-blocking that kills\n  false-green CI when an upstream case fails.\n\n## A note on rigor\n\nAQA dogfoods itself: a Stop hook blocks \"done\" while coverage gaps are open, and CI records its\nown runs. It raises the reliability *floor* — no silent regressions, no invisible blind spots,\nno self-graded work — a ratchet that compounds across commits and sessions.\n","gallery":[],"date_start":"2026","date_end":null,"hours":null,"client":null,"tags":["ai","mcp","testing","devtools"],"outcomes":["43 MCP tools spanning authoring, planning, execution, builds/CI, and failure memory","Semantic failure search via pgvector + local MiniLM embeddings","Cascade-blocking merge gate that kills false-green CI"],"tech_stack":["Python","FastAPI","FastMCP","PostgreSQL","pgvector","MCP"],"links":[{"label":"GitHub","url":"https://github.com/nt1412/aqa"}],"image":{"childImageSharp":{"fluid":{"tracedSVG":"data:image/svg+xml,%3csvg%20xmlns='http://www.w3.org/2000/svg'%20width='400'%20height='229'%20viewBox='0%200%20400%20229'%20preserveAspectRatio='none'%3e%3cpath%20d='M0%20115v114h401V0H0v115m157-64c0%202%203%202%2026%202l26-1c-1-2-52-3-52-1m0%2015l26%201%2026-1-26-1-26%201m0%2015l24%201%2026-1c5-1%200-2-24-2-22%200-26%200-26%202m0%2014c0%202%2051%201%2052-1l-26-1c-23%200-26%200-26%202m0%2014l26%201c24%200%2032-1%2021-2-10-2-47-1-47%201m0%2014l26%201c24%200%2032-1%2021-2-10-2-47-1-47%201m0%2015l16%201a806%20806%200%200111-3c-25%200-27%200-27%202m0%2013c0%202%203%202%2026%202l27-1c0-1-10-2-27-2l-26%201'%20fill='%23d3d3d3'%20fill-rule='evenodd'/%3e%3c/svg%3e","aspectRatio":1.7543859649122806,"src":"/static/ca23aab0a7634bcdabeb461580f26f8d/ee604/aqa.png","srcSet":"/static/ca23aab0a7634bcdabeb461580f26f8d/69585/aqa.png 200w,\n/static/ca23aab0a7634bcdabeb461580f26f8d/497c6/aqa.png 400w,\n/static/ca23aab0a7634bcdabeb461580f26f8d/ee604/aqa.png 800w","srcWebp":"/static/ca23aab0a7634bcdabeb461580f26f8d/58556/aqa.webp","srcSetWebp":"/static/ca23aab0a7634bcdabeb461580f26f8d/61e93/aqa.webp 200w,\n/static/ca23aab0a7634bcdabeb461580f26f8d/1f5c5/aqa.webp 400w,\n/static/ca23aab0a7634bcdabeb461580f26f8d/58556/aqa.webp 800w","sizes":"(max-width: 800px) 100vw, 800px"}}},"stack_icons":[{"name":"Python","icon":{"childImageSharp":{"fixed":{"tracedSVG":"data:image/svg+xml,%3csvg%20xmlns='http://www.w3.org/2000/svg'%20width='24'%20height='24'%20viewBox='0%200%2024%2024'%20preserveAspectRatio='none'%3e%3cpath%20d='M1%201v22c2%202%2020%201%2022-1s3-22%201-21H1'%20fill='%23d3d3d3'%20fill-rule='evenodd'/%3e%3c/svg%3e","width":24,"height":24,"src":"/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/6d1ba/python.png","srcSet":"/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/6d1ba/python.png 1x,\n/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/a9c35/python.png 1.5x,\n/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/559c9/python.png 2x","srcWebp":"/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/f8bad/python.webp","srcSetWebp":"/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/f8bad/python.webp 1x,\n/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/f81b6/python.webp 1.5x,\n/static/64d0f7b1b208f14bd8dd5134b3ed7ff5/804d1/python.webp 2x"}}}},{"name":"API","icon":{"childImageSharp":{"fixed":{"tracedSVG":"data:image/svg+xml,%3csvg%20xmlns='http://www.w3.org/2000/svg'%20width='24'%20height='24'%20viewBox='0%200%2024%2024'%20preserveAspectRatio='none'%3e%3cpath%20d='M1%201v22c2%202%2020%201%2022-1s3-22%201-21H1'%20fill='%23d3d3d3'%20fill-rule='evenodd'/%3e%3c/svg%3e","width":24,"height":24,"src":"/static/fb76c0da90f90c016ad01e4dc810443f/6d1ba/api.png","srcSet":"/static/fb76c0da90f90c016ad01e4dc810443f/6d1ba/api.png 1x,\n/static/fb76c0da90f90c016ad01e4dc810443f/a9c35/api.png 1.5x,\n/static/fb76c0da90f90c016ad01e4dc810443f/559c9/api.png 2x","srcWebp":"/static/fb76c0da90f90c016ad01e4dc810443f/f8bad/api.webp","srcSetWebp":"/static/fb76c0da90f90c016ad01e4dc810443f/f8bad/api.webp 1x,\n/static/fb76c0da90f90c016ad01e4dc810443f/f81b6/api.webp 1.5x,\n/static/fb76c0da90f90c016ad01e4dc810443f/804d1/api.webp 2x"}}}},{"name":"Embeddings","icon":{"childImageSharp":{"fixed":{"tracedSVG":"data:image/svg+xml,%3csvg%20xmlns='http://www.w3.org/2000/svg'%20width='24'%20height='24'%20viewBox='0%200%2024%2024'%20preserveAspectRatio='none'%3e%3cpath%20d='M1%201v22c2%202%2020%201%2022-1s3-22%201-21H1'%20fill='%23d3d3d3'%20fill-rule='evenodd'/%3e%3c/svg%3e","width":24,"height":24,"src":"/static/b0940db6930f1b27d8451c404e8a5e5c/6d1ba/embeddings.png","srcSet":"/static/b0940db6930f1b27d8451c404e8a5e5c/6d1ba/embeddings.png 1x,\n/static/b0940db6930f1b27d8451c404e8a5e5c/a9c35/embeddings.png 1.5x,\n/static/b0940db6930f1b27d8451c404e8a5e5c/559c9/embeddings.png 2x","srcWebp":"/static/b0940db6930f1b27d8451c404e8a5e5c/f8bad/embeddings.webp","srcSetWebp":"/static/b0940db6930f1b27d8451c404e8a5e5c/f8bad/embeddings.webp 1x,\n/static/b0940db6930f1b27d8451c404e8a5e5c/f81b6/embeddings.webp 1.5x,\n/static/b0940db6930f1b27d8451c404e8a5e5c/804d1/embeddings.webp 2x"}}}}]}},"pageContext":{"slug":"aqa-system-of-record-for-agentic-verification"}},"staticQueryHashes":["3724428426"]}