Continuous Evals in CI Pipeline (Stage 4: Test)

Continuous Evals in CI Pipeline (Stage 4: Test) A workflow diagram generated by Archify. 01 / CI Trigger & Config 02 / CI Runner Orchestrator 03 / Deterministic & Evals 04 / LLM Evals Grader 05 / Quality Gate Trigger & Dispatch Parallel Test & Eval Grading & Gate Commit / PR / Cron · CLAUDE.md & config diff · CI Trigger & Config › Trigger & Dispatch · change event Commit / PR / Cron CLAUDE.md & config diff change event CI Runner · headless Claude Code · CI Runner Orchestrator › Trigger & Dispatch · non-interactive CI Runner headless Claude Code non-interactive Deterministic Tests · make test / build / lint · Deterministic & Evals › Parallel Test & Eval · pass / fail Deterministic Tests make test / build / lint pass / fail Eval Suite · 20-50 curated tasks · Deterministic & Evals › Parallel Test & Eval · regression set Eval Suite 20-50 curated tasks regression set Incident Regressions · postmortem eval cases · Deterministic & Evals › Grading & Gate · permanent guard Incident Regressions postmortem eval cases permanent guard LLM Evals Grader · rubric & output check · LLM Evals Grader › Grading & Gate · scoring LLM Evals Grader rubric & output check scoring Quality Gate · pass-rate threshold check · Quality Gate › Grading & Gate · block merge Quality Gate pass-rate threshold check block merge dispatch run unit tests run eval matrix regression suite grade outputs deterministic green pass rate >= 95% Legend User UI Agent logic Policy Tool action

Continuous Evals Contract

  • • Kich hoat tu dong moi khi CLAUDE.md, skills, hoac hooks thay doi
  • • Danh gia hanh vi agent tren bo 20-50 task thuc te gan nhat

Regression Immunity

  • • Moi incident production duoc ma hoa thanh eval case vinh vien
  • • Neu ty le pass sut giam, Quality Gate se chan PR truoc khi merge