From fb2f8f5b004fe2f768e5daa6fc45d73512badb2a Mon Sep 17 00:00:00 2001 From: Michael Dieringer <65093775+MichaelDieringer@users.noreply.github.com> Date: Wed, 24 Jun 2026 00:06:42 +0200 Subject: [PATCH] Add CURABIS-MCP-008: AI eval scores must be posted to BC posting table --- ...-eval-scores-must-be-posted-to-bc-table.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 custom/knowledge/mcp/ai-eval-scores-must-be-posted-to-bc-table.md diff --git a/custom/knowledge/mcp/ai-eval-scores-must-be-posted-to-bc-table.md b/custom/knowledge/mcp/ai-eval-scores-must-be-posted-to-bc-table.md new file mode 100644 index 0000000..19ec687 --- /dev/null +++ b/custom/knowledge/mcp/ai-eval-scores-must-be-posted-to-bc-table.md @@ -0,0 +1,98 @@ +# CURABIS-MCP-008 — AI eval scores must be posted to the BC posting table + +## Rule + +When an AI agent completes a hill climbing eval iteration on a BC sub-task, all +resulting scores — compile result, test score, BCQuality score, F1 score, verdict, +and model identity — must be posted to the `CUR Project AI Score` table in Business +Central via the designated MCP tool (`bc_post_ai_score`). + +Scores must **not** be stored as: +- task comments +- local files or agent memory +- inline in agent files or knowledge files +- any other location outside the BC posting table + +## Why + +The `CUR Project AI Score` table is a **posting table**: one immutable entry per +iteration, with a clustered key on `Entry No.`. It is the single source of truth for +hill climbing history on a sub-task. + +Storing scores elsewhere breaks this guarantee: + +| Alternate location | Problem | +|---|---| +| Task comment | 250-char limit, unstructured, not queryable, mixed with human notes | +| Local file | Session-scoped, repo-specific, invisible to other agents and BC reporting | +| Agent memory | Volatile, not persisted between sessions | +| Hard-coded in agent file | Frozen at time of writing, violates CURABIS-MCP-007 pattern | + +The BC posting table enables: +1. Reporting across tasks and projects (MatchRate over time) +2. The Court reviewing objective score data from Edison +3. The orchestrator reading prior iterations via `bc_get_ai_scores` to decide verdict +4. BC users seeing hill climbing progress directly on the sub-task + +## Compliant + +After each eval iteration, the orchestrator calls: + +``` +bc_post_ai_score( + projectNo = "DEV2026-00010", + subTaskNo = "0014", + iterationNo = 3, + compile = true, + testScore = 0.80, + bcquality = 0.86, + f1Score = 0.83, + verdict = "Keep", + model = "claude-sonnet-4-6" +) +``` + +BC sets `Eval DateTime` automatically. The orchestrator may additionally post a +brief human-readable comment ("Iteration 3: F1=0.83 → Keep") — this is allowed, +as it communicates progress; the score itself is in BC. + +## Non-compliant + +``` +# Storing score as task comment only +bc_add_comment( + projectNo = "DEV2026-00010", + subTaskNo = "0014", + comment = "Iter 3: compile ✅ tests 4/5 BCQ 6/7 F1=0.83 Keep" +) +# → Score is unstructured text. Not queryable. Lost to reporting. +``` + +``` +# Storing score in agent file +## Hill climbing log +- Iteration 1: F1=0.43 Revert +- Iteration 2: F1=0.71 Keep +- Iteration 3: F1=0.83 Keep ← frozen, session-specific, wrong location +``` + +## False positive + +An agent that posts a human-readable summary comment **in addition to** calling +`bc_post_ai_score` is **not** violating this rule. The comment is human +communication; the score is in BC. Both are permitted. + +The violation is using the comment or any other location **instead of** posting to +the BC table. + +## API reference + +- Page: `CUR MCP Project AI Scores` (PAG6102906) +- Entity: `projectAIScores` +- Publisher: `curabis`, Group: `projectMgmt`, Version: `v2.0` +- Insert: allowed. Modify: never. Delete: never. +- `Eval DateTime` is set by BC `OnInsertRecord` — do not pass it. + +## Applies to + +Agent files that implement hill climbing eval loops on BC sub-tasks.