bcquality/tools/New-AuthoringAssistComment.ps1
dayland 94d3d4358d Authoring-assist: promote to first-class tool with non-blocking PR advisory
Promotes the authoring-assist prototype to a first-class BCQuality feature that
reviews knowledge-article front-matter and proposes missing routing `signals:`
(raise triggers and `effect: suppress` suppressors). The suggestion is ADVISORY
ONLY: it never fails a build; authors apply it via a normal PR under existing
R29 + CI + CODEOWNERS review.

Feature:
- tools/Suggest-ArticleSignals.ps1: suggestion engine hardened with a stable JSON
  contract (schemaVersion/toolVersion) and a deterministic, effect-sensitive
  per-proposal suggestionId, plus -ChangedFiles scoping for PR-diff runs. Runs
  self-contained: the routing seed is now OPTIONAL (built-in domain map), so this
  does not depend on the separate routing-index tuning thread.
- tools/New-AuthoringAssistComment.ps1: renders one upsertable, feedback-instrumented
  advisory comment (stable anchor + aa:meta/aa:article markers carrying suggestionIds
  + reaction footer) for the downstream acceptance-measurement work.
- .github/workflows/authoring-assist*.yml: unprivileged pull_request intake +
  trusted workflow_run runner (review/publish split) + in-repo self-test.
- tools/Test-SuggestArticleSignals.ps1: 23-check smoke test (contract, determinism,
  scoping, suppressor/prohibition, renderer markers, seed-free graceful degradation).

Dormant schema support:
- .github/scripts/validate_frontmatter.py: adds R29, validating the optional
  `signals` block shape (bare token or mapping with token + optional
  pattern/domain/effect in {raise,suppress}). Reserved & dormant — no production
  pipeline consumes it; the existing PR Reviewer only reads `effect: suppress`
  behind its BCQ_INDEX_V2 flag and falls back safely when absent. Existing rules
  (incl. R24 skill-id uniqueness, R27, R28) are unchanged.

Excludes the routing-index tuning thread entirely (separate work).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 75d852d6-18a6-4c58-986f-ed5ab16618fa
2026-07-24 10:03:29 +01:00

168 lines
7 KiB
PowerShell

<#
.SYNOPSIS
Renders the JSON output of Suggest-ArticleSignals.ps1 into a single,
non-blocking Markdown PR advisory comment with embedded feedback metadata.
.DESCRIPTION
The automated authoring-assist advisory posts ONE consolidated comment per
PR (update-in-place) built from the per-article suggestions. Each article
section carries HTML metadata markers so downstream feedback tooling (the
BC-ALAgentsInternal harvester) can correlate reactions and applied/not-applied
outcomes back to a specific suggestion via its stable `suggestionId`.
The comment is ADVISORY ONLY: it never fails a build, it only suggests
front-matter an author eyeballs and applies via a normal PR. R29
(validate_frontmatter.py), CI, and CODEOWNERS remain the gate.
Markers emitted (stable contract — do not change without bumping the tool
schema and updating the harvester):
<!-- authoring-assist-advisory --> top anchor (find/update)
<!-- aa:meta schema="X" tool="Y" generated="Z" articles="N" -->
<!-- aa:article path="..." domain="..." effect="raise|suppress"
suppressor="true|false" suggestions="id1,id2,..." -->
.PARAMETER JsonPath
Path to a file containing the `Suggest-ArticleSignals.ps1 -AsJson` output.
If omitted, JSON is read from stdin.
.PARAMETER RepoUrl
Base repo URL used to build article permalinks (e.g.
https://github.com/microsoft/BCQuality). Optional; when omitted, article
paths are rendered as inline code without links.
.PARAMETER Ref
Git ref/SHA the suggestions were computed against, used in permalinks and
surfaced in the footer for provenance. Optional.
.PARAMETER DocUrl
URL surfaced in the feedback footer ("reply with why"). Optional.
.OUTPUTS
Writes the Markdown comment body to stdout.
#>
[CmdletBinding()]
param(
[string] $JsonPath,
[string] $RepoUrl,
[string] $Ref,
[string] $DocUrl
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$Anchor = '<!-- authoring-assist-advisory -->'
if ($JsonPath) {
if (-not (Test-Path -LiteralPath $JsonPath)) { throw "JSON file not found: $JsonPath" }
$raw = Get-Content -LiteralPath $JsonPath -Raw
} else {
$raw = [Console]::In.ReadToEnd()
}
if ([string]::IsNullOrWhiteSpace($raw)) { throw 'No JSON input provided (empty file/stdin).' }
$data = $raw | ConvertFrom-Json
$reports = @()
if ($data.PSObject.Properties.Name -contains 'reports' -and $data.reports) {
$reports = @($data.reports)
}
function Format-Provenance {
param([object] $Proposal)
$tags = [System.Collections.Generic.List[string]]::new()
if ($Proposal.keywordBacked) { $tags.Add('keyword-backed') | Out-Null }
if ($Proposal.inBad) { $tags.Add('anti-pattern sample') | Out-Null }
if ($Proposal.proseOnly) { $tags.Add('prose-only') | Out-Null }
if ($tags.Count -eq 0) { $tags.Add('sample') | Out-Null }
return ($tags -join ', ')
}
function Format-SignalsBlock {
param([object] $Report)
$lines = [System.Collections.Generic.List[string]]::new()
$lines.Add('signals:') | Out-Null
foreach ($p in $Report.proposals) {
if ($Report.effect -eq 'suppress') {
$lines.Add(" - token: $($p.token)") | Out-Null
$lines.Add(' effect: suppress') | Out-Null
} else {
$lines.Add(" - $($p.token)") | Out-Null
}
}
return ($lines -join "`n")
}
$body = [System.Collections.Generic.List[string]]::new()
$body.Add($Anchor) | Out-Null
$schema = if ($data.PSObject.Properties.Name -contains 'schemaVersion') { $data.schemaVersion } else { 'unknown' }
$toolVer = if ($data.PSObject.Properties.Name -contains 'toolVersion') { $data.toolVersion } else { 'unknown' }
$generated = if ($data.PSObject.Properties.Name -contains 'generatedAt') { $data.generatedAt } else { '' }
$body.Add(('<!-- aa:meta schema="{0}" tool="{1}" generated="{2}" articles="{3}" -->' -f $schema, $toolVer, $generated, @($reports).Count)) | Out-Null
$body.Add('') | Out-Null
$body.Add('## 🧭 Authoring-assist advisory') | Out-Null
$body.Add('') | Out-Null
if (@($reports).Count -eq 0) {
$body.Add('No routing head-matter suggestions for the knowledge articles changed in this PR. ✅') | Out-Null
$body.Add('') | Out-Null
$body.Add('<sub>Advisory only — never blocks a merge. Generated by `tools/Suggest-ArticleSignals.ps1`.</sub>') | Out-Null
return ($body -join "`n")
}
$body.Add(("This PR changes knowledge articles whose routing head-matter could be improved. " +
"These are **non-blocking suggestions** — paste the proposed ``signals:`` block into the " +
"article's front-matter if it looks right. R29 + CODEOWNERS still review the change.")) | Out-Null
$body.Add('') | Out-Null
foreach ($r in ($reports | Sort-Object domain, path)) {
$effect = $r.effect
$badge = if ($r.suppressor) { '🛡️ suppressor' } else { '🎯 trigger' }
$suggestionIds = @($r.proposals | ForEach-Object { $_.suggestionId }) -join ','
$body.Add(('<!-- aa:article path="{0}" domain="{1}" effect="{2}" suppressor="{3}" suggestions="{4}" -->' -f `
$r.path, $r.domain, $effect, ([string]$r.suppressor).ToLowerInvariant(), $suggestionIds)) | Out-Null
$title = if ($RepoUrl -and $Ref) { "[``$($r.path)``]($RepoUrl/blob/$Ref/$($r.path))" } else { "``$($r.path)``" }
$body.Add("### $title") | Out-Null
$body.Add("**Domain:** ``$($r.domain)`` &nbsp;·&nbsp; $badge &nbsp;·&nbsp; ``effect: $effect``") | Out-Null
$body.Add('') | Out-Null
if (@($r.proposals).Count -gt 0) {
$body.Add('Proposed front-matter:') | Out-Null
$body.Add('') | Out-Null
$body.Add('```yaml') | Out-Null
$body.Add((Format-SignalsBlock -Report $r)) | Out-Null
$body.Add('```') | Out-Null
$body.Add('') | Out-Null
foreach ($p in $r.proposals) {
$body.Add(("- ``$($p.token)`` — {0} · {1}" -f (Format-Provenance -Proposal $p), $p.note)) | Out-Null
}
$body.Add('') | Out-Null
}
if ($r.mismatch) {
$body.Add("> ⚠️ **Domain check:** $($r.mismatch)") | Out-Null
$body.Add('') | Out-Null
}
if ($r.applicability) {
$body.Add("> ⚠️ **Applicability:** $($r.applicability)") | Out-Null
$body.Add('') | Out-Null
}
if (@($r.keywordSuggestions).Count -gt 0) {
$kw = @($r.keywordSuggestions | ForEach-Object { '`' + $_ + '`' }) -join ', '
$body.Add("> 💡 **Keywords:** consider adding $kw") | Out-Null
$body.Add('') | Out-Null
}
}
$replyLink = if ($DocUrl) { " - <a href=`"$DocUrl`">reply with why</a>" } else { '' }
$provenance = if ($Ref) { " against ``$Ref``" } else { '' }
$body.Add('---') | Out-Null
$body.Add("<sub>👍 useful · ❤️ especially valuable · 👎 wrong$replyLink</sub>") | Out-Null
$body.Add("<sub>Advisory only — never blocks a merge. Generated by ``tools/Suggest-ArticleSignals.ps1``$provenance. Suggestions carry stable ids for feedback; applying them is optional and human-reviewed.</sub>") | Out-Null
return ($body -join "`n")