mirror of
https://github.com/evosoftie/AILint.git
synced 2026-08-06 04:56:51 +01:00
Add AILint Integrity Heuristics documentation
This document outlines integrity heuristics for maintaining provenance in AI-generated content, including drift detection, missing declaration nudges, and review prioritization.
This commit is contained in:
parent
ecad9e429e
commit
4b87a02223
1 changed files with 125 additions and 0 deletions
125
spec/intergrity.md
Normal file
125
spec/intergrity.md
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
# AILint Integrity Heuristics (v1.0)
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Integrity heuristics help maintain accurate provenance over time by:
|
||||
- Detecting drift between declared provenance and current content
|
||||
- Suggesting missing declarations (non-coercively)
|
||||
- Flagging areas for review prioritization
|
||||
|
||||
**Critical distinction:**
|
||||
- These are **advisory tools**, not detection systems
|
||||
- They **suggest**, not enforce
|
||||
- They **maintain hygiene**, not catch violations
|
||||
|
||||
## 2. Scope
|
||||
|
||||
Integrity heuristics operate when:
|
||||
- Explicit provenance exists (checking accuracy)
|
||||
- Content matches AI patterns (suggesting declaration)
|
||||
- Risk factors combine (prioritizing review)
|
||||
|
||||
They do NOT:
|
||||
- Override explicit declarations
|
||||
- Make definitive claims
|
||||
- Enable surveillance
|
||||
|
||||
## 3. Heuristic Types
|
||||
|
||||
### 3.1 Provenance Drift Detection
|
||||
|
||||
**Purpose:** Alert when content has materially changed since provenance was declared
|
||||
|
||||
**Example:**File: auth.py
|
||||
Last provenance: role=generated (commit abc123, 3 weeks ago)
|
||||
Current state: 45% edit distance from that version
|
||||
Advisory: "Content has diverged significantly; consider updating role to 'edited'"
|
||||
|
||||
**Implementation:**
|
||||
- Compare `content_hash` from last provenance to current hash
|
||||
- Calculate edit distance if mismatch
|
||||
- Threshold: >40% change suggests role update
|
||||
|
||||
**Output:**
|
||||
- Non-blocking advisory
|
||||
- Suggests action, doesn't require it
|
||||
- Can be dismissed
|
||||
|
||||
### 3.2 Missing Declaration Nudge
|
||||
|
||||
**Purpose:** Suggest provenance when content has AI-typical patterns but lacks metadata
|
||||
|
||||
**Critical constraint:**
|
||||
This MUST NEVER claim "this is AI" - only suggest "if AI was used, consider declaring"
|
||||
|
||||
**Example:**File: utils.ts
|
||||
Provenance: None
|
||||
Pattern confidence: 0.82
|
||||
Advisory: "This content matches patterns commonly produced by AI systems.
|
||||
If AI assistance was used, consider adding provenance."
|
||||
|
||||
**UX Requirements:**
|
||||
- Non-intrusive (tooltip, not modal)
|
||||
- "Remind me later" option
|
||||
- One-click template to add declaration
|
||||
- Permanently dismissible per-file
|
||||
|
||||
### 3.3 Risk-Aware Review Prioritization
|
||||
|
||||
**Purpose:** Flag combinations of AI provenance + risk factors for elevated review
|
||||
|
||||
**Example:**File: payment.py
|
||||
Provenance: role=generated, tool=gpt-4
|
||||
Risk factors:
|
||||
|
||||
Security-sensitive context
|
||||
Introduces crypto library (hallucination-prone domain)
|
||||
No test coverage
|
||||
Advisory: "AI-generated code in high-consequence area; prioritize review"
|
||||
|
||||
|
||||
**Risk factor categories:**
|
||||
- Domain-specific (crypto, date/time, regex, SQL)
|
||||
- Context-sensitive (security, financial, PII)
|
||||
- Coverage gaps (no tests, no peer review)
|
||||
|
||||
## 4. Configuration
|
||||
|
||||
All heuristics MUST be configurable:
|
||||
```json{
|
||||
"drift_detection": {
|
||||
"enabled": true,
|
||||
"threshold": 0.4,
|
||||
"prompt_style": "advisory" // or "warning" or "silent"
|
||||
},
|
||||
"missing_declaration_nudge": {
|
||||
"enabled": true,
|
||||
"confidence_threshold": 0.7,
|
||||
"dismissible": true
|
||||
},
|
||||
"review_prioritization": {
|
||||
"enabled": true,
|
||||
"risk_domains": ["crypto", "security", "financial"],
|
||||
"require_tests": true
|
||||
}
|
||||
}
|
||||
|
||||
## 5. Non-Goals
|
||||
|
||||
These heuristics are NOT:
|
||||
- Detection of undeclared AI usage (that's inference, not integrity)
|
||||
- Quality assessment (we don't judge code quality)
|
||||
- Performance monitoring (no developer-level metrics)
|
||||
- Enforcement (no blocking actions)
|
||||
|
||||
## 6. Precedence
|
||||
|
||||
Integrity heuristics ALWAYS defer to explicit provenance:
|
||||
- If valid provenance exists → heuristics check accuracy, not truth
|
||||
- If no provenance exists → heuristics suggest, not claim
|
||||
|
||||
## 7. Summary
|
||||
|
||||
Integrity heuristics maintain hygiene, not compliance.
|
||||
Advisory, not adversarial.
|
||||
Suggestions, not suspicions.
|
||||
Loading…
Add table
Add a link
Reference in a new issue