mirror of
https://github.com/evosoftie/AILint.git
synced 2026-08-06 04:56:51 +01:00
Add CLAUDE.md for collaboration guidelines
This document provides guidelines for collaborating with Claude on the AILint project, emphasizing privacy, detection methodologies, and the importance of standards in AI-assisted development.
This commit is contained in:
parent
304943a467
commit
3a6d912fbd
1 changed files with 261 additions and 0 deletions
261
CLAUDE.md
Normal file
261
CLAUDE.md
Normal file
|
|
@ -0,0 +1,261 @@
|
||||||
|
# Working with Claude on AILint
|
||||||
|
|
||||||
|
This document explains how to effectively work with Claude (Anthropic's AI assistant) when contributing to or extending AILint. It serves as both a guide for human contributors and context for Claude itself.
|
||||||
|
|
||||||
|
## Project Context
|
||||||
|
|
||||||
|
AILint addresses a critical infrastructure gap in AI-assisted development: **code provenance tracking**. As AI coding assistants become ubiquitous, we're creating a feedback loop where:
|
||||||
|
|
||||||
|
1. AI generates code
|
||||||
|
2. That code gets committed to repositories
|
||||||
|
3. Future AI models train on that code
|
||||||
|
4. Quality degrades (model collapse)
|
||||||
|
|
||||||
|
This is the "Irish motorway problem" - building infrastructure without planning for future needs, then having to retrofit at 10x the cost.
|
||||||
|
|
||||||
|
## Core Philosophy
|
||||||
|
|
||||||
|
When working on AILint, remember:
|
||||||
|
|
||||||
|
### 1. Privacy First
|
||||||
|
We're building a transparency tool, not a surveillance system. Every design decision must consider:
|
||||||
|
- Can this be abused for employee monitoring?
|
||||||
|
- Does data stay local by default?
|
||||||
|
- Can developers opt out where appropriate?
|
||||||
|
|
||||||
|
### 2. Forensic Over Declarative
|
||||||
|
We **cannot** rely on developers honestly declaring AI use because:
|
||||||
|
- Imposter syndrome / job security fears
|
||||||
|
- Performance review concerns
|
||||||
|
- Simply forgetting what was AI-assisted
|
||||||
|
- Competitive pressures
|
||||||
|
|
||||||
|
Therefore: detect rather than ask, classify as "potentially AI influenced" rather than making definitive claims.
|
||||||
|
|
||||||
|
### 3. Standards Over Solutions
|
||||||
|
The goal isn't just "a tool that works" - it's creating infrastructure that becomes industry standard. Design for:
|
||||||
|
- Interoperability with other tools
|
||||||
|
- Extensibility for future detection methods
|
||||||
|
- Clear metadata formats others can implement
|
||||||
|
- Open standards over proprietary formats
|
||||||
|
|
||||||
|
## Technical Architecture
|
||||||
|
|
||||||
|
### Detection Pipeline
|
||||||
|
```
|
||||||
|
Editor Events → Local Analysis → Metadata → Git Storage
|
||||||
|
↓ ↓ ↓ ↓
|
||||||
|
(typing, (heuristics, (JSON, (notes,
|
||||||
|
paste, patterns, scores, attributes,
|
||||||
|
timing) ML models) metadata) hooks)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key Constraint**: No code content leaves the local machine during analysis.
|
||||||
|
|
||||||
|
### Heuristic Categories
|
||||||
|
|
||||||
|
When Claude helps develop detection algorithms, focus on:
|
||||||
|
|
||||||
|
1. **Behavioral Patterns**
|
||||||
|
- Typing velocity anomalies
|
||||||
|
- Large paste events
|
||||||
|
- Commit timing (3am perfect commits are suspicious)
|
||||||
|
- Edit patterns (AI tends toward complete rewrites)
|
||||||
|
|
||||||
|
2. **Code Fingerprints**
|
||||||
|
- Comment style and density
|
||||||
|
- Variable naming conventions
|
||||||
|
- Documentation completeness
|
||||||
|
- Error handling patterns
|
||||||
|
- Boilerplate structure
|
||||||
|
|
||||||
|
3. **Metadata Correlation**
|
||||||
|
- Known AI tool telemetry
|
||||||
|
- Cross-reference with training data patterns
|
||||||
|
- Git blame granularity
|
||||||
|
- Commit message linguistic analysis
|
||||||
|
|
||||||
|
### Marker Implementation
|
||||||
|
|
||||||
|
For embedded markers, the design principles:
|
||||||
|
|
||||||
|
**Unicode Steganography:**
|
||||||
|
```python
|
||||||
|
# Example: Embed model metadata in zero-width characters
|
||||||
|
def embed_marker(code: str, metadata: dict) -> str:
|
||||||
|
"""
|
||||||
|
Insert zero-width Unicode characters encoding:
|
||||||
|
- Model type (GPT-4, Claude, Copilot, etc.)
|
||||||
|
- Timestamp
|
||||||
|
- Confidence score
|
||||||
|
"""
|
||||||
|
marker = encode_to_zwc(metadata)
|
||||||
|
# Insert after first comment or at file start
|
||||||
|
return inject_marker(code, marker)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Requirements:**
|
||||||
|
- Non-breaking: must not affect code execution
|
||||||
|
- Parseable: tools can extract metadata
|
||||||
|
- Removable: developers can strip if needed (but default is preserve)
|
||||||
|
- Format-agnostic: works across languages
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
### False Positives
|
||||||
|
**Problem**: Experienced developers writing clean code quickly look like AI.
|
||||||
|
|
||||||
|
**Mitigation**:
|
||||||
|
- Multi-factor analysis (never rely on single indicator)
|
||||||
|
- Confidence scores rather than binary classification
|
||||||
|
- Calibration period to learn developer baseline
|
||||||
|
- Whitelisting for known-good patterns
|
||||||
|
|
||||||
|
### Evasion
|
||||||
|
**Problem**: Developers deliberately writing messier code to avoid detection.
|
||||||
|
|
||||||
|
**Mitigation**:
|
||||||
|
- Make evasion require more effort than honest marking
|
||||||
|
- Focus on hard-to-fake patterns (structural, not stylistic)
|
||||||
|
- Audit trail even for evasion attempts
|
||||||
|
- Culture shift: marking becomes expected, like signing commits
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
**Problem**: Analysis can't slow down commits significantly.
|
||||||
|
|
||||||
|
**Mitigation**:
|
||||||
|
- Async analysis where possible
|
||||||
|
- Caching of results
|
||||||
|
- Progressive detail (quick scan, deep analysis on demand)
|
||||||
|
- Configurable depth vs. speed tradeoff
|
||||||
|
|
||||||
|
## Prompting Claude for AILint Work
|
||||||
|
|
||||||
|
### Effective Patterns
|
||||||
|
|
||||||
|
**For Detection Algorithms:**
|
||||||
|
```
|
||||||
|
"Design a heuristic to detect [specific AI pattern] in [language] code.
|
||||||
|
Requirements:
|
||||||
|
- Low false positive rate (<5%)
|
||||||
|
- Computationally cheap (must run on commit)
|
||||||
|
- Resistant to simple evasion
|
||||||
|
Consider: [context about how AI generates this pattern]"
|
||||||
|
```
|
||||||
|
|
||||||
|
**For Integration Code:**
|
||||||
|
```
|
||||||
|
"Implement a [VS Code/Git/Azure DevOps] integration that:
|
||||||
|
1. Hooks into [specific event]
|
||||||
|
2. Analyzes [specific aspect]
|
||||||
|
3. Stores results as [format]
|
||||||
|
Privacy constraint: no code content leaves local machine"
|
||||||
|
```
|
||||||
|
|
||||||
|
**For Standards Design:**
|
||||||
|
```
|
||||||
|
"Design a metadata format for AI provenance that:
|
||||||
|
- Works across version control systems
|
||||||
|
- Doesn't break existing tools
|
||||||
|
- Allows future extension
|
||||||
|
- Can be implemented by other tools
|
||||||
|
Provide JSON schema and examples"
|
||||||
|
```
|
||||||
|
|
||||||
|
### What Claude Knows About This Project
|
||||||
|
|
||||||
|
Claude has context about:
|
||||||
|
- The motorway analogy and why timing matters
|
||||||
|
- Detection vs. declaration approach
|
||||||
|
- Privacy-first architecture
|
||||||
|
- Multi-platform goals (Microsoft stack initially, broader later)
|
||||||
|
- Open source strategy for standard creation
|
||||||
|
|
||||||
|
Claude should reference this context when helping, rather than explaining it back.
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
### When Adding Features
|
||||||
|
|
||||||
|
1. **Check privacy implications**: Does this require data to leave the machine?
|
||||||
|
2. **Consider evasion**: Can developers easily bypass this?
|
||||||
|
3. **Test false positive rate**: Run against known-human code
|
||||||
|
4. **Document heuristic**: Why does this pattern indicate AI?
|
||||||
|
5. **Make configurable**: Different teams have different risk tolerance
|
||||||
|
|
||||||
|
### When Reviewing PRs
|
||||||
|
|
||||||
|
Key questions:
|
||||||
|
- Does this maintain privacy guarantees?
|
||||||
|
- Could this be used for employee surveillance?
|
||||||
|
- Is the detection logic explainable?
|
||||||
|
- Does it integrate with existing tools?
|
||||||
|
- Is the metadata format documented?
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
### Ground Truth Datasets
|
||||||
|
|
||||||
|
We need labeled data:
|
||||||
|
- **Known human code**: commits from pre-AI era (before 2020)
|
||||||
|
- **Known AI code**: synthetic test sets from various models
|
||||||
|
- **Hybrid code**: realistic human + AI collaboration
|
||||||
|
|
||||||
|
### Validation Metrics
|
||||||
|
|
||||||
|
- **Precision**: Of flagged commits, how many were actually AI-assisted?
|
||||||
|
- **Recall**: Of AI-assisted commits, how many did we catch?
|
||||||
|
- **F1 Score**: Balance of both
|
||||||
|
- **Adversarial robustness**: How well do we handle evasion attempts?
|
||||||
|
|
||||||
|
## Future Directions
|
||||||
|
|
||||||
|
Areas where Claude can help:
|
||||||
|
|
||||||
|
### Machine Learning Integration
|
||||||
|
As we accumulate data, we can train models to:
|
||||||
|
- Identify new AI signature patterns
|
||||||
|
- Distinguish between different AI tools
|
||||||
|
- Predict likelihood scores more accurately
|
||||||
|
|
||||||
|
Privacy constraint: training must happen on anonymized aggregated patterns, never on raw code.
|
||||||
|
|
||||||
|
### Standards Advocacy
|
||||||
|
Help draft:
|
||||||
|
- IETF RFC for AI provenance metadata
|
||||||
|
- W3C standard for embedded markers
|
||||||
|
- Industry best practices documentation
|
||||||
|
|
||||||
|
### Ecosystem Integration
|
||||||
|
Design APIs for:
|
||||||
|
- Code review tools (prioritize high-AI-score commits)
|
||||||
|
- Security scanners (flag supply chain risks)
|
||||||
|
- Compliance systems (generate audit reports)
|
||||||
|
|
||||||
|
## Questions to Ask Claude
|
||||||
|
|
||||||
|
When stuck or designing new features:
|
||||||
|
|
||||||
|
1. "How could this be abused for surveillance, and how do we prevent it?"
|
||||||
|
2. "What's the simplest implementation that still works?"
|
||||||
|
3. "How would a motivated developer evade this, and how do we make that harder?"
|
||||||
|
4. "What existing standards or tools can we build on?"
|
||||||
|
5. "How does this help the 'Bronze Age transition' for AI tooling?"
|
||||||
|
|
||||||
|
## Contributing Context
|
||||||
|
|
||||||
|
When Claude helps write documentation or code for AILint, it should:
|
||||||
|
- Assume reader familiarity with the core problem
|
||||||
|
- Focus on implementation details, not philosophical justification
|
||||||
|
- Prioritize privacy and interoperability
|
||||||
|
- Use concrete examples from Microsoft stack (but design for portability)
|
||||||
|
- Reference the motorway analogy sparingly (it's in README, don't repeat)
|
||||||
|
|
||||||
|
## Meta-Note
|
||||||
|
|
||||||
|
This document itself demonstrates AILint's philosophy: **explicit marking** of AI assistance. It was co-authored with Claude, and that fact is openly stated rather than hidden. As AILint develops, we should maintain this transparency about our own AI usage.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*Last updated: 2024-12-28*
|
||||||
|
*This document will evolve as the project matures. Contributions welcome.*
|
||||||
Loading…
Add table
Add a link
Reference in a new issue