mirror of
https://github.com/microsoft/BCQuality.git
synced 2026-10-05 14:46:55 +01:00
d2e2f2adb4
1. text-artifacts-require-explicit-utf8-handling (Type B): to produktionshaendelser, samme sygdom - smiley double-encodet via HTTP-streng, og en Mode B-batch der skrev mojibake i tre CLAUDE.md fordi PS 5.1 laeser BOM-loese .ps1 som cp1252. Reglen: raa bytes ved transfer, eksplicit UTF-8 ved write, tekst-transformation i Python, grep for maerket foer commit. 2. log-writes-must-survive-rollback (Type B): fejl-logs skrevet i samme transaktion forsvinder ved rollback - loggen mister praecis de fejl den findes for. Isoleret session (StartSession -> insert + commit) er moensteret. Evidens: Wareco IC web-service-log 2026-07-02; havde tidligere kostet en udvikler det meste af en dag. 3. Skaerpelse af setup-doc-must-not-reference-unpromoted-stable-files: kanal-tilstand verificeres via git (ls-remote/klon), aldrig CDN; to sande observationer kan modsige hinanden naar verden flytter sig imellem dem - tidsstempl al kanal-evidens. Felt-verificeret af Conzept-sessionens selvkorrektion 2026-07-03. Foerste PR foedt direkte i QualityHub. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
66 lines
2.9 KiB
Markdown
66 lines
2.9 KiB
Markdown
---
|
|
bc-version: [all]
|
|
domain: architecture
|
|
keywords: [encoding, utf-8, powershell, mojibake, here-string, bom, raw-bytes]
|
|
technologies: [al]
|
|
countries: [w1]
|
|
application-area: [all]
|
|
---
|
|
|
|
# Text artifacts require explicit UTF-8 handling — no ambient encoding
|
|
|
|
## Description
|
|
|
|
Two production incidents, same disease, different vectors. (1)
|
|
`smiley.agent.md` shipped double-UTF-8-encoded for weeks: fetched as string
|
|
content over HTTP and re-saved with ambient encoding — every em-dash became
|
|
`â€"`, and `file` misclassified the agent as Nim source code. (2) A Mode B
|
|
batch script wrote the SAME mojibake into three project CLAUDE.md files:
|
|
Windows PowerShell 5.1 reads a BOM-less `.ps1` as cp1252, so UTF-8 em-dashes
|
|
inside a here-string were mis-decoded before they were ever written. Both
|
|
incidents corrupted deployed instruction files silently; both were caught by
|
|
inspection, not by tooling.
|
|
|
|
The root cause is never the characters — it is trusting an AMBIENT encoding
|
|
(HTTP string decoding, PS 5.1 script parsing, `Out-File` defaults) anywhere
|
|
between a text artifact's source and its destination.
|
|
|
|
## Rule
|
|
|
|
When creating, fetching or transforming CURABIS text artifacts (agent files,
|
|
CLAUDE.md, knowledge files, scripts):
|
|
|
|
1. **Transfers are raw bytes.** Filesystem `Copy-Item` or
|
|
`Invoke-WebRequest -OutFile` — never via `.Content` strings.
|
|
2. **Writes are explicit UTF-8.** `[System.IO.File]::WriteAllText(path,
|
|
text, [System.Text.UTF8Encoding]::new($false))` — never `Out-File` /
|
|
`Set-Content` defaults for content with non-ASCII.
|
|
3. **Text transformation with non-ASCII happens in Python** (explicit
|
|
`encoding="utf-8"` on read AND write) — or, if it must be PowerShell, the
|
|
`.ps1` file itself carries a UTF-8 BOM so PS 5.1 parses its literals
|
|
correctly. A BOM-less `.ps1` with non-ASCII literals is a latent bug.
|
|
4. **Verify after generation:** grep the output for `â€` (the mojibake
|
|
signature) before committing or deploying. One line, catches the class.
|
|
|
|
## What NOT to do
|
|
|
|
- Do not put em-dashes, æøå or any non-ASCII inside a here-string in a
|
|
`.ps1` that lacks a BOM — the corruption happens at parse time, before
|
|
your code runs
|
|
- Do not fetch a file via `.Content` and re-save it — the double-decode is
|
|
invisible until someone reads the artifact
|
|
- Do not assume "it looked fine in my editor" — editors auto-detect; parsers
|
|
and runtimes do not
|
|
|
|
## Signal to watch for
|
|
|
|
The literal byte sequence `â€` (or `æ`, `ø`, `Ã¥`) in any committed or
|
|
deployed text file. Also: `file`/tooling misclassifying a markdown file as
|
|
source code — that is encoding damage until proven otherwise.
|
|
|
|
## Message to developer
|
|
|
|
When mojibake is found in an artifact, report which file, repair by
|
|
cp1252→UTF-8 reversal (line-by-line fallback preserves already-correct
|
|
lines), and identify WHICH transfer step used ambient encoding — the repair
|
|
without the root cause just schedules the next incident.
|