Chuoi trong C# duoc ma hoa bang UTF-16. Phan lon ky tu chi gom mot don vi ma 16 bit. Cac ky tu nam ngoai Basic Multilingual Plane, bao gom emoji, can hai don vi: mot high surrogate (U+D800 den U+DBFF) dung truoc mot low surrogate (U+DC00 den U+DFFF). Surrogate di kem sai loai hoac dung mot minh deu tao thanh chuoi khong hop le.
Ban khong nen ghi chuoi khong hop le ra dia hoac gui qua mang. Do la mot thoi quen khong tot.
Trong JavaScript, chung ta co cac ham nhanh de sua chuoi hoac kiem tra xem chuoi co can sua hay khong:
String.prototype.toWellFormed()thay moi surrogate dung mot minh bang U+FFFD.isWellFormed()cho biet co can thay the hay khong.
Toi da them ca hai ham nay vao thu vien C# SimdUnicode trong pull request 54. Thuat toan nay cung la thuat toan ma chung toi da dong gop cho engine V8 cua JavaScript, vi the Chrome da sua chuoi theo cach nay.
string s = UTF16.ToWellFormed(input); // cung instance neu input da hop le
bool ok = UTF16.IsWellFormed(span);Neu input da hop le, ToWellFormed tra lai chinh no. Khong can cap phat bo nho.
Chuoi duoc sua nhu the nao? Ve co ban, cac gia tri dau vao sai duoc thay bang ky tu thay the U+FFFD.
Cac bo xu ly hien dai co cac lenh dac biet goi la SIMD, cho phep xu ly song song du lieu: ban co the so sanh nhieu gia tri cung luc. Cac bo xu ly x64 gan day cua AMD va Intel co kha nang xu ly song song du lieu tot hon chip ARM, du ca hai deu co cac lenh manh.
Cach thong thuong de sua chuoi trong C# la dung mot ham nhu sau:
static void Repair(ReadOnlySpan<char> input, Span<char> output)
{
input.CopyTo(output);
int i = NextError(output, 0);
while (i >= 0)
{
output[i] = '\uFFFD';
i = NextError(output, i + 1);
}
}
// Vi tri surrogate dung mot minh dau tien tai hoac sau 'start',
// hoac -1 neu khong co.
static int NextError(ReadOnlySpan<char> s, int start)
{
int i = start;
while (true)
{
int k = s.Slice(i).IndexOfAnyInRange('\uD800', '\uDFFF');
if (k < 0) return -1;
i += k;
if (char.IsHighSurrogate(s[i]) && i + 1 < s.Length && char.IsLowSurrogate(s[i + 1]))
i += 2; // cap hop le, bo qua
else
return i;
}
}Trong SimdUnicode, toi cung su dung xu ly song song du lieu.
Do toc do
Tren Intel Xeon Gold 6548N, voi AVX-512, viec kiem tra chuoi Latin dat 69 GB/s, trong khi IndexOfAnyInRange chi dat 33 GB/s. Du lieu Emoji la hop le va chi gom cac cap surrogate. Toc do tim kiem cua runtime giam xuong 0,4 GB/s, con bo kiem tra cua chung toi dat 53 GB/s.
Ket qua tren Apple M4 Max tuong tu, du khong an tuong bang ket qua tren Intel.
Phep kiem tra co the dung ngay khi gap surrogate dung mot minh dau tien. Phien ban ToWellFormed dung buffer ghi moi don vi ma: hoac la ban sao cua input, hoac la U+FFFD. Khi input hop le, thao tac nay ve thuc chat la sao chep bo nho. Vi vay, chung ta co the so sanh toc do voi thao tac sao chep.
Nhìn chung, toc do cua chung toi on dinh gan bang toc do sao chep.
Cac phien ban da su dung: .NET SDK 10.0.400 tren Linux, 10.0.103 tren macOS. Intel Xeon Gold 6548N (Emerald Rapids) va Apple M4 Max.
Clausecker, R. va Lemire, D. (2026). Fixing ill-formed UTF-16 strings with SIMD instructions. Software: Practice and Experience. Xem them tai arXiv.
