StyloMail la mot thu nghiem de xem co the di xa den dau khi dung Jev de xay dung mot proxy chan spam dua tren hanh vi bang .NET Core. He thong xem xet email theo ca hai chieu: thu den hop thu va thu roi khoi mot tai khoan co the da bi chiem quyen.
StyloMail la phan mem nghien cuu. Muc tieu la xay dung du de chay, thay doi va danh gia y tuong co dang theo duoi hay khong. Day la mot cuoc dieu tra co gioi han thoi gian, trong do phan mem dang chay la cong cu de dat ra va kiem chung cac cau hoi.
Gan day, toi cung xay dung mot he thong xu ly spam gui ra cho mot khach hang bang Azure Service Bus va mau interceptor. Vi vay, spam gui ra khong phai mot van de xa la. StyloMail cho phep toi thu nghiem rieng xem Jev co the lam don gian hon phan suy luan hanh vi hay khong.
Toi da xay dung nhieu bien the cua cung mot y tuong: phat hien bot, phan tich khach hang va xu ly tai lieu. Dau vao va hanh dong co the khac nhau, nhung cau truc cot loi van tuong tu: thu thap bang chung tung phan, ghi nho phan huu ich, suy ra dieu dang xay ra va ap dung chinh sach ro rang.
Cau hoi trung tam la:
Mot he thong suy luan hanh vi co the don gian den muc nao khi danh gia ngu nghia duoc cung cap duoi dang mot ham goi nho, co kieu ro rang?
Trong bai nay, ngu nghia la viec nhan ra hai thong diep dung tu khac nhau nhung cung yeu cau mot dieu. Co kieu nghia la ung dung biet truoc dang hop le cua cau tra loi, tuong tu nhu mot ham C# tra ve bool, enum hay mot con so.
Toi dung Jev cua TypeSafe de thu nghiem. Jev nhan trang thai, tuc thong diep va du lieu boi canh do ung dung cung cap, cung voi cac cau hoi co kieu, sau do tra ve cac danh gia co cau truc.
Email la bai toan thu nghiem phu hop: ngon ngu mo ho, moi quan he thay doi, tinh khan cap hop le, tinh khan cap co tinh xau, va nhung tai khoan van hoat dong binh thuong cho den khi bi nguoi khac kiem soat.
Mục lục
Bat dau tu hanh vi
Hay xem thong diep sau:
Vui long dung thong tin tai khoan da cap nhat cho khoan thanh toan thang nay. Toi can xu ly viec nay trong hom nay.
Thong diep nay dua ra thong tin thanh toan va tao ap luc ve thoi gian. No co the hoan toan binh thuong neu den tu nha cung cap quen thuoc, trong mot chuoi trao doi dung ve khoan thanh toan do.
Nhung neu cung mot noi dung bat dau den tu mot tai khoan dot nhien gui thu cho hang chuc nguoi la, bang chung xung quanh da thay doi. Van ban khong doi, nhung boi canh thi co.
Do la dieu toi goi la suy luan hanh vi. Ta quan sat cac manh hoat dong, xem xet moi quan he, lich su va su thay doi cua chung de suy ra dieu co the dang xay ra. Sau do ta chon phan ung phu hop, dong thoi van giu lai mot muc do khong chac chan ve nguyen nhan.
Trong mo hinh hanh vi cua Stylo.Bot, bang chung huu ich den tu hinh dang hoat dong cua client tren nhieu tin hieu. Dia chi IP hay danh tinh trinh duyet dang tin chi ke mot phan cau chuyen. Email cung co van de tuong tu: tai khoan bi chiem quyen van co the xac thuc thanh cong, nhung hanh vi cua no da thay doi rat manh.
Ta khong can tuyen bo rang da doc duoc y nghi cua nguoi gui. Ta chi can du bang chung de quyet dinh yeu cau nay co can duoc xem xet ky hon hay khong.
Vi sao dung Jev thay vi mot LLM thong thuong?
Mot LLM da nang co the phan loai email: du doan email thuoc nhom nao. Ta dua thong diep, danh sach nhom va yeu cau ket qua. Cac API LLM hien dai cung ho tro dau ra bi rang buoc boi schema, trong do cau truc va cac gia tri hop le da duoc quy dinh.
Jev dua ra mot danh doi khac. Theo mo ta cua TypeSafe, Jev tap trung vao cac quyet dinh xac suat co kieu, giu lai muc do kha nang cua cac phuong an, cho phep xu ly nhieu cau hoi song song va huong den cac danh gia duoc hieu chinh xac.
Dao tao la viec dieu chinh mo hinh bang cac vi du. Suy luan la viec dung mo hinh da dao tao tren dau vao moi. Goi Jev la suy luan; he thong khong dao tao lai mo hinh cho tung email.
Hieu chinh dat cau hoi: xac suat co phu hop voi ket qua quan sat hay khong? Neu nhieu cau hoi nhan gia tri gan 0.8, thi khoang 80 phan tram trong so do nen co cau tra loi la “co”, neu cac du doan duoc hieu chinh tot. Day la dieu phai do tren nhieu mau, khong phai loi hua cho mot thong diep rieng le.
Voi thu nghiem nay, day la mot danh doi hap dan. Toi can danh gia mot thuoc tinh cu the cua thong diep, khong can mot doan van giai thich dai trong moi lan goi.
API cung cap ba dang cong cu:
- Choice: chon mot gia tri tu danh sach do ung dung dinh nghia, chang han “bien lai”, “hoi dap ban hang” hay “tin nhan ca nhan”.
- Score: danh gia theo cac muc co thu tu, chang han “khong khan cap”, “khan cap vua” va “rat khan cap”. Cac mo ta nay tao thanh tieu chi cua thang do.
- Noul: tra ve xac suat mot menh de dung/sai la dung, chang han “thong diep nay yeu cau thong tin dang nhap”.
Nhieu cau hoi co the dung chung mot trang thai va chay song song. Loi ich thuc tien la xem mo hinh chuyen biet co du nhanh va re de dung thuong xuyen hay khong. Loi ich kien truc la giao dien tu nhien khuyen khich cac cau hoi nho, de ung dung tu ket hop ket qua.
Ta van co the ap dung kien truc nay voi LLM da nang. Jev chi bien no thanh cach su dung mac dinh cua mo hinh. Jev co tra loi chinh xac hon, hieu chinh tot hon va re hon voi workload email cu the hay khong van can duoc do luong.
Ngoai ra, dau ra dung schema chi dam bao ve hinh dang. Mo hinh van co the tra ve mot cau tra loi hop le cho cach hieu sai ve thong diep. An toan kieu khong dong nghia voi tinh dung.
Jev khac Stylo.Bot nhu the nao?
Stylo.Bot da thuc hien suy luan hanh vi. Cac bo phat hien dong gop cac tin hieu: dac diem giao thuc, thoi diem, dieu huong va tinh nhat quan. “Hinh dang” la to hop gia tri cua cac tin hieu. Hai client co the gui voi cung tan suat nhung truy cap trang theo thu tu rat khac nhau. So sanh ca to hop se huu ich hon chi so sanh tan suat.
Day la mot he thong hoan chinh. Jev la mot mo hinh co the duoc dung ben trong he thong do.
Stylo.Bot cung co mot tuy chon lam giau bang LLM khi cac tin hieu bat dong hoac hoat dong khong giong cac mau quen thuoc. Vi vay, khac biet khong phai la “Stylo.Bot co luat, Jev co AI”. Gia tri lon cua Stylo.Bot den tu moi quan he giua cac quan sat, bo nho duoc giu lai va cach he thong hoc mau nao dang quan trong.
Voi email, mot phan lon bai toan quan sat mang tinh ngon ngu. “Hay dung thong tin ngan hang thay the cua chung toi” va “cac khoan chuyen tien sau nay hay gui den tai khoan nay” co the mo ta cung mot hanh vi. Thay doi dac diem TLS co the do truc tiep; yeu cau am tham bo qua quy trinh phe duyet thi can duoc dien giai.
Jev cung cap cac chieu ngu nghia co ten de dat canh cac tin hieu do truc tiep. Mot chieu co the dai dien cho viec yeu cau thong tin dang nhap, chuyen huong thanh toan hay tao ap luc ve thoi gian. Cac cau tra loi cua Jev tro thanh feature cho he thong xung quanh.
- Phan loai ngu nghia: co the thay the mot lan goi LLM da nang hoac mot so bo nhan dien tu viet.
- Quan sat truc tiep: bo sung cho cac su that do duoc nhu so nguoi nhan, dich den cua lien ket va thoi gian gui.
- Bieu dien hanh vi: them cac chieu co y nghia da duoc lua chon, nhu yeu cau thong tin dang nhap hay chuyen huong thanh toan.
- Bo nho va so sanh: cung cap bang chung cho ho so; ung dung van giu lich su va tinh muc thay doi.
- Hoc va hanh dong: cung cap danh gia; ung dung van kiem soat viec cap nhat dang tin va chinh sach.
Muoi hai cau hoi ngu nghia khong the thay the toan bo cac bo phat hien cua Stylo.Bot hay cac mau ket hop ma he thong da hoc. Chung quan sat nhung thuoc tinh khac nhau trong mot linh vuc khac. Thu nghiem nay xem chung co giup xay dung phien ban danh cho email voi it co che ngu nghia tu viet hon hay khong.
Vi sao phai xu ly hai chieu?
Email den dat cau hoi: yeu cau tu ben gui co phu hop voi moi quan he, nguoi gui va noi dung da quan sat hay khong. Email gui ra cho phep ta xem mot tai khoan noi bo co bat dau hanh xu khac di hay khong.
Mot tai khoan nhan vien co the xac thuc dung, nhung bat dau gui mot dang yeu cau bat thuong den mot tap nguoi nhan dang mo rong. Xac thuc la that. Su thay doi hanh vi cung la that.
Hai chieu la hai phep thu cho cung mot y tuong. Cau hoi ngu nghia co the giong nhau, nhung lich su, bang chung va hau qua cua viec can thiep thi khac nhau. Nguoi gui noi bo quen thuoc khong lam moi email gui ra tro nen an toan; nguoi gui ben ngoai la khong quen cung khong co nghia moi email den deu la xau.
Xac suat cua dieu gi?
Gia su cau hoi la:
Thong diep nay co dang dua vao hoac thay doi thong tin dich den cua khoan thanh toan khong?
Gia tri Noul 0.95 nghia la mo hinh gan xac suat cao cho cau tra loi “co”. No khong co nghia xac suat gian lan la 95 phan tram. Mot thong bao thay doi tai khoan hop le cung co the nhan gia tri cao.
Tuong tu, 0.5 nghia la mo hinh gan kha nang gan nhu nhau cho “co” va “khong”. No khong phai muc do nghiem trong. Noul cung khong co truong confidence rieng.
Neu ta dat ten bien la fraudScore, mot danh gia ngu nghia huu ich co the bi bien thanh mot ket luan manh hon o phia sau. Vi vay, can giu dung noi dung cau hoi ban dau.
StyloMail hien dat muoi hai cau hoi ve viec moi chao, yeu cau thong tin dang nhap, chuyen huong thanh toan, tuyen bo quyen han, ap luc thoi gian, yeu cau giu bi mat, thong tin nhay cam, lien ket, tep dinh kem, tinh giao dich va tinh lien tuc cua hoi thoai.
Cac thuoc tinh nay co the cung ton tai. Mot yeu cau thanh toan co the khan cap, mang tinh giao dich va hoan toan hop le. Tach rieng tung thuoc tinh giup he thong hanh vi co du lieu phong phu hon de so sanh theo thoi gian.
Loi goi Jev trong C# thuc te
Dinh nghia y nghia cua cau hoi
Cau hoi ve chuyen huong thanh toan la mot muc trong tap dinh nghia chieu ngu nghia:
Id = "semantic.payment_redirection",
Instructions = "Does this message attempt to introduce or change payment destination details?",
CriteriaTrue = "Supplies or changes bank account, routing, invoice or payment destination details.",
CriteriaFalse = "No introduction or change of payment destination.",Id cho phep ma nguon gan cau tra loi voi mot chieu. Y nghia nam trong Instructions va mo ta cho hai phia co va khong. Neu thay doi cac mo ta nay, ta thay doi dieu dang do, du ten thuoc tinh C# van giu nguyen.
private static Dictionary<string, JevQuestion> BuildQuestions(IEnumerable<SemanticDimension> dimensions)
{
var questions = new Dictionary<string, JevQuestion>(StringComparer.Ordinal);
foreach (var dimension in dimensions)
{
questions[dimension.Id] = new JevQuestion
{
Type = NoulType,
Instructions = dimension.Instructions,
Criteria = new JevNoulCriteria
{
True = dimension.CriteriaTrue,
False = dimension.CriteriaFalse,
},
};
}
return questions;
}NoulType la hang so co gia tri noul. Moi muc trong tu dien la mot cau hoi co/khong rieng.
Dat cau hoi canh trang thai
var state = BuildState(input.Message, input.TaggedContext, input.Profile);
var questions = BuildQuestions(askable);
var request = new JevRequest
{
State = state,
Model = _options.Model,
Questions = questions,
};askable chi chua cac cau hoi ma dau vao can thiet dang ton tai. Chang han, cau hoi ve tinh lien tuc cua hoi thoai duoc bo qua khi khong co hoi thoai de so sanh.
BuildState dua vao thong diep da phan tich cung mot so boi canh duoc chon, trong do co ho so nguoi gui. Thong diep la du lieu duoc danh gia; dinh nghia cau hoi thuoc ve ma nguon cua ung dung. Tat ca cau hoi di trong mot request.
Gui bang HttpClient
using var httpRequest = new HttpRequestMessage(HttpMethod.Post, _options.Endpoint)
{
Content = JsonContent.Create(request, options: JsonOptions),
};
httpRequest.Headers.Authorization = new AuthenticationHeaderValue("Bearer", _options.ApiKey);Phien ban thu nghiem su dung endpoint mac dinh https://api.typesafe.ai/v1/systemone va khoa mo hinh jev-1.13.0. API key duoc nap tu bien moi truong TYPESAFE_API_KEY, khong nhung truc tiep vao ma nguon. Khoa phien ban mo hinh giup ket qua de so sanh hon khi nha cung cap phat hanh ban moi.
response = await _http
.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead, timeout.Token)
.ConfigureAwait(false);Token huy la mot nguon lien ket, cho phep ben goi hoac thoi han da cau hinh huy thao tac HTTP. Neu thanh cong, phan hoi duoc doc vao record JevResponse.
if (response.IsSuccessStatusCode)
{
return await response.Content
.ReadFromJsonAsync<JevResponse>(JsonOptions, cancellationToken)
.ConfigureAwait(false);
}Phan con lai cua ham xu ly loi va thu lai. Diem dang chu y la giao dien mo hinh can rat it co che dac biet: trang thai va cau hoi di vao duoi dang JSON, con cau tra loi quay ve theo cac ID cau hoi.
Dang phan hoi
Mot fixture kiem thu dung tieu de Invoice attached va noi dung:
Vui long xac nhan mat khau cua ban de xem hoa don.
Day la du lieu gia lap dung de kiem thu adapter, khong phai du doan Jev truc tiep:
{
"model": "jev-1.13.0",
"answers": {
"semantic.credential_request": {
"type": "noul",
"noul": 0.93
},
"semantic.payment_redirection": {
"type": "noul",
"noul": 0.11
},
"semantic.urgency_pressure": {
"type": "noul",
"noul": 0.11
}
},
"usage": {
"input_tokens": 296,
"output_tokens": 20
}
}Fixture co tinh gan 0.93 cho cau hoi ve thong tin dang nhap va 0.11 cho cac chieu con lai. Muc dich la kiem tra adapter, khong phai do Jev hieu thong diep tot den dau.
Trong phep kiem tra anh xa:
Assert.Equal(EvidenceAvailability.Available, credential.Availability);
Assert.Equal(0.93, credential.Value);
Assert.Equal(EvidenceOrigin.Semantic, credential.Origin);
Assert.Null(credential.Confidence);Neu cau tra loi bi thieu, no tro thanh Unavailable, khong phai 0. Neu ton tai, gia tri Noul duoc ghi lai nhu mot quan sat ngu nghia, kem phien ban mo hinh va thoi diem quan sat. Khong co Block hay Allow trong ket qua. Doan ma nay chi ghi nhan dong gop cua mo hinh; chinh sach moi quyet dinh dong gop do cho phep lam gi.
Ket qua tu lan chay thuc te
Mot lan chay rieng voi thong diep lua dao tong hop ghi nhan cac gia tri sau:
credential_request: 0.99urgency_pressure: 0.99link_lure: 0.99sensitive_data_request: 0.98threat_reward_inducement: 0.98identity_authority_claim: 0.76secrecy_bypass: 0.23transactional_character: 0.11payment_redirection: 0.09unsolicited_solicitation: 0.07attachment_lure: 0.03
Adapter ghi nhan muoi mot chieu co du lieu va mot chieu NotApplicable: tinh lien tuc cua hoi thoai, vi khong co hoi thoai duoc cung cap. Cac cau tra loi Noul deu khong co truong confidence rieng.
Hinh dang nay huu ich hon mot nhan “spam” duy nhat. Mau thu nghiem the hien manh yeu cau thong tin dang nhap va hanh dong bam lien ket, nhung it the hien chuyen huong thanh toan hay loi keo mo tep dinh kem. Tuy nhien, mot mau tong hop van chua noi duoc nhieu ve bao dong gia hay hieu nang tren email that.
Bien y nghia thanh thu co the so sanh
Buoc chuyen doi quan trong la dua ngon ngu da dang ve mot tap nho cac chieu ngu nghia co the so sanh.
“Hay dung thong tin ngan hang thay the” va “cac khoan chuyen tien sau nay gui den tai khoan sau” khac nhau ve chuoi ky tu, nhung co the mo ta cung mot hanh vi. Mot cau hoi ngu nghia hep cho phep nhan ra moi quan he nay ma khong can liet ke moi cach dien dat.
Cac cau tra loi co the ket hop voi tin hieu duoc tinh truc tiep:
- Noi dung thong diep: yeu cau hay tuyen bo dieu gi?
- Mau gui: co bao nhieu hoat dong va no dang thay doi ra sao?
- Lich su nguoi nhan: day la moi quan he quen thuoc hay mot tap nguoi moi?
- Moc dang tin: hanh vi binh thuong cua tai khoan nay trong lich su la gi?
- Do phu va ho tro: co bao nhieu tin hieu du kien dang co va co bao nhieu quan sat lam co so?
Dieu nay gan voi y tuong cua Reduced RAG: rut gon dau vao thanh bang chung huu ich truoc khi cac buoc sau suy luan. Phan quyet dinh van giu tinh xac dinh: cung dau vao thi ma nguon tao ra cung ket qua. Dien giai ngu nghia duoc giao mot nhiem vu co gioi han rieng.
Ta co tinh loai bo thong tin trong qua trinh rut gon. Muoi hai cau hoi khong the giu lai moi thu ve mot email. Chung giu lai cac phan biet duoc cho la quan trong trong thu nghiem nay. Chon dung cac phan biet do la mot phan cua cong viec.
Thoi gian bo sung dieu ma bo phan loai khong tu co
Danh gia ngu nghia mo ta mot thong diep tai mot thoi diem. Hanh vi chi lo ro qua nhieu thoi diem.
Bo phan ke toan gui thong diep thanh toan. Dich vu giam sat gui cac dot canh bao. Doi ban hang tiep can nguoi nhan moi. Moi hoat dong deu co the hop ly trong lich su rieng cua no.
Dieu dang chu y thuong la su roi khoi lich su: mot dang yeu cau moi, tap nguoi nhan mo rong, toc do tang nhanh bat thuong, hoac nhieu thay doi cung xuat hien.
StyloMail duy tri ho so hanh vi voi gioi han ve luong lich su. He thong so sanh hoat dong gan day voi moc dang tin, tuc hinh anh ve hanh vi da duoc chap nhan la binh thuong.
- Drift: hanh vi gan day cach moc dang tin bao xa?
- Velocity: hanh vi dang thay doi nhanh den muc nao?
- Acceleration: toc do thay doi co dang tang nhanh hon khong?
Khong can mo hinh AI de tinh cac dai luong nay. Tuy nhien, can du quan sat, cac cua so co the so sanh va kha nang thua nhan khi bang chung khong du.
Moi dai luong them vao deu phai co ich. Acceleration nghe co ve thong minh, nhung neu mot phep do toc do don gian da bat duoc cung cac truong hop voi it bao dong gia hon, he thong don gian se tot hon.
Boi canh quay tro lai danh gia
Co hai chieu bo tro nhau. Danh gia ngu nghia giup mo ta hanh vi theo thoi gian. Boi canh hanh vi lai giup dien giai thong diep tiep theo.
Boi canh gui cho Jev la mot mo ta co gioi han kich thuoc ro rang, do ma nguon lap ra tu cac quan sat, so dem, cua so thoi gian, moc so sanh va muc do ho tro cua bang chung. Ta co the noi tap nguoi nhan cua mot nguoi gui da mo rong ma khong can dua toan bo lich su trao doi vao mo hinh.
["distinct_recipients_last_hour"] = profile.DistinctRecipientsLastHour,
["distinct_recipients_last_30_days"] = profile.DistinctRecipientsLast30Days,
["recipients_novel_to_sender"] = profile.RecipientsNovelToSender,
["messages_last_hour"] = profile.MessagesLastHour,
["messages_last_24_hours"] = profile.MessagesLast24Hours,
["baseline_messages_per_hour"] = profile.BaselineMessagesPerHour,Neu chua co moc so sanh, gia tri van la null. Tai khoan chua co du lich su khong bi trinh bay nhu tai khoan co toc do gui binh thuong bang khong.
Tuy nhien, co mot bay. Neu boi canh noi “tai khoan nay dang dang ngo” va mo hinh sau do cung danh gia thong diep la dang ngo, ta da thu duoc bao nhieu thong tin moi? Co the la rat it. Ta chi dang yeu cau mo hinh dong y voi ket luan co san.
Do do, boi canh nen mo ta quan sat va muc do ho tro, de mo ngo ket luan. Ngoai ra, khong nen xem hai cau hoi chay rieng la hai bang chung doc lap. Cau “hay hanh dong ngay neu khong se mat quyen truy cap” co the cung luc kich hoat danh gia khan cap va de doa, vi ca hai deu den tu cung mot cau. Dem chung nhu hai xac nhan doc lap se phong dai luong bang chung.
Hoc cung can co ranh gioi
Khi bo nho anh huong den cach dien giai tuong lai, viec hoc tro thanh mot quyet dinh co hau qua. O day, hoc nghia la cap nhat ho so va thong ke dang tin cua ung dung, khong phai dao tao lai Jev hay thay doi tham so noi bo cua mo hinh.
Hay tuong tuong mot tai khoan bat dau gui thong diep lam dung. Neu moi quan sat lap tuc day moc binh thuong theo huong moi, hanh vi tan cong keo dai cuoi cung se tro thanh “binh thuong”. He thong se ngay cang thoai mai khi cuoc tan cong tiep dien.
Can tach hai dang bo nho: dieu da xay ra va dieu da duoc xac nhan la dang tin.
StyloMail giu ro su khac biet nay. Hoat dong quan sat co the tich luy, nhung viec dua vao moc dang tin phai can phan hoi duoc uy quyen hoac mot quy tac duoc cho phep. He thong cung luu nguon goc va ly do cua phan dinh. Viec gui thanh cong hay “khong ai phan nan” khong du de xac nhan hanh vi la dang tin.
Vong phan hoi can bang chung ve ket qua. Lap lai du doan cua chinh he thong khong tao ra bang chung moi.
Giao dien nho van can su khong chac chan trung thuc
Ngoai xac suat cao va thap, con mot kha nang thu ba: ta khong thu duoc danh gia co the su dung.
Nha cung cap co the khong san sang. Noi dung co the bi ma hoa. Cau hoi ve tinh lien tuc cua hoi thoai co the khong co hoi thoai de so sanh. Nguoi gui moi co the chua co moc lich su co y nghia.
Day la khoang trong cua bang chung. Dien chung bang khong se lam he thong trong co ve an toan dung luc no biet it nhat.
StyloMail theo doi kha nang co du lieu rieng va dua do phu vao chinh sach. He thong ket hop cac tin hieu co san bang trong so cau hinh. Ket qua la mot chi so rui ro, dung de so sanh voi cac nguong hanh dong, khong phai xac suat gian lan da duoc hieu chinh.
Mot chi so nho nhung dua tren rat it bang chung phai duoc doi xu khac voi chi so nho duoc ho tro boi mot danh gia day du. Ta cung co the hien thi cac quan sat va danh gia da dong gop vao hanh dong. Do la duong quyet dinh co the kiem tra, khong phai suy nghi noi bo cua mo hinh.
Vay quyet dinh nam o dau?
Quyet dinh nam trong ma nguon ma ta co the doc va kiem thu. Chinh sach la cac quy tac ro rang cua ung dung de chon hanh dong tu bang chung, khong phai mot mo hinh khac duoc dao tao de quyet dinh thay ta.
Jev dong gop bang chung ngu nghia. Lop hanh vi dong gop bang chung lich su va thoi gian. Chinh sach quyet dinh to hop do cho phep hanh dong gi.
Voi vi du ve thanh toan, phan ung co the la tam dung de xem xet. Ta co the co du bang chung de kiem tra yeu cau ma khong can tuyen bo da chung minh tai khoan bi chiem quyen.
Chi phi cua viec sai phai duoc dua vao chinh sach. Gian doan mot khoan thanh toan hop le va bo sot mot giao dich xau la hai loai hau qua khac nhau. Xac suat do mo hinh cung cap khong the tu chon thay ta danh doi do.
Chinh sach xac dinh khong lam mot danh gia kem tro thanh dung. No lam cho hau qua tro nen ro rang, co gioi han va co the kiem thu. Voi cung bang chung va cung chinh sach, ta co the lap lai hanh dong va hoi xem chinh sach co phu hop hay khong.
Phan mem nghien cuu: Agile spike co AI ho tro
Toi can phan mem giup tim ra mot y tuong co dang theo duoi hay khong. Doc tai lieu va phac thao kien truc chi dua toi di mot doan. Chay y tuong tao ra cac cau hoi khac: dieu gi kho xu ly, dieu gi that bai, gia dinh nao con dung va phan nao co the loai bo.
Do la ly do toi xay dung cac cong cu co gioi han thoi gian. Thu y tuong, lam cho no chay, lap lai nhanh, sau do quyet dinh co nen dau tu them hay khong. Gioi han thoi gian la gioi han cho viec mua them su hieu biet, khong phai loi hua bien ket qua thanh san pham.
Mot Agile spike cung cap mot khoang thoi gian co gioi han de giai quyet su khong chac chan. Phat trien co AI ho tro giup toi dua nhieu hon vao khoang gioi han do. Thay vi chi danh gia mot thu vien hay lam mot tich hop nho, toi co the xay dung du mot y tuong gom dau vao, trang thai, quyet dinh, phan hoi va cach kiem tra ket qua.
Do la Agile spike co AI ho tro ma toi quan tam. Ket qua chinh la su hieu biet; doi khi mot cong cu huu ich cung duoc tao ra.
Trong cau chuyen nay co hai cach dung AI. Mot LLM viet ma giup toi xay dung va sua thu nghiem. Jev la thanh phan dang duoc danh gia ben trong thu nghiem. Xay dung nhanh hon cho phep thu nhieu bien the hon, nhung khong chung minh bien the nao hoat dong tot.
Phan mem nghien cuu van can can than. Neu du lieu thieu bien thanh diem so co ve an toan, hoac bai kiem thu am tham dung sai lich su, ta se hoc sai bai hoc. Can du tinh dung va kha nang quan sat de tin vao thu nghiem: nhin duoc dau vao, ket qua trung gian va quyet dinh.
StyloMail thuoc dang phan mem do. Co mot he thong mail xung quanh vong suy luan vi toi muon xem xet y tuong trong boi canh thuc. Van chuyen, luu tru ben vung va giao mail van can nhieu ky thuat. Su ton tai cua chung khong bien day thanh mot thong bao ra mat san pham.
Dieu gi khien y tuong dang de theo duoi?
Dieu toi dang kiem tra rat cu the: dung Jev cho phan loai ngu nghia, giu cac tin hieu do truc tiep trong ma nguon thong thuong, dong thoi bao toan bo nho va phep so sanh lam nen he thong hanh vi.
Ta co tap cau hoi nho, bo nho co gioi han, cac phep tinh thay doi theo thoi gian, quy trinh hoc co kiem soat va chinh sach ro rang. Moi phan deu tao ra mot diem de do luong. Ta co the thay doi cau hoi, bo mot chieu, rut ngan cua so hoac so sanh he thong co va khong co boi canh ngu nghia.
Mo hinh nen tang phuc tap va hien duoc truy cap qua dich vu luu tru. Giao dien nho khong co nghia toan bo he thong don gian ve mat tinh toan. Dieu dang kiem tra la chung ta can xay dung bao nhieu co che xung quanh danh gia ngu nghia de tao ra hanh vi huu ich.
Mot phep so sanh dung can nhieu phien ban:
- Chi dung bang chung hanh vi cuc bo.
- Cung duong ong nhung de LLM da nang tra loi cau hoi ngu nghia.
- Dung Jev voi chi noi dung thong diep.
- Dung Jev voi boi canh hanh vi co gioi han.
Can do bao dong gia, tuc mail hop le bi danh dau la lam dung, va bo sot, tuc mail xau khong bi phat hien. Can do thoi gian phan hoi va chi phi tren cung tap mau. Cach nay tach loi ich cua danh gia ngu nghia khoi loi ich cua lich su, dong thoi tach ca hai khoi lua chon mo hinh.
Sau do, hay thu cac truong hop kho: thay doi hop le, tai khoan moi chua co lich su, bang chung bi thieu va phan hoi vo tinh day moc binh thuong theo huong chap nhan hanh vi xau. Day la noi kien truc gon gang phai chung minh gia tri.
Ket thuc khoang thoi gian nghien cuu, toi muon co du bang chung de chon: tiep tuc, doi huong, giu lai mot thanh phan huu ich hay dung lai. Phat hien rang co che them vao khong mang lai nhieu gia tri cung la mot ket qua nghien cuu co ich.
Diem lam Jev thu vi la giao dien cua no phu hop tu nhien voi cach lam nay. Y nghia tro thanh mot dong gop co gioi han de he thong tich luy, so sanh va phan bien. Bo nho them tinh lien tuc. Chinh sach bien su khong chac chan thanh hau qua duoc kiem soat.
Do la dieu toi muon tim hieu khi theo Jev: xay dung y tuong, thu nghiem, do nhung thay doi va quyet dinh xem khoang thoi gian tiep theo co dang dau tu hay khong.
