{"id":11194,"date":"2026-11-20T13:48:22","date_gmt":"2026-11-20T13:48:22","guid":{"rendered":"https:\/\/www.hostinger.com\/blog\/?p=11194"},"modified":"2026-11-20T13:48:26","modified_gmt":"2026-11-20T13:48:26","slug":"compressed-ai-models-for-real-products","status":"publish","type":"post","link":"https:\/\/www.hostinger.com\/blog\/compressed-ai-models-for-real-products\/","title":{"rendered":"What broke when we compressed AI models for real products"},"content":{"rendered":"<p class=\"wp-block-paragraph\">A mixture-of-experts (MoE) model is made of many small sub-networks called experts, and it only calls on a few of them for each piece of text it reads or writes. Every expert still has to sit in GPU memory, even when it isn&rsquo;t in use, so the obvious way to cut costs is to remove the ones that contribute least.<\/p><p class=\"wp-block-paragraph\">When <a data-click-id=\"hgr-post-content-link-our-researchers-tried-that-on-five-tasks-from-hostinger-prod\" href=\"https:\/\/arxiv.org\/abs\/2611.12240\" target=\"_blank\" rel=\"noopener\">our researchers tried that on five tasks from Hostinger products<\/a>, the compressed models could still write well, but they stopped reliably producing answers the products could use, and benchmark scores didn&rsquo;t show it.<\/p><p class=\"wp-block-paragraph\">There are two ways to remove experts: prune them (delete them) or merge them into similar experts you keep. On standard benchmarks, the two score so close that picking one looks like a coin flip. On our tasks, they weren&rsquo;t close.<\/p><p class=\"wp-block-paragraph\">Merging away a quarter of the experts in Qwen3-30B, an openly available model from Alibaba, left four in ten of the marketing emails it wrote for our email tool broken on a phone screen. Pruning the same number kept the broken share to about one in ten, roughly what the original model does.<\/p><p class=\"wp-block-paragraph\">Merging also switched languages mid-sentence and got stuck in loops far more often. If you&rsquo;re compressing an MoE model for a product, prune it, treat a quarter of the experts as the limit for anything that writes text, since deeper cuts hurt both methods, and test on your own tasks before you trust a benchmark score.<\/p><p class=\"wp-block-paragraph\">Here&rsquo;s what broke in our tests, why merging caused it, and why benchmarks didn&rsquo;t catch it.<\/p><h2 class=\"wp-block-heading h-t-title-2\" id=\"h-three-ways-the-compressed-models-broke\">Three ways the compressed models broke<\/h2><p class=\"wp-block-paragraph\">Our five tasks ranged from short labels (spotting style cues in website descriptions, extracting brand names, and categorizing products) to writing (domain descriptions and full email templates). On the short-label tasks, the two methods stayed close.<\/p><p class=\"wp-block-paragraph\">Writing was a different story.<\/p><p class=\"wp-block-paragraph\">With a quarter of the experts removed, pruning kept 93.6% of <a data-click-id=\"hgr-post-content-link-qwen3-30b\" href=\"https:\/\/huggingface.co\/Qwen\/Qwen3-30B-A3B-Instruct-2507\" target=\"_blank\" rel=\"noopener\">Qwen3-30B<\/a>&lsquo;s score on the writing tasks while merging kept 62.5%, and the failures behind that gap fell into three patterns.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa668310\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa668310\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-writing-quality-kept.png\" alt=\"Column chart of how much of Qwen3-30B&rsquo;s original writing-task score each method kept: pruning kept 93.6% and merging 62.5% with 25% of experts removed, and pruning kept 70.3% and merging 51.8% with 50% removed\" class=\"wp-image-11206\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-writing-quality-kept.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-writing-quality-kept-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-writing-quality-kept-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-writing-quality-kept-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><h3 class=\"wp-block-heading h-t-title-3\">Emails that broke on phones<\/h3><p class=\"wp-block-paragraph\">Our <a data-click-id=\"hgr-post-content-link-email-marketing-tool\" href=\"https:\/\/www.hostinger.com\/email-marketing\">email marketing tool<\/a> asks the model for a complete, responsive HTML template, wrapped in JSON so the product can read it.<\/p><p class=\"wp-block-paragraph\">Before the design or the copy gets graded, the template has to pass a set of hard checks, including valid JSON, a complete HTML document, an unsubscribe link, and a layout that holds up on a narrow screen.<\/p><p class=\"wp-block-paragraph\">Miss any one of them, and it scores zero.<\/p><p class=\"wp-block-paragraph\">The phone check did the most damage. With a quarter of Qwen3-30B&rsquo;s experts merged, 40% of its templates overflowed a narrow screen, against 10% for the original model.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa669659\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa669659\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/email-templates-broken-on-mobile.png\" alt=\"Column chart of Qwen3-30B email templates that overflowed a phone screen: 8.8% with pruning and 40.1% with merging at 25% of experts removed, and 35.1% and 51.7% at 50%, against 10% for the original model\" class=\"wp-image-11205\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/email-templates-broken-on-mobile.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/email-templates-broken-on-mobile-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/email-templates-broken-on-mobile-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/email-templates-broken-on-mobile-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><p class=\"wp-block-paragraph\">Nearly all of the lost score on this task came from templates that failed a check. The ones that did pass scored almost as well as the original model&rsquo;s. The model could still write a decent email. It just couldn&rsquo;t reliably hand over one the product could send.<\/p><h3 class=\"wp-block-heading h-t-title-3\">Answers that never stopped<\/h3><p class=\"wp-block-paragraph\">Some compressed models got stuck repeating the same text until they hit our length limit. In 2,000 email requests, the original Qwen3-30B did this six times. With half its experts merged, it did it 215 times.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa66a6d2\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa66a6d2\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/compressed-models-runaway-answers.png\" alt=\"Column chart of Qwen3-30B email answers that repeated text until the length limit, out of 2,000: 14 with pruning and 61 with merging at 25% of experts removed, and 102 and 215 at 50%, against 6 for the original model\" class=\"wp-image-11204\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/compressed-models-runaway-answers.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/compressed-models-runaway-answers-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/compressed-models-runaway-answers-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/compressed-models-runaway-answers-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><p class=\"wp-block-paragraph\">Each loop burns GPU time on output nobody can use. That compressed model generated about twice as many tokens on the email task as the original, which is the opposite of why you&rsquo;d compress a model in the first place.<\/p><p class=\"wp-block-paragraph\">Pruning looped far less: with a quarter of the experts removed, merging got stuck about four times as often.<\/p><h3 class=\"wp-block-heading h-t-title-3\">&ldquo;This domain adalah&hellip;&rdquo;<\/h3><p class=\"wp-block-paragraph\">Our domain tool asks the model for a sentence or two about what a domain could be used for, written in whatever language the user picked.<\/p><p class=\"wp-block-paragraph\">The instructions are in English and tell the model to begin with &ldquo;This domain is,&rdquo; so a good Indonesian answer starts with the Indonesian version of that phrase, &ldquo;Domain ini adalah&hellip;&rdquo;<\/p><p class=\"wp-block-paragraph\">The compressed Qwen3-30B models often kept the English opener, or part of it, and then carried on in the user&rsquo;s language, producing half-and-half sentences like &ldquo;This domain adalah&hellip;&rdquo; in Indonesian or &ldquo;This dominio es&hellip;&rdquo; in Spanish.<\/p><p class=\"wp-block-paragraph\">With a quarter of the experts merged, about four in ten non-English answers started this way, against about 3% for the original model.<\/p><p class=\"wp-block-paragraph\">Every answer is graded by an AI judge that checks whether it&rsquo;s in the requested language, opens with the right phrase, and stays a sensible length, and it rejects these mixed-language answers.<\/p><p class=\"wp-block-paragraph\">That one habit explains most of the merged model&rsquo;s drop outside English. When it kept to the right language, its answers passed almost as often as the original model&rsquo;s.<\/p><p class=\"wp-block-paragraph\"><a data-click-id=\"hgr-post-content-link-nemotron-3-5-lightning\" href=\"https:\/\/build.nvidia.com\/nvidia\/nemotron-3.5-lightning-30b-a3b\" target=\"_blank\" rel=\"noopener\">Nemotron-3.5-Lightning<\/a>, the second model we tested, failed differently. It more often skipped the target language altogether and answered in English.<\/p><h2 class=\"wp-block-heading h-t-title-2\" id=\"h-why-merging-broke-more-than-pruning\">Why merging broke more than pruning<\/h2><p class=\"wp-block-paragraph\">Both methods start the same way. They run a sample of calibration data (real examples of the work the model will do) through the model in a single pass, with no retraining. Ours was a few thousand production conversations.<\/p><p class=\"wp-block-paragraph\">Along the way, they score each expert by how much it contributes when it handles a token (the small unit of text a model reads and writes, usually a word or part of one). Then the two methods split on what to do with the low scorers.<\/p><p class=\"wp-block-paragraph\"><a data-click-id=\"hgr-post-content-link-reap\" href=\"https:\/\/arxiv.org\/abs\/2510.13999\" target=\"_blank\" rel=\"noopener\">REAP<\/a>, the leading pruning method, deletes them, and every expert that survives stays exactly as it was.<a data-click-id=\"hgr-post-content-link-arxiv-org-abs-2604-04356\" aria-label=\"2604\" href=\"https:\/\/arxiv.org\/abs\/2604.04356\" target=\"_blank\" rel=\"noopener\"><\/a><\/p><p class=\"wp-block-paragraph\"><a data-click-id=\"hgr-post-content-link-ream\" href=\"https:\/\/arxiv.org\/abs\/2604.04356\" target=\"_blank\" rel=\"noopener\">REAM<\/a>, the leading merging method, tries to save some of what the low scorers knew by folding them into the high scorers. It groups similar experts around each one it keeps and averages each group into one.<\/p><p class=\"wp-block-paragraph\">The catch is that some of the experts you keep aren&rsquo;t the originals anymore. They&rsquo;re averages of their group.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa66bca5\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa66bca5\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-experts-diagram.png\" alt=\"Diagram of one model layer with eight experts, two marked as low scorers: pruning deletes those two and leaves the other six unchanged, while merging folds them into similar kept experts, which become averages of their group\" class=\"wp-image-11207\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-experts-diagram.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-experts-diagram-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-experts-diagram-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-vs-merging-experts-diagram-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><p class=\"wp-block-paragraph\">That difference matters most where exact behavior counts, like format rules, a translated phrase, or long structured output, which is exactly where merging broke.<\/p><p class=\"wp-block-paragraph\">We ran both methods on two models built in different ways, so we could check that the result wasn&rsquo;t tied to one design:<\/p><ul class=\"wp-block-list\">\n<li>Qwen3-30B is a standard Transformer, the architecture behind most of today&rsquo;s language models.<\/li>\n\n\n\n<li>NVIDIA&rsquo;s Nemotron-3.5-Lightning is a hybrid that mixes Transformer layers with Mamba layers, a newer design that keeps a running summary of the text instead of looking back at every token.<\/li>\n<\/ul><p class=\"wp-block-paragraph\">Across five tasks, both models, and two compression levels, pruning won 16 of 20 head-to-head matchups. Merging&rsquo;s answers also drifted further from the original model&rsquo;s behavior in all 20.<\/p><h2 class=\"wp-block-heading h-t-title-2\" id=\"h-why-benchmarks-miss-failures-like-these\">Why benchmarks miss failures like these<\/h2><p class=\"wp-block-paragraph\">Each of those three failures points to something benchmarks don&rsquo;t measure. The study wasn&rsquo;t built to test these one at a time, so we can&rsquo;t say which matters most, but together they help explain how two methods that look interchangeable on benchmarks can split apart on a real product.<\/p><p class=\"wp-block-paragraph\">&ldquo;A score that barely moves can still hide a lot of change, even with the better method,&rdquo; said Povilas Kvedaras, AI researcher at Hostinger.<\/p><p class=\"wp-block-paragraph\">&ldquo;When we pruned a quarter of Qwen3-30B&rsquo;s experts, the share of its product categories our AI judge accepted moved by about one point, but more than one in five products landed in a different category than before. You only see that if you compare the answers themselves, not just the totals.&rdquo;<\/p><h3 class=\"wp-block-heading h-t-title-3\">Products have strict output rules<\/h3><p class=\"wp-block-paragraph\">A product needs JSON it can parse, HTML that renders, and an unsubscribe link it can find. Many benchmarks are more forgiving and give credit for anything close to the right answer. Our email tool doesn&rsquo;t.<\/p><p class=\"wp-block-paragraph\">With half of Qwen3-30B&rsquo;s experts merged, invalid JSON went from about 1% of answers to 12%.<\/p><p class=\"wp-block-paragraph\">The tasks with a schema told a different story. Brand names and categorization were generated with structured outputs, a feature of vLLM (the open-source engine we ran the models on) that forces every answer to follow a JSON schema, and almost all of those answers stayed valid in every model, compressed or not.<\/p><p class=\"wp-block-paragraph\">The email templates had no schema to enforce. The study doesn&rsquo;t prove the schema is what protected them, but if your output has one, keep it switched on when you compress.<\/p><h3 class=\"wp-block-heading h-t-title-3\">Products serve many languages<\/h3><p class=\"wp-block-paragraph\">Our test data covered up to 35 languages per task, because that&rsquo;s what our users write in.<\/p><p class=\"wp-block-paragraph\">Neither of the original papers on these methods tested generation in multiple languages, so a habit like copying an English opener into an Indonesian answer had no way to show up there.<\/p><p class=\"wp-block-paragraph\">Take our domain task with a quarter of the experts removed. English answers kept passing about 98% of the time with either method, while non-English answers fell to 77% with pruning and 55% with merging.<\/p><p class=\"wp-block-paragraph\">Testing in English alone would have shown almost no damage on that task.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa66d3f5\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa66d3f5\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/domain-descriptions-english-vs-non-english.png\" alt=\"Column chart of Qwen3-30B&rsquo;s pass rate on domain descriptions by answer language: English answers stayed near 98% for the original model and both compressed versions, while non-English answers fell from 92.6% to 76.6% with pruning and 55.1% with merging\" class=\"wp-image-11203\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/domain-descriptions-english-vs-non-english.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/domain-descriptions-english-vs-non-english-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/domain-descriptions-english-vs-non-english-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/domain-descriptions-english-vs-non-english-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><h3 class=\"wp-block-heading h-t-title-3\">Long answers leave more room for loops<\/h3><p class=\"wp-block-paragraph\">A typical email template from the original Qwen3-30B runs to about 2,700 tokens, and every one of those tokens is another chance to slip into a loop.<\/p><p class=\"wp-block-paragraph\">Many benchmarks ask for short answers, like a single multiple-choice letter, which gives a model far fewer chances to show the habit.<\/p><p class=\"wp-block-paragraph\">Nor was it one model&rsquo;s quirk: with half its experts merged, about one in seven of Nemotron&rsquo;s email answers also ran to the length limit.<\/p><h2 class=\"wp-block-heading h-t-title-2\" id=\"h-what-wed-do-before-shipping-a-compressed-model\">What we&rsquo;d do before shipping a compressed model<\/h2><p class=\"wp-block-paragraph\">If we were compressing a mixture-of-experts model for one of our products, we&rsquo;d follow four rules.<\/p><h3 class=\"wp-block-heading h-t-title-3\">Choose pruning over merging<\/h3><p class=\"wp-block-paragraph\">Pruning won most of our matchups, and it leaves you with a checkpoint the same size as merging does, so you don&rsquo;t give up any memory savings by choosing it.<\/p><p class=\"wp-block-paragraph\">Merging may catch up with changes like smaller expert groups, router repair, or choosing between pruning and merging expert by expert, but for now pruning is the safer bet.<\/p><div class=\"wp-block-image\">\n<figure data-wp-context='{\"imageId\":\"6b006fa66e785\"}' data-wp-interactive=\"core\/image\" data-wp-key=\"6b006fa66e785\" class=\"aligncenter size-full wp-lightbox-container\"><img decoding=\"async\" width=\"1460\" height=\"820\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on--click=\"actions.showLightbox\" data-wp-on--load=\"callbacks.setButtonStyles\" data-wp-on--pointerdown=\"actions.preloadImage\" data-wp-on--pointerenter=\"actions.preloadImageWithDelay\" data-wp-on--pointerleave=\"actions.cancelPreload\" data-wp-on-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-lead-over-merging-by-task.png\" alt=\"Column chart of pruning&rsquo;s lead over merging on Qwen3-30B with 25% of experts removed: +0.2 on website cues, +4.2 on brand names, +4.0 on categorization, +18.1 on domain descriptions, and +30.2 on email templates\" class=\"wp-image-11202\" srcset=\"https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-lead-over-merging-by-task.png 1460w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-lead-over-merging-by-task-300x168.png 300w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-lead-over-merging-by-task-1024x575.png 1024w, https:\/\/www.hostinger.com\/blog\/wp-content\/uploads\/sites\/4\/2026\/11\/pruning-lead-over-merging-by-task-768x431.png 768w\" sizes=\"(max-width: 1460px) 100vw, 1460px\" \/><button class=\"lightbox-trigger\" type=\"button\" aria-haspopup=\"dialog\" data-wp-bind--aria-label=\"state.thisImage.triggerButtonAriaLabel\" data-wp-init=\"callbacks.initTriggerButton\" data-wp-on--click=\"actions.showLightbox\" data-wp-style--right=\"state.thisImage.buttonRight\" data-wp-style--top=\"state.thisImage.buttonTop\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewbox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\"><\/path>\n\t\t\t<\/svg>\n\t\t<\/button><\/figure>\n<\/div><h3 class=\"wp-block-heading h-t-title-3\">Treat 25% as the limit for anything that writes text<\/h3><p class=\"wp-block-paragraph\">With a quarter of the experts removed, pruning barely touched Qwen3-30B&rsquo;s short-label tasks, though domain descriptions still dropped 13 percentage points. With half removed, even pruning lost more than 20 points on both writing tasks.<\/p><p class=\"wp-block-paragraph\">Nemotron also lost ground on the short-label tasks with a quarter removed, where Qwen3-30B didn&rsquo;t, so test each model on its own rather than borrowing someone else&rsquo;s ratio.<\/p><h3 class=\"wp-block-heading h-t-title-3\">Check the format and every language, not just the average<\/h3><p class=\"wp-block-paragraph\">Track JSON validity, rendering, answer length, and loop rate next to your quality score, and split the results by language.<\/p><p class=\"wp-block-paragraph\">For an early warning before a full evaluation, two cheap signals tracked quality closely in our study: the loop rate, and how surprising the original model finds the compressed model&rsquo;s answers (cross-perplexity). Neither replaces testing on your own tasks.<\/p><h3 class=\"wp-block-heading h-t-title-3\">Calibrate on the output you care about<\/h3><p class=\"wp-block-paragraph\">Answers made up less than a tenth of our calibration data, and at least two-thirds of the text was English. Yet the failures showed up in what the model wrote, and the language failures hit non-English users.<\/p><p class=\"wp-block-paragraph\">English-heavy calibration also <a data-click-id=\"hgr-post-content-link-eroded-lithuanian-in-an-earlier-expert-pruning-study-we-cond\" href=\"https:\/\/arxiv.org\/abs\/2611.07654\" target=\"_blank\" rel=\"noopener\">eroded Lithuanian in an earlier expert-pruning study we conducted<\/a>. Giving answers more weight, or calibrating on the model&rsquo;s own outputs, is worth testing.<\/p><p class=\"wp-block-paragraph\">A research paper can try several calibration samples and report the best one, but you have to pick yours before you see the scores, so build the mix around what your product needs.<\/p><h2 class=\"wp-block-heading h-t-title-2\" id=\"h-what-this-means-before-you-ship-a-smaller-model\">What this means before you ship a smaller model<\/h2><p class=\"wp-block-paragraph\">We&rsquo;d expect a similar gap between benchmark scores and product behavior with other changes to a production model, like quantization (storing the model&rsquo;s numbers at lower precision so it takes up less memory) or switching to a newer one, though this study didn&rsquo;t test them.<\/p><p class=\"wp-block-paragraph\">Our team saw a version of it in separate research on <a data-click-id=\"hgr-post-content-link-handing-a-conversation-from-one-model-to-another\" href=\"https:\/\/www.hostinger.com\/blog\/kv-cache-translation\/\">handing a conversation from one model to another<\/a>: the model that took over the conversation picked up the first model&rsquo;s habit of refusing some requests, and the standard quality scores didn&rsquo;t flag it.<\/p><p class=\"wp-block-paragraph\">&ldquo;Whenever you swap or shrink a model, the benchmark numbers will probably look fine,&rdquo; said Tomas Rasymas, AI research lead at Hostinger.<\/p><p class=\"wp-block-paragraph\">&ldquo;I&rsquo;d spend that time on the boring checks instead, like whether it keeps the format, knows when to stop, and refuses the same things it used to, because that&rsquo;s what your users notice first.&rdquo;<\/p><p class=\"wp-block-paragraph\">There are also a few things this study can&rsquo;t tell you. We didn&rsquo;t measure serving memory, latency, or throughput, so how much compression saves in practice is still an open question.<\/p><p class=\"wp-block-paragraph\">If you <a data-click-id=\"hgr-post-content-link-rent-gpus-by-the-hour\" href=\"https:\/\/www.hostinger.com\/gpu-hosting\">rent GPUs by the hour<\/a>, a smaller checkpoint is where the savings would show up, though answers that loop until the length limit can eat into them. We also ran each setup once, on two models of one size, so treat this as one careful study rather than a universal rule.<\/p><p class=\"wp-block-paragraph\">Even so, the pattern was clear. The compressed models could still write a decent email, but they couldn&rsquo;t reliably hand over one the product could send, and no benchmark score showed it.<\/p><p class=\"wp-block-paragraph\">Check for that on your own tasks before you ship, and see <a data-click-id=\"hgr-post-content-link-our-paper-on-expert-compression\" href=\"https:\/\/arxiv.org\/abs\/2611.12240\" target=\"_blank\" rel=\"noopener\">our paper on expert compression<\/a> for the full methods and per-task results.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A mixture-of-experts (MoE) model is made of many small sub-networks called experts, and it only calls on a few of them for each piece of text it reads or writes. Every expert stil\u2026<\/p>\n","protected":false},"author":455,"featured_media":11201,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_content_updated_date":"","footnotes":""},"categories":[2417],"tags":[],"hashtags":[],"class_list":["post-11194","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-insights"],"hreflangs":[],"_links":{"self":[{"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/posts\/11194","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/users\/455"}],"replies":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/comments?post=11194"}],"version-history":[{"count":1,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/posts\/11194\/revisions"}],"predecessor-version":[{"id":11208,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/posts\/11194\/revisions\/11208"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/media\/11201"}],"wp:attachment":[{"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/media?parent=11194"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/categories?post=11194"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/tags?post=11194"},{"taxonomy":"hashtags","embeddable":true,"href":"https:\/\/www.hostinger.com\/blog\/wp-json\/wp\/v2\/hashtags?post=11194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}