[Services](https://noodleseed.com/services) / Custom AI and evaluation Custom AI and evaluation # AI designed for one hard problem, and proven by evaluation. Some problems no single model solves well: a market’s own language, voice notes that mix scripts, a regulated decision. We design the workflow, measure every candidate against your own cases, and run the version that wins, with its cost capped. [Talk to us](https://noodleseed.com/contact) [All services](https://noodleseed.com/services) Your problem and your casesYou Framework, candidates, evaluationsWe design The version that wins, in productionWe run ## What we do ### Frame the problem What a right answer looks like, the hard cases, and the test set that proves it: synthetic where you have none, your real data where you do. ### Design the workflow A higher-level framework around the models: passes, routing, guards and fallbacks, each with settings that evaluation decides. ### Choose the models Ours, a partner’s or open source, chosen by results on your cases, even when the winner competes with our own platform. ### Run the evaluations Error rates, cost and speed for every candidate and setting, on the same cases, so decisions rest on numbers rather than demos. ### Gate every release A change to a model, prompt or setting ships only when it passes the regression set, with nothing silently made worse. ### Run it in production Cost limits per item, monitoring, and re-evaluation as models and prices change, under the same controls as the rest of our platform. Case study ## Voice notes in Urdu, English and both at once. Customers on WhatsApp send voice notes instead of typing: Urdu, English, mixed, full of product and place names. A single model misheard the names or, given silence, invented words. So we designed a two-pass framework and let evaluation set it. 01 ### First pass: speech to text A transcription model primed with the business’s own words. It hears nothing in silence or noise, so an empty note ends there and no model that could invent words ever hears it. 02 ### Second pass: careful correction A multimodal model hears the audio with the first transcript, the business’s words and the recent conversation, and corrects only what it clearly hears differently. 03 ### Guards around both A correction that changes a number, or swaps a common word for a product name, is refused. Urdu that comes back in Devanagari is rewritten in Urdu script. Each note has a capped cost. ### What the evaluation decided - **117 of 117** hard reads correct with the chosen design, against 23 of 39 when the second pass ran by rule alone and 53 to 68 of 117 when the multimodal model heard the audio without the first transcript. - **1.9% character error** across 25 languages for the first-pass model we selected, a partner’s model, with nothing written for silence or noise. - **Where the second pass earns its cost:** it is kept for the languages where it measurably helped, and skipped elsewhere. Character error, first pass alone and with the second pass | Language | First pass | With second pass | | --- | --- | --- | | Cantonese | 28% | 3% | | Tamil | 7% | 0% | | Bengali | 4.5% | 1% | Figures are from synthetic recordings built for the evaluation, with hard cases chosen on purpose. Real recordings decide the final list of languages for each deployment. Not tied to our own stack ## Whatever works best, even when it isn’t ours. We recommend and run partner products and open source wherever they win on your cases, including tools that compete with our own platform. You get the best result, under the same controls. - **Partner products** , from model providers to the systems you already pay for. - **Open-source models and tools** , run in our managed cloud, yours or on your premises. - **Competing platforms** , when they fit better than ours. - **Our own platform and Noodle Core** , when they are the best fit. Run it anywhere Multi-tenant ### Our managed cloud The fastest start, run by us, with each customer’s data, credentials and records kept apart. Single-tenant ### Our managed cloud A dedicated environment run by us, for stricter isolation or a region of your choice. Your account ### Your cloud Deployed in your own cloud account, or on capacity from our cloud partners, run by our team or yours. Your hardware ### On premises On your own servers and network, including disconnected sites, for data that cannot leave. Talk to us ## Tell us what you need. We will suggest where to start. Any layer, any size: a review, a project, a programme or a partnership. Who you areWhat you need firstSize Suggested starting point **An evaluation on your own cases, then the workflow and models that win it: ours, a partner's or open source, tuned where needed and measured against your current results.** Layer: Custom AI and evaluation Delivery: Advise, Build, Run Scale: Review, then a first project Your nameWork emailCompany website Ask us to get in touch We use your details only to reply. See our [privacy policy](https://noodleseed.com/privacy). --- Source: https://noodleseed.com/services/custom-ai [All public pages in Markdown](https://noodleseed.com/llms.txt)