The engineers have a pitch that sounds almost too good: swap one line of code, and we'll shrink your AI costs by 75 percent or more.
Orchestra, which emerged from Y Combinator's latest batch, launched Tuesday with software that watches how companies use expensive large language models, then trains cheaper replacements tailored to those exact tasks. The San Francisco startup claims early tests show cost reductions between 75 and 83 percent when its system routes work away from frontier models like Claude or GPT-4 and onto specialized alternatives it builds from customer data.
Whether that math holds at scale remains an open question. The company has two employees, won't name production customers, and bases its public case on a handful of internal benchmarks run on workloads with names like "CRM agent" and "operations." Still, the underlying bet feels familiar to anyone tracking the infrastructure layer beneath the generative AI boom: as model costs balloon, someone will figure out how to make them cheaper without sacrificing quality.
Luis Manrique and Aamir Poonawalla think they have.
Manrique spent five years across Google's ads machine learning and programmatic teams and later closed what he describes as roughly $2 million in first-year sales at Gumloop, an automation startup. Poonawalla logged 10 years at Instacart building the ads auction platform and experimentation systems. Both have felt the sting of inference bills firsthand. "I've seen a single workflow go from costing $400K annually to just a few hundred dollars a month to run," Manrique wrote in August, though he didn't specify where or when.
Orchestra's method is straightforward in concept, if complex in execution. Companies keep their existing OpenAI or Anthropic integrations. They change the base URL to Orchestra's gateway, and the platform begins capturing every request. It generates evaluation datasets from real traffic, trains or fine-tunes smaller open-source models, then promotes those models to production only after they beat the original on held-out tests. If scores slip, the system rolls back automatically. Traffic starts at 5 percent of volume, creeping to full scale if performance holds.
The company positions itself against what the founders call a $104 billion inference market growing fourteen-fold year over year at Anthropic, citing their own calculations. That figure reflects the breakneck pace of enterprise adoption, but also the founders' tendency to frame the opportunity in venture-friendly terms. They're pitching investors as much as customers.
Published results offer a window into how the system performs under controlled conditions, though these numbers reflect specific workloads on particular dates rather than aggregated production data across diverse customers. A CRM workflow running seven tasks showed costs dropping from $1.12 per run on Claude Sonnet to 27 cents on a fine-tuned Qwen model with a revised prompt. Mean task scores improved slightly, to 0.630 from 0.557. Another test on operations tasks found a custom 8-billion-parameter model delivered median latency of 369 milliseconds versus nearly two seconds for Sonnet, while token costs fell by roughly 83 percent across 90 attempts. Mean accuracy dipped to 0.963 from a perfect 1.0, a trade-off the founders argue most enterprises would accept.
The most dramatic savings appeared in a batch job labeling 39,962 YouTube comments pulled from Snowflake. A 30-billion-parameter model trained by Orchestra cost $2.82 to process the full set, compared to $12.48 for Sonnet and nearly $140 for Opus. Valid output counts hovered near identical across all three models, though Orchestra's own documentation notes the obvious caveat: without human ground truth, model agreement proves little about actual accuracy.
These numbers carry an asterisk. The company's homepage touts an 83.4 percent cost reduction, but that figure comes from a single illustration rather than a composite claim. RuntimeWire, an AI infrastructure newsletter, observed in September that Orchestra's "100x" cost-reduction tagline—visible in marketing materials—lacks named customer validation, with first-party studies showing savings ranging from 4.4 times to around 50 times depending on the task.
Manrique and Poonawalla are entering a crowded field. OpenRouter lets developers pick models by price and latency. Portkey markets a unified interface to more than 250 models with routing and observability built in. Tokenless, another company from the same YC batch, claims automatic model switching delivers similar savings. Evaluation platforms like Braintrust and serverless providers like Fireworks address adjacent pieces of the inference stack.
Orchestra's angle is coupling routing with custom training. The platform doesn't just find the cheaper model; it builds one from your data. That's a technical gamble and a business model rolled into one. Training costs money and time, and not every workload will tolerate the accuracy trade-offs that come with smaller models. The company also faces the classic cold-start problem: until it accumulates enough traffic from a customer, it can't train an effective replacement.
The startup raised a $125,000 seed from Y Combinator in June under the accelerator's standard deal structure at the time, a post-money SAFE for 7 percent equity plus a $375,000 uncapped most-favored-nation SAFE totaling $500,000. Dealroom's database lists the round. Orchestra hasn't disclosed additional funding, pricing details, or revenue, which is typical for a company this early but makes it hard to assess commercial traction beyond the technical demonstrations.
Some operational details hint at the learning curve ahead. The company's developer documentation lives at a domain referencing "Understudy," an earlier brand name before a September rebrand to Orchestra. Integration requires passing upstream provider keys via request headers, and response headers expose which model handled each request. It's plumbing that matters to engineers but rarely factors into the glossy launch narrative.
Orchestra shouldn't be confused with a London-based data orchestration platform also called Orchestra, which announced a $3.3 million seed on September 1. The naming collision is unfortunate but not uncommon in a sector generating startups faster than trademark lawyers can keep pace.
What matters now is whether Manrique and Poonawalla can convert benchmarks into contracts, and benchmarks into production workloads that companies trust with revenue-critical tasks. Plenty of infrastructure startups have solved the demo. Fewer survive contact with enterprise procurement cycles, compliance reviews, and the grinding reality that swapping out a foundation model often breaks more than it fixes.
For now, Orchestra is a bet that inference optimization is less about finding the right off-the-shelf model and more about training the right bespoke one. If they're correct, the two-person team could carve out a meaningful position in the infrastructure stack. If not, they'll join the long list of YC startups that had a clever idea, a clean demo, and no one willing to bet their production environment on it.
