What Neon and Castform’s 4B Retrieval Model Changes for Agentic Search

Neon and Castform say they have post-trained a 4B open model that can match GPT-5.6 Sol on retrieval tasks while costing about 100x less to run. If the claim holds up in production benchmarks, this is not just another small-model headline. It changes the economics of agentic search: the kind of search where an AI system does not retrieve one chunk, summarize it, and stop, but instead plans, queries, checks evidence, follows references, and answers with citations.

That matters because retrieval-heavy agents burn tokens fast. A single support investigation, sales research task, policy lookup, or internal operations question can require 5-20 retrieval calls, multiple reranking passes, and a final synthesis step. Frontier models work well for the reasoning layer, but they are expensive when used for every search, filter, evidence-selection, and citation-validation step. A strong 4B retrieval specialist gives teams a practical routing option: use the small model for the repetitive search loop, then reserve premium models for final judgment.

This post breaks down what changed, why multi-hop retrieval costs explode, and what founders, product teams, and developers can build next. We will cover 7 practical workflows, two copyable implementation outlines, recommended model stacks, cost-per-run math, cheaper fallbacks, risks, and when teams should still pay for premium models.

---

What changed in agentic retrieval

Traditional RAG was simple: embed documents, retrieve the top chunks, send them to a model, and generate an answer. That pattern works for narrow FAQ bots, but it breaks down when questions require joining evidence across several systems.

Agentic retrieval adds a planning loop. Instead of one search, the model decides what to search next. It can ask: “Which policy applies?”, “What changed after March?”, “Which customer account is this tied to?”, “Do we have conflicting evidence?”, and “Is this answer supported by the source text?” That makes retrieval more accurate, but it also multiplies cost.

Neon and Castform’s claim points to a more efficient architecture: a post-trained 4B open model specialized for retrieval behaviors such as query rewriting, document selection, evidence scoring, citation checking, and multi-hop search control. The model is small enough to deploy cheaply, but targeted enough to compete with a frontier model on the retrieval slice of the workflow.

For product teams, the important distinction is task specialization. You do not need a premium generalist model to decide whether chunk A or chunk B is more relevant to a query. You do need a premium model when the answer requires nuanced synthesis, legal interpretation, deep coding, high-stakes decisioning, or user-facing prose that must be excellent.

Why the market cares now

Three market forces make this timely:

1. Internal AI search is moving from demo to production. Teams are no longer satisfied with a chatbot over PDFs. They want systems that can answer account questions, inspect support histories, reconcile policies, and produce auditable answers.
2. Agentic workflows are expensive on frontier models. Multi-hop retrieval can turn a cheap-looking app into a costly workload because every step adds tokens.
3. Open specialist models improve control. A 4B model can be self-hosted, fine-tuned, quantized, monitored, and routed differently from closed frontier models.

The result is a new pattern: use a small retrieval agent as the “search operator” and a frontier model as the “decision editor.”

[stat] ~100x lower retrieval cost Neon and Castform say their post-trained 4B open model can match GPT-5.6 Sol on retrieval while costing about two orders of magnitude less.

---

Why multi-hop search costs explode on frontier models

The cost problem comes from iteration. A normal chatbot answer might use 4,000 input tokens and 800 output tokens. A multi-hop retrieval agent can use 30,000-150,000 total input tokens across planning, search results, reranking, citation checking, and final synthesis.

Here is a realistic retrieval-heavy internal query:

Step	What happens	Typical token load
User question analysis	Parse intent, entities, constraints	500-1,500 tokens
Query expansion	Generate 3-8 searches	1,000-3,000 tokens
Retrieval result review	Inspect top chunks from docs, tickets, CRM, database text	10,000-60,000 tokens
Reranking and evidence selection	Score, dedupe, and select sources	5,000-25,000 tokens
Gap check	Decide if more searches are needed	2,000-10,000 tokens
Final synthesis	Produce answer with citations	4,000-20,000 tokens

If every stage uses GPT-5.6 Sol, pricing is $5 per 1M input tokens and $30 per 1M output tokens. That is reasonable for final synthesis, but expensive for repetitive retrieval loops. A 100,000-input-token investigation costs $0.50 in input alone on Sol before output. At 100,000 runs per month, that becomes $50,000/month just for input-side retrieval traffic.

Now compare that with a specialized small model that runs at roughly 100x lower retrieval cost. The same retrieval loop drops from $0.50 to about $0.005 before final synthesis. That changes which products are viable: support copilots, internal research agents, sales intelligence tools, compliance search, and operations assistants can run at high volume without blowing up gross margins.

---

Seven workflows this unlocks now

A cheap retrieval-specialist model is most valuable when the workflow has many search steps and relatively little final prose. Here are the best near-term use cases.

1. Internal docs agent with evidence trails

Build an assistant that answers questions across Notion, Google Drive, Confluence, Slack exports, GitHub issues, and internal runbooks. The 4B model handles query expansion, source selection, and citation validation. A stronger model such as GPT-5.6 Terra, Claude Sonnet 5, or Gemini 3 Pro handles final synthesis when the answer goes to executives or customers.

Best for: engineering enablement, HR policy, onboarding, security procedures, and product documentation.

2. Support investigation agent

Support tickets often require multi-hop retrieval: product docs, prior tickets, account data, changelogs, incident reports, and known bugs. A retrieval specialist can gather evidence cheaply before a final model drafts the response.

Best for: B2B SaaS support teams with high ticket volume and lots of historical context.

3. Sales account research copilot

Before a renewal or expansion call, the agent can pull CRM notes, support tickets, product usage summaries, emails, meeting transcripts, and public company signals. The retrieval model finds and ranks evidence. A premium model writes a concise account brief.

Best for: customer success, sales engineering, renewals, and RevOps.

4. Compliance and policy lookup

For internal policy questions, the hard part is not elegant writing. It is finding the exact clause, checking whether it has been superseded, and citing the right version. Retrieval-specialist models can cheaply inspect many policy fragments and flag conflicts.

Best for: procurement, security review, legal operations, and HR.

5. Engineering incident memory

Incident response benefits from fast retrieval across past incidents, logs, postmortems, alerts, runbooks, and code references. The 4B retrieval model can locate similar incidents and relevant mitigations, while a coding or reasoning model helps produce an action plan.

Best for: SRE, platform engineering, security operations, and DevOps.

6. Product feedback synthesis

Teams can run nightly agents over support tickets, sales notes, app reviews, community posts, and NPS comments. The retrieval model clusters evidence and finds examples. A stronger model writes the weekly product insights memo.

Best for: PMs, growth teams, UX researchers, and founders.

7. Operational data Q&A

Many internal questions require a blend of structured and unstructured data: “Which customers affected by the EU billing bug also asked about invoice exports?” The agent can search docs, tickets, and database-derived summaries, then escalate final reasoning to a stronger model.

Best for: operations, finance, growth analytics, and internal tooling.

---

Workflow 1: Build an internal docs agent with cited answers

This is the most direct workflow for founders and product teams. The goal is not a generic chatbot. The goal is a system that can answer internal questions with traceable citations and confidence labels.

Recommended stack

Layer	Recommended choice	Why
Document ingestion	Scheduled connectors for Drive, Notion, Confluence, GitHub, Slack exports	Keeps knowledge current
Chunking	500-1,200 token chunks with document metadata	Balances recall and context size
Embeddings	Use your existing embedding store; keep metadata filters	Retrieval quality depends on clean indexing
Retrieval controller	Neon/Castform-style 4B post-trained model	Cheap query expansion and evidence selection
Final answer model	GPT-5.6 Terra or Claude Sonnet 5	Strong synthesis without top-tier pricing
Premium escalation	GPT-5.6 Sol or GPT-5.2 pro	Use for complex executive or high-risk questions
Cost testing	AI Cost Check	Compare per-run and monthly scenarios

Step-by-step implementation

Step 1: Normalize and tag documents

Ingest documents into a single index with metadata fields:

• source: Drive, Notion, Confluence, GitHub, Slack
• owner: team or department
• created_at and updated_at
• access_level
• document_type
• version
• canonical_url

Do not skip permissions. Retrieval agents are powerful enough to leak sensitive information if access control is applied only after generation.

Step 2: Chunk for retrieval, not storage

Chunk each document into 500-1,200 token segments. Include headings and nearby breadcrumb context. For policies and runbooks, preserve section numbers. For code docs, preserve file paths and function names.

A good chunk title looks like:

Security / Vendor Review / SOC2 evidence requirements / updated 2026-06-14

That title gives the retrieval controller enough context to rank the chunk correctly.

Step 3: Let the 4B model rewrite the query

For each user question, ask the retrieval model to produce:

• A short intent summary
• Required entities
• Time constraints
• 3-6 search queries
• Metadata filters
• Ambiguities to resolve

Example prompt pattern:

“Given the user question, generate retrieval queries and metadata filters. Return JSON with intent, entities, filters, search_queries, and missing_context. Do not answer the question.”

This is a cheap call and should not use a premium model.

Step 4: Retrieve broadly, then rerank cheaply

Run vector search and keyword search for each query. Pull the top 20-50 chunks. Send titles, metadata, and short excerpts to the 4B retrieval model for reranking. Ask it to select:

• Top evidence chunks
• Contradictory chunks
• Outdated chunks
• Gaps requiring another search

This is where cost savings compound. A frontier model may be reviewing tens of thousands of tokens per question if you use it for reranking.

Step 5: Run a gap-check loop

Ask the 4B model: “Can this evidence answer the question with citations?” If no, it generates one more retrieval pass. Cap the loop at 2-3 iterations for normal internal search. Unlimited retrieval agents are a cost and latency trap.

Step 6: Synthesize with a mid-tier model

Send only the selected evidence to GPT-5.6 Terra at $2 input / $12 output per 1M tokens or Claude Sonnet 5 at $2 input / $10 output per 1M tokens. Require citations after each claim.

For simple internal answers, GPT-5.6 Luna at $0.20 input / $1.20 output per 1M tokens can be enough. Use Terra or Sonnet when the answer is cross-functional, sensitive, or likely to be shared.

Step 7: Add confidence and escalation

Return one of three labels:

• Answered: evidence is direct and current
• Partial: evidence is relevant but missing one required detail
• Escalate: conflicting, stale, or high-risk evidence

Escalate to GPT-5.6 Sol only when evidence is complex or the user asks for a decision, not just a lookup.

Cost per run

A typical internal docs query might use:

• 4B retrieval loop: 100K input-equivalent tokens, about $0.005 if it delivers the claimed 100x savings versus Sol input-side retrieval
• Final synthesis on GPT-5.6 Terra: 12K input + 1.5K output
• Terra cost: (12,000 / 1,000,000 * $2) + (1,500 / 1,000,000 * $12) = $0.024 + $0.018 = $0.042
• Total estimated run: $0.047

At 10,000 queries/month, that is about $470/month before infrastructure and embedding costs. If you used GPT-5.6 Sol for the retrieval loop and final synthesis, the same pattern could land around $0.59/run, or $5,900/month.

---

Workflow 2: Build a support investigation agent

Support is the highest-ROI version of agentic retrieval because each answer has business value and the knowledge is scattered. The agent should not replace support reps on day one. It should produce a cited investigation packet that a rep can approve.

What the agent should retrieve

For each ticket, pull evidence from:

• Help center articles
• Internal troubleshooting docs
• Previous similar tickets
• Account plan and feature flags
• Recent incidents
• Product changelog
• Known bugs
• CRM notes
• Customer usage summaries

Step-by-step implementation

Step 1: Create a ticket context object

When a new ticket arrives, create a normalized object:

customer_id
plan
region
product_area
ticket_subject
ticket_body
error_codes
recent_feature_flags
open_incidents
priority

Keep the object short. The retrieval model should use it to search, not receive the entire account history in one prompt.

Step 2: Classify the issue with the 4B model

Ask the retrieval model to classify the ticket into:

• Product area
• Likely issue type
• Required evidence sources
• Search queries
• Customer-specific filters
• Urgency indicators

This call should cost a fraction of a cent. It is routing work, not final reasoning.

Step 3: Search across public and private support knowledge

Run separate searches for:

1. Official docs
2. Internal runbooks
3. Prior solved tickets
4. Changelogs and incidents
5. Account-specific data summaries

Do not blend all sources too early. A prior ticket may be useful but should not override official documentation.

Step 4: Rerank with source priority

Use the 4B model to score evidence with source-specific weights:

Source	Priority	Reason
Active incident	Highest	Current operational truth
Internal runbook	High	Troubleshooting steps
Official docs	High	Customer-safe wording
Similar solved ticket	Medium	Useful but may be outdated
CRM note	Medium	Customer context
Old ticket	Low	Often stale

Ask the model to mark stale evidence and conflicting evidence explicitly.

Step 5: Generate an investigation packet

The final answer model should produce:

• Short diagnosis
• Evidence table
• Recommended next action
• Draft customer response
• Internal-only notes
• Confidence level
• Escalation trigger

Use Claude Sonnet 5 or GPT-5.6 Terra for the packet. Use GPT-5.6 Sol only when the issue is enterprise-critical, legally sensitive, or involves an ambiguous technical root cause.

Step 6: Log outcomes for post-training

When reps accept, edit, or reject the packet, log the event. These labels are valuable for improving retrieval ranking later. The main advantage of an open 4B model is that teams can build a feedback loop around their own corpus.

Cost per support investigation

A medium investigation might use:

• 4B retrieval and reranking: equivalent to 150K Sol input tokens, about $0.0075 at 100x lower retrieval cost
• Final packet on Claude Sonnet 5: 18K input + 2K output
• Sonnet 5 cost: (18,000 / 1,000,000 * $2) + (2,000 / 1,000,000 * $10) = $0.036 + $0.020 = $0.056
• Total estimated run: $0.0635

At 50,000 investigations/month, that is about $3,175/month. A Sol-heavy version could exceed $40,000/month depending on retries and context size. For a high-volume SaaS support team, the model routing decision becomes a gross-margin decision.

---

Model Choice and Cost

The right architecture is not one model. It is a routing stack. Use the cheapest model that can perform each stage reliably, then escalate only when risk or complexity requires it.

Recommended stack by task

Task	Best default	Cheaper fallback	Premium escalation
Query rewriting	Neon/Castform-style 4B retrieval model	GPT-5 nano or Gemini 2.5 Flash-Lite	GPT-5.6 Terra
Reranking chunks	4B retrieval model	DeepSeek V4 Flash	GPT-5.6 Sol
Evidence gap check	4B retrieval model	Mistral Small 4	Claude Sonnet 5
Final internal answer	GPT-5.6 Luna or Claude Haiku 4.5	Gemini 2.5 Flash-Lite	GPT-5.6 Terra
Final customer answer	Claude Sonnet 5 or GPT-5.6 Terra	GPT-5.6 Luna for low-risk replies	GPT-5.6 Sol
Complex reasoning	Gemini 3 Pro, o3, or Terra	GPT-5 mini	GPT-5.2 pro

Real API pricing for common comparison models

Model	Input / 1M tokens	Output / 1M tokens	Context	Best use
GPT-5.6 Sol	$5	$30	1,050,000	Premium synthesis and hard reasoning
GPT-5.6 Terra	$2	$12	1,050,000	Strong production default
GPT-5.6 Luna	$0.20	$1.20	1,050,000	Cheap long-context internal answers
Claude Sonnet 5	$2	$10	1,000,000	Customer-safe synthesis
Gemini 3 Pro	$2	$12	2,000,000	Long-context reasoning
DeepSeek V4 Flash	$0.14	$0.28	1,000,000	Ultra-low-cost retrieval fallback
GPT-5 nano	$0.05	$0.40	128,000	Cheap classification and routing

The 4B open model’s exact hosting cost will depend on hardware, batch size, quantization, and provider margin. Neon and Castform’s public claim is the useful planning anchor: about 100x less than GPT-5.6 Sol on retrieval while matching retrieval performance. For budgeting, model teams should test three scenarios:

Scenario	Retrieval model	Final model	Estimated cost/run	Best for
Ultra-cheap internal lookup	4B retrieval	GPT-5.6 Luna	$0.015-$0.030	Employee docs Q&A
Balanced production	4B retrieval	GPT-5.6 Terra or Claude Sonnet 5	$0.045-$0.080	Support and ops copilots
Premium high-stakes	4B retrieval + Sol escalation	GPT-5.6 Sol	$0.120-$0.300	Legal, exec, enterprise incidents
Frontier-only baseline	GPT-5.6 Sol	GPT-5.6 Sol	$0.500-$1.000+	Eval baseline, not daily routing

For model-by-model price testing, run your own scenarios in AI Cost Check. If you are comparing premium defaults, start with GPT-5 vs Gemini 3 Pro, GPT-5 vs DeepSeek V3.2, and Claude Opus 4.6 vs Gemini 3 Pro.

When the premium model is overkill

GPT-5.6 Sol is overkill for:

• Query rewriting
• Chunk reranking
• Metadata filtering
• Duplicate detection
• “Find the policy section” tasks
• Citation presence checks
• Low-risk internal summaries
• Batch clustering of support tickets

Use Sol when the task requires judgment that changes a business decision. Do not use it as a search operator.

---

Architecture pattern: small retrieval agent, strong answer model

A practical architecture has five layers:

1. Index layer: vector store, keyword index, metadata store, access control.
2. Retrieval controller: 4B model for query planning, reranking, and gap checks.
3. Tool layer: search APIs, SQL summaries, ticket system, CRM, docs.
4. Synthesis layer: mid-tier or premium model for final answer.
5. Evaluation layer: citation accuracy, answer acceptance, human edits, regression tests.

The core loop looks like this:

User question
→ classify intent and risk
→ generate retrieval plan
→ search across allowed sources
→ rerank and select evidence
→ gap check
→ repeat if needed
→ synthesize answer
→ verify citations
→ log outcome

Set hard limits:

• Maximum 3 retrieval iterations
• Maximum 50 chunks reviewed per iteration
• Maximum 10 selected evidence chunks for final synthesis
• Maximum 1 premium escalation per user question
• Cache results for repeated questions

These controls keep agentic retrieval predictable. Without them, agents can silently burn money on long-tail questions.

---

Evaluation: how to know the 4B model is good enough

Do not evaluate retrieval models by vibes. Build a task set from real questions and grade specific retrieval behaviors.

Minimum eval set

Create at least 200 questions across:

• Simple lookup questions
• Multi-hop questions
• Stale policy traps
• Conflicting evidence cases
• Access-controlled documents
• Customer-specific support issues
• Questions with no answer
• Ambiguous questions requiring clarification

For each question, label:

• Required source documents
• Acceptable evidence chunks
• Forbidden stale chunks
• Correct final answer
• Whether escalation is required

Metrics that matter

Metric	Target	Why
Evidence recall	90%+	Did the model find the right sources?
Citation precision	85%+	Are citations actually supporting the claim?
Stale-source rejection	95%+	Old docs cause bad answers
No-answer detection	90%+	Prevents hallucinated answers
Escalation accuracy	90%+	Routes high-risk tasks correctly
Cost per accepted answer	Lower than baseline	Measures real production value

Compare the 4B retrieval model against GPT-5.6 Sol on the retrieval-only portion first. Then compare the full system: 4B retrieval plus Terra/Sonnet synthesis versus Sol-only. The claim that a 4B model matches Sol on retrieval is only useful if it holds on your corpus, not just public retrieval benchmarks.

---

Risks and limits

The biggest risk is false confidence. A retrieval specialist can be excellent at finding plausible chunks while still missing a newer policy, a hidden permission boundary, or an edge-case source. Teams should deploy with guardrails.

Risk 1: Corpus quality dominates model quality

If documents are stale, duplicated, or poorly permissioned, the retrieval model will produce polished answers from bad evidence. Fix document ownership, canonical URLs, and update timestamps before scaling.

Risk 2: Open model hosting has operational overhead

A 4B model is much easier to host than a frontier model, but it still needs monitoring, batching, autoscaling, GPU or accelerator planning, and latency testing. If your team cannot operate inference reliably, use a managed provider or a cheap API fallback such as DeepSeek V4 Flash.

Risk 3: Retrieval matching is not final reasoning

Matching GPT-5.6 Sol on retrieval does not mean matching it on synthesis, planning, coding, legal analysis, or strategic judgment. Keep task boundaries clean.

Risk 4: Evaluation drift

Internal corpora change every day. Add regression tests for common questions and high-risk policies. Re-run evals after major document migrations, product launches, or support taxonomy changes.

Risk 5: Permission leakage

Agentic search crosses systems. Enforce access control at retrieval time, not just display time. The model should never see chunks the user is not allowed to access.

---

When teams should still pay for premium models

Premium models remain worth the cost when the answer carries high downside risk or requires deep synthesis.

Use GPT-5.6 Sol, GPT-5.2 pro, o3, or Gemini 3 Pro for:

• Legal or regulatory interpretation
• Security incident decisions
• Enterprise customer escalations
• Complex technical root-cause analysis
• Multi-document contradictions where the final answer needs judgment
• Board or executive briefings
• Code changes that touch production systems
• Financial or procurement decisions

The winning pattern is selective escalation. Let the 4B model gather and organize the evidence. Let the premium model decide what it means.

A good escalation rule is:

• Low risk + direct evidence: 4B retrieval + cheap synthesis
• Medium risk + cross-source evidence: 4B retrieval + Terra/Sonnet synthesis
• High risk + ambiguity or external impact: 4B retrieval + premium synthesis + human approval

This keeps quality high without paying premium rates for every intermediate step.

---

Practical rollout plan for founders and product teams

Start with one workflow, not a platform. The best first deployment is usually internal docs search or support investigation because both have measurable outcomes.

Week 1: Build the eval set

Collect 200 real questions and label evidence. Include failed searches from your current system. Define cost and latency baselines.

Week 2: Index and permission the corpus

Ingest documents, chunk them cleanly, and enforce access control. Add metadata for owner, freshness, source, and document type.

Week 3: Implement the retrieval loop

Use the 4B model for query planning, reranking, and gap checks. Add loop caps and logging from day one.

Week 4: Add synthesis and escalation

Use GPT-5.6 Luna for low-risk internal answers, Terra or Claude Sonnet 5 for production answers, and Sol only for escalations.

Week 5: Run shadow mode

Show answers to employees or support reps without auto-sending. Track acceptance rate, citation accuracy, and missing evidence.

Week 6: Launch with guardrails

Enable production for low-risk use cases. Keep human approval for customer-facing and high-risk answers.

The most important metric is not raw model accuracy. It is cost per accepted answer. A cheap retrieval agent that produces useful evidence packets can outperform a more expensive chatbot that writes fluent but unverifiable answers.

---

Frequently asked questions

What is Neon and Castform’s 4B retrieval model claim?

Neon and Castform say a post-trained 4B open model can match GPT-5.6 Sol on retrieval tasks while costing about 100x less. The practical takeaway is that teams can route query planning, reranking, and evidence checking to a small specialist model instead of using a frontier model for every retrieval step.

How much can agentic retrieval cost on frontier models?

A multi-hop retrieval run can use 30,000-150,000+ input tokens across search, reranking, gap checks, and final synthesis. On GPT-5.6 Sol, 100,000 input tokens costs $0.50 before output; at 100,000 monthly runs, that input-side retrieval alone can reach $50,000/month.

What should teams build first with a cheap retrieval-specialist model?

Start with an internal docs agent or support investigation copilot. Both workflows benefit from multi-hop search, citations, and evidence ranking, and both can be evaluated with real questions before customer-facing deployment.

Which models should I pair with a 4B retrieval model?

Use the 4B model for retrieval control, then synthesize with GPT-5.6 Terra at $2/$12 per 1M tokens or Claude Sonnet 5 at $2/$10 per 1M tokens. Use GPT-5.6 Luna for cheap internal answers and GPT-5.6 Sol for high-risk escalations.

When should I still use GPT-5.6 Sol for retrieval or answers?

Use GPT-5.6 Sol when the task is high-risk, ambiguous, or requires premium reasoning: legal analysis, security incidents, enterprise escalations, executive briefings, or complex technical decisions. For routine query rewriting and reranking, a specialized 4B model or cheaper fallback is the better default.

---

Build the retrieval stack, then price it before launch

The Neon and Castform news points to a broader shift: agentic search is becoming a systems problem, not just a frontier-model problem. The best teams will combine cheap retrieval specialists, strong synthesis models, strict evaluation, and clear escalation rules.

Before you ship, price the workflow at 1,000, 10,000, and 100,000 runs per month. Compare a frontier-only baseline against a routed stack using the AI Cost Check calculator. For model selection, review GPT-5.6 Sol, GPT-5.6 Terra, Claude Sonnet 5, and lower-cost fallbacks such as DeepSeek V4 Flash.

The next production advantage in AI search will not come from sending every token to the biggest model. It will come from knowing which tokens deserve it.