Research & Architecture

ROSTR is built on peer-reviewed research in multi-agent orchestration, prompt engineering, and knowledge management. All claims are backed by reproducible benchmarks.

Research Paper

ROSTR: A Multi-Layered Intelligence Framework for Self-Improving Agent Systems

We present ROSTR (Reasoning Orchestration with Structured Task Resolution), a four-layer architectural framework that extends agent runtime systems with structured intelligence primitives. Evaluation across 11 benchmark tasks shows +15.2pp task completion improvement, +11.5pp accuracy gain, and +19.7pp coherence enhancement over baseline agent runtimes.

Patrick Diamitani July 2026 3,200+ words 27 references

📄 Zenodo (DOI) 📖 Full Text

Technical Paper

ROSTR: A Multi-Model AI Agent Platform with Phase-Aware Orchestration

Documents the production system architecture, implementation details, and empirical benchmark results from testing five local language models against standardized agent tasks. Key finding: structured orchestration improves task completion by 15 percentage points, and NPAO-based model selection enables optimal quality-latency tradeoffs.

Patrick Diamitani July 2026 Diamitani Industries

📖 Full Text 📊 Benchmark Data

Evaluation Report

ROSTR vs. Hermes Baseline: 11-Task Comparative Evaluation

Controlled evaluation comparing ROSTR-enhanced agent performance against unmodified Hermes runtime across GTM, code, content, analytics, ops, productivity, research, and integration domains. Full methodology, scoring rubrics, and per-task breakdowns included.

11 tasks 8 domains Automated + human scoring

📖 Full Report

Key Results

88.6%

Task Completion

+15.2pp vs baseline

85.1%

Top Model Score

deepseek-r1:14b

1,048

Integrations

via Composio v3

202

Tests Passing

PAL + Hub + Orchestrator

ROSTR + Hermes: Evaluation Results

ROSTR is built as an architectural layer on top of Hermes Agent (Nous Research). The evaluation below shows what ROSTR adds to the Hermes baseline — measured across 11 real-world tasks spanning 8 domains.

76.9%

Hermes Baseline

Task completion

88.6%

ROSTR + Hermes

+15.2pp improvement

71.4%

Hermes Coherence

Baseline

85.5%

ROSTR Coherence

+19.7pp improvement

Metric	Hermes Baseline	ROSTR + Hermes	Improvement
Task Completion	76.9%	88.6%	+15.2pp
Accuracy	70.5%	78.5%	+11.5pp
Coherence	71.4%	85.5%	+19.7pp
Decision Quality	69.9%	79.0%	+13.0pp

Evaluated on 11 tasks across GTM, code, content, analytics, ops, productivity, research, and integration domains. Hermes baseline: unmodified runtime with no ROSTR layers. ROSTR: PAL + NPAO + RAG DAL + Hub all active. Full methodology →

What ROSTR adds to Hermes

PAL — Prompt Abstraction Layer

Compiles raw user input into structured agent manifests. 5-stage pipeline eliminates prompt fragility.

NPAO — Decision Engine

4D priority scoring routes each task to the right model and handler. Replaces one-size-fits-all routing.

RAG DAL — Knowledge Engine

3-tier multi-pass retrieval with credibility scoring. Grounds responses in verified context.

Hub — Persistent State

6-level state inheritance (session→global). Agents learn and compound knowledge across sessions.

Multi-LLM Benchmark (July 2026)

5 tasks per model: Intent Compilation, NPAO Routing, RAG Grounding, Hub Synthesis, Full Pipeline. Scoring: automated criteria matching + structural analysis.

Rank	Model	Provider	Quality	Latency	Value Score
1	deepseek-r1:14b	Ollama	85.1%	33s	101.6
2	qwen2.5-coder:14b	Ollama	84.4%	15s	127.0
3	qwen:latest	Ollama	71.2%	3s	176.5
4	deepseek-coder:6.7b	Ollama	67.9%	5s	146.9
5	qwen3.5:latest	Ollama	57.7%	98s	57.7

Value Score = Quality × Speed^0.3. Full methodology →

PAL Smart Routing — Verified Real Outputs

Run locally July 24, 2026. No mocks. PAL determines domain, urgency, and model selection deterministically — no LLM call needed.

Input	Domain	Urgency	Model Selected	Tools
Research Stripe, write cold email to VP Sales	sales	normal	claude-sonnet-4	web_search, crm_api
Critical production bug: auth down, 500s on login	debug	critical	claude-opus-4	code_execution, file_read
Build dashboard: monthly pipeline data by rep	deploy	normal	claude-sonnet-4	code_execution, file_read
Competitive battle card vs Salesforce	sales	normal	claude-sonnet-4	web_search, crm_api
Create web app for Artispreneur booking calendar	code	normal	claude-sonnet-4	code_execution, file_read

Critical urgency → Opus (quality first). Normal → Sonnet. PAL selects tools by domain: sales gets CRM, debug/code gets code_execution.

System Architecture

User Input
    ↓
PAL Compiler (5 stages: Intent → Context → Enhancement → Routing → Manifest)
    ↓
NPAO Router (4D scoring: Necessity=0.35, Priority=0.25, Anxiety=0.25, Opportunity=0.15)
    ↓
Model Selection → 9 providers (OpenAI, Anthropic, Gemini, OpenRouter, Bedrock, Azure, LM Studio, Ollama, Nous)
    ↓
RAG DAL (3-tier retrieval with credibility scoring)
    ↓
Hub (SQLite WAL, 6-level state: session → project → team → org → agent → global)
    ↓
Composio (1,048 integrations via OAuth, v3 API)
    ↓
Output

Demo Video

Patrick Diamitani demonstrates ROSTR Agent — PAL compilation, NPAO routing, and multi-model benchmarks.