Rullst AI

Vision preserved: capability-typed schema support, local-model boundaries, and autonomous-agent ideas remain itemized with an implementation opinion in the capability ledger.

rullst-ai is a provider-agnostic LLM client with mandatory outbound prompt-injection checks, PII masking, deterministic offline fixtures, JSON mode, explicit JSON Schema output, and a bounded tenant-aware RAG pipeline.

Provider capabilities

Unsupported capabilities return AiError::UnsupportedCapability; the client does not silently switch to an unrelated endpoint or represent a fixture as a live-provider result.

Anthropic requests allow up to 16,000 output tokens unless AnthropicProvider::with_max_tokens sets another limit (the Messages API requires one, and adaptive thinking counts toward it; a longer reply may also need a longer with_request_timeout). A reply the API reports as cut short (stop_reason max_tokens or model_context_window_exceeded) or declined (refusal) returns AiError::ApiError instead of the partial text. OpenAiProvider sends no output limit on text or vision requests, so the model’s own limit applies, and a reply whose finish_reason is length or content_filter likewise returns AiError::ApiError.

OpenAiCompatibleProvider covers servers implementing the named OpenAI /chat/completions and optional /embeddings shapes. It defaults to chat-only; vision, embeddings, JSON mode, and JSON Schema must be declared for the exact endpoint/model pair. try_local permits unauthenticated HTTP only on a literal loopback IP, try_local_with_bearer adds explicit local authentication, and try_cloud requires HTTPS plus a Bearer credential. All three disable redirects and environment proxies and bound response bodies. Different protocols use a custom public AiProvider, not an arbitrary-HTTP mode.

Guarded client

#![allow(unused)]
fn main() {
use rullst_ai::{AiClient, AiError, providers::openai::OpenAiProvider};

async fn answer(api_key: String, user_text: &str) -> Result<String, AiError> {
    let client = AiClient::new(OpenAiProvider::new(api_key));
    client
        .chat()
        .system("Answer concisely.")
        .user(user_text)
        .send()
        .await
}
}

AiClient::prompt, chat, vision, embedding, JSON, and structured-output calls run the same guardrail stage before provider dispatch. Built-in providers repeat the check on direct trait calls. Call custom AiProvider implementations through AiClient when the application needs the same mandatory boundary.

The current guardrail blocks deterministic injection patterns, provider delimiter tokens, external Markdown image beacons (inline or reference images with a remote or unclassifiable destination, read with CommonMark escapes and character references and outside code spans and code blocks; an unmatched ASCII reference such as vec![x] is literal text; raw HTML <img> is not inspected), and invisible Unicode controls, including zero-width, bidirectional embedding/isolate, tag and other default-ignorable characters with no ordinary use in text. Soft hyphens, bidirectional marks and emoji variation selectors are removed before phrase matching instead of being blocked. Check-digit-valid CPF/CNPJ numbers (canonical formatted or unformatted), card-like digit runs and email usernames are masked before outbound transmission; alphanumeric CNPJs and other identifiers are not recognized. This is a bounded heuristic control, not proof that arbitrary input or model output is safe; authorization, tool permissions, output encoding, and domain validation remain application responsibilities.

Adaptive evaluation runner

AdaptiveAiEvaluator<P> runs application-defined multi-turn strategies over static provider dispatch and the mandatory prompt guardrail. One scenario is limited to 32 turns, 16 KiB per generated prompt and 2 MiB per response, with an independent deadline and explicit cancellation. Each observation exposes a bounded response only while the synchronous strategy chooses pass, fail, inconclusive or its next prompt.

The versioned JSON report keeps the exact caller-supplied suite/subject labels, provider name, status, terminal code and per-turn byte counts/outcomes. It does not retain prompts, responses or provider error bodies and can itself be sent through AuditDeliveryClient. The subject label is not automatic model discovery, and the deterministic offline runner test is not live-model evidence. Operators must version their scenario code/corpus and execute it against each exact model/configuration they intend to approve; no passing suite proves universal safety, groundedness or jailbreak resistance.

Authenticated audit delivery

AuditDeliveryClient can export an application-minimized RAG, tool or provider event to one exact endpoint. Cloud configuration requires HTTPS; local development allows HTTP(S) only on a literal loopback IP. Each JSON envelope is limited to 16 KiB and HMAC-SHA256 authenticates the exact bytes together with the key ID and Unix-millisecond timestamp. A caller-generated event ID remains stable across at most five attempts, and success requires a closed JSON acknowledgement that repeats that ID. Cancellation covers request, response and retry waits; empty or mock_* keys select a deterministic offline fixture.

Only transport/deadline failures, HTTP 429 and HTTP 5xx are retryable. Because a timeout can happen after remote acceptance, the receiver must enforce idempotency by event ID as well as signature/freshness validation. The client does not minimize arbitrary serialized data, retain a durable outbox, rotate keys, authorize operators or provide a SIEM receiver. Those remain explicit application/deployment responsibilities.

Token usage (v13)

AiProvider::chat_with_usage, ChatBuilder::send_with_usage and StreamSummary::usage() expose the token counts a provider reports (input, output, total and cached input) from its documented response fields. Absent counts stay None; nothing is estimated or priced.

Bounded streaming and cancellation

StreamingAiClient<P> is a static-dispatch extension for genuinely incremental providers. The OpenAI-compatible adapter implements the strict SSE path only when the exact endpoint/model configuration opts into with_streaming(). It checks the prompt before I/O, requires text/event-stream and [DONE], bounds the raw response, chunk count, each chunk and aggregate output, and rejects malformed or truncated events and a finish_reason of length or content_filter with AiError::ApiError.

Anthropic and Gemini stream through their native SSE protocols (v13) with the same bounds and cancellation; a truncated or withheld reply fails in both the streaming and non-streaming paths.

AiCancellation is cloneable and aborts a supported request while it is waiting for headers or another body chunk. That drops the local request future; it cannot prove that an upstream server stopped work or billing. The other built-in transports and ordinary non-streaming calls retain deadline/drop semantics until their different wire protocols have equivalent tests.

Tenant-aware chat memory

StatefulChat<M> is a static-dispatch orchestration boundary over ChatMemory. It binds every conversation to trusted TenantContext, loads a bounded even history, calls the guarded client, and atomically appends the user and assistant halves after successful generation. A response that the guardrail would block when the history is replayed is rejected as StatefulChatError::Generation(AiError::BlockedByFirewall(_)) and is not stored. InMemoryChatMemory is a bounded deterministic offline store.

With the opt-in umbrella ai-sql-memory feature, SqlChatMemory supplies a dedicated SQLx Any pool and fixed schema for SQLite, PostgreSQL, MySQL, and MariaDB. Its revision compare-and-swap rejects stale cross-process writers. It does not retry the provider call, because doing so could duplicate cost or side effects. Tenant and conversation IDs compare case-sensitively on every backend: new MySQL/MariaDB tables use ascii_bin key columns, and every tenant-scoped MySQL/MariaDB statement also compares the key byte-exactly, so an older case-insensitive table fails closed until the migration in the crate README is applied. The AI integration tutorial shows the complete setup and application-owned security/retention boundary.

Tenant-aware RAG pipeline

RagPipeline::answer performs guarded embedding, calls a static-dispatch RagRetriever, applies per-document and total Unicode-safe budgets, guards and masks every selected passage, generates a grounded response, returns source metadata, and records one terminal audit event. It requires a trusted TenantContext, rejects differently tagged documents, and fails with RagError::NoContext instead of asking the model to answer without retrieved evidence.

The bundled InMemoryRagRetriever is a bounded tenant-partitioned cosine index for offline tests, development, and small ephemeral datasets. It is neither durable nor distributed. A production application can implement RagRetriever over ORM pgvector or Qdrant, but that adapter must bind the trusted tenant and ownership predicates in the authoritative datastore. The pipeline’s tag check is defense in depth, not a replacement for datastore authorization.

The mandatory audit event stores the tenant, a SHA-256 correlation digest, counts, character budget, and outcome. It deliberately omits raw questions, documents, embeddings, provider bodies, and model answers. The digest is not encryption and can be guessed for low-entropy questions. DurableRagAuditTrail and DurableToolAuditTrail provide bounded synchronous local files with distinct version headers, SHA-256 frame integrity and restart/quota/corruption validation. They are single-process writers and do not supply authenticity, rotation, retention, backup or external delivery; multi-instance deployments can implement the same audit traits over their destination.

Follow the tenant-bound RAG tutorial for the complete offline flow and production integration boundary.

Versioned offline evals

The packaged evals/guardrails-v1.json corpus freezes deterministic injection, jailbreak, and PII regressions. The repository gate validates unique IDs and required categories, then runs every case across all six built-in transports in offline mode. It is deliberately not presented as a safety benchmark: adaptive attacks, tool selection, hallucination, and live provider/model versions require separate eval suites.

Strict egress policy

EgressPolicy::strict() starts with no permitted destination. After an exact host allowlist is configured, EgressFetcher permits HTTPS and explicit ports, blocks credentials/local/private/metadata/reserved addresses, validates every DNS answer, pins those answers in a proxy-free reqwest client, verifies the connected peer, follows redirects only after repeating policy, and bounds time and streamed bytes. The fetcher is opt-in: it cannot protect arbitrary application/provider HTTP clients, and tenant authorization, content schema and data minimization remain caller contracts.

Offline mode

OpenAI, Gemini, Anthropic, DeepSeek, and compatible cloud endpoints use deterministic offline mode when their API key is empty or begins with mock_. Ollama uses an empty or mock_* host; the compatible adapter also exposes an explicit mock constructor. Plain try_local is deliberately live because no credential is its valid loopback configuration. Offline branches return before HTTP dispatch and cover each capability the provider declares. Unsupported capabilities remain typed errors in offline mode.

AiClient::auto() checks OPENAI_API_KEY (with OPENAI_BASE_URL and OPENAI_MODEL for a compatible endpoint), ANTHROPIC_API_KEY, GEMINI_API_KEY, DEEPSEEK_API_KEY, GROQ_API_KEY (with GROQ_MODEL) and OLLAMA_HOST (with OLLAMA_MODEL, default llama3). If none is configured, it selects an offline OpenAI fixture; it does not probe localhost implicitly. OLLAMA_HOST (and the host given to OllamaProvider::new) is read the way Ollama reads it: a scheme-less 127.0.0.1:11434 or localhost means http and port 11434 unless a port is named.

JSON mode and structured output

JSON mode requests a parseable JSON value and deserializes it in Rust:

#![allow(unused)]
fn main() {
use rullst_ai::{AiClient, AiError, providers::openai::OpenAiProvider};
async fn example() -> Result<(), AiError> {
let client = AiClient::new(OpenAiProvider::new("mock_local"));
let value: serde_json::Value = client.json_prompt("Summarize this record").await?;
let _ = value;
Ok(())
}
}

Native structured output requires an explicit schema and fails when the provider cannot enforce it:

#![allow(unused)]
fn main() {
use rullst_ai::{AiClient, AiError, StructuredOutputSchema, providers::openai::OpenAiProvider};
async fn example() -> Result<(), AiError> {
let client = AiClient::new(OpenAiProvider::new("mock_local"));
let schema = StructuredOutputSchema::new("answer", serde_json::json!({
    "type": "object",
    "properties": {"ok": {"type": "boolean"}},
    "required": ["ok"],
    "additionalProperties": false
}))?;
let value: serde_json::Value = client
    .structured_prompt_with_schema("Evaluate the input", &schema)
    .await?;
let _ = value;
Ok(())
}
}

Provider-side schema enforcement and Rust deserialization do not replace application-specific semantic validation. In offline mock mode the deterministic fixture follows at most 16 nesting levels, 32 items per array and 4,096 generated values; larger schemas fail with AiError::InvalidSchema.

Current boundaries

Streaming for the OpenAI, DeepSeek and Ollama transports (outside the compatible adapter), provider-native tool execution loops, first-party external vector-store RagRetriever adapters, maintained domain-specific evaluation corpora, and compile-time schema derivation remain roadmap work. The SQL memory does not supply raw-text encryption, ownership within a tenant, retention or provider auditing; the in-memory vector utilities and tool registry do not create an authorization boundary by themselves. Authenticated audit delivery does not replace a durable outbox or certify a receiver’s retention, availability or security operations.