ABLITERATEDAIAPI Documentation ↗

Developer guides / Integration

Using an OpenAI-compatible inference API

OpenAI-compatible APIs let you use a familiar request and response format while changing the provider that serves the model. Compatibility can reduce integration work, but it does not guarantee identical model behavior, tokenization, tools, streaming support or capacity across providers.

1. Choose a real model and confirm availability

Start by checking the current AbliteratedAI model and service status. An advertised model ID tells your request which backend to use; it does not guarantee immediate inference capacity. Check access requirements and billing before sending requests. The separately listed 2.8T model is an upcoming offering, not a promise of live access.

2. Keep API credentials off the client

Use a server-side environment variable for the API key. Never paste a production key into browser JavaScript, a public Git repository or a shared screenshot. Rotate a credential when exposure is suspected. Grant access at the smallest practical project scope.

export ABLITERATED_API_KEY="your-secret-key"

This shell example is for a local session; production deployments should use their platform's protected secrets facility.

3. Make a request with a familiar SDK

The Python OpenAI SDK accepts a custom base URL. Set it to the AbliteratedAI API endpoint, supply your key from the environment and pass the exact model ID. Install the SDK with pip install openai.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.abliteratedai.dev/v1",
    api_key=os.environ["ABLITERATED_API_KEY"],
    max_retries=0,
)

response = client.chat.completions.create(
    model="abliterated-235b-a22b-fp8",
    messages=[{"role": "user", "content": "Explain a basic API retry strategy."}],
    max_tokens=128,
)
print(response.choices[0].message.content)

Requests may consume prepaid credits. The code is an integration example, not a live benchmark or proof of current availability.

4. Handle provider warm-up and HTTP errors

Some hosted GPU backends sleep during quiet periods. If the API returns HTTP 503 with a provider warm-up error and a Retry-After header, pause for that interval before retrying. Use a bounded retry policy with backoff and jitter for transient failures. Do not blindly repeat non-idempotent requests or retry authentication failures. Log the request ID rather than secrets or full user prompts.

Different error types need different responses: 401 suggests checking credentials, 429 suggests reducing request rate, and 5xx suggests a temporary backend issue. Consult the API error reference for this platform's actual response semantics.

5. Evaluate fit for your application

Test with representative prompts, latency measurements and cost limits before moving production traffic. Compare output quality, refusal behavior, supported context windows, throughput and response consistency under load. OpenAI-compatible transport does not imply identical safety behavior or feature support.

Start with a small canary integration, verify authentication and metering, and then expand usage once the application's requirements are met. Keep client libraries pinned and monitor provider changes.

Next steps

Explore the quickstart, inspect the documentation, or open the developer console to create a project and API key.

Open developer console →