Multimodal (image input)

AI Nexus currently supports image input (image understanding): attach images to a request and have the model describe, recognize or analyze them. Image generation, audio and video are outside the scope of these docs.

Image input support depends on the model itself. Check the model details on the model square or try it in the Playground first.

Replace your-model-id in the samples with a model ID from the model square (or call GET /v1/models).

Option 1: Chat Completions (image_url)Copy Anchor Link

Use an array for messages[].content to combine text and images. image_url.url can be a publicly reachable image URL or a Base64 data: URL.

import base64
import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["MYAIEXP_API_KEY"],
    base_url="https://ai.myaiexp.com/v1",
)

# read a local image and turn it into a data: URL
with open("photo.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="your-model-id",  # replace with a model ID from the model square
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in one sentence."},
                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64}"}},
            ],
        }
    ],
)

print(resp.choices[0].message.content)

Replace https://example.com/your-image.jpg with your own public image URL.

Option 2: Anthropic Messages (Claude-native format)Copy Anchor Link

When calling a model that supports the Claude-native format through /v1/messages, images use the native Anthropic image content block. source takes two forms:

• {"type": "base64", "media_type": "image/jpeg", "data": "<base64>"}: send the image data directly; data has no data:image/...;base64, prefix.
• {"type": "url", "url": "https://..."}: send a public image URL.

import base64
import os

import anthropic

client = anthropic.Anthropic(
    api_key=os.environ["MYAIEXP_API_KEY"],
    base_url="https://ai.myaiexp.com",  # note: no /v1
)

with open("photo.jpg", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

message = client.messages.create(
    model="your-model-id",  # replace with a model ID from the model square
    max_tokens=512,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image", "source": {"type": "base64", "media_type": "image/jpeg", "data": b64}},
                {"type": "text", "text": "Describe this image in one sentence."},
            ],
        }
    ],
)

print(message.content[0].text)

NotesCopy Anchor Link

• Images are converted into input tokens according to the model's rules and billed as input; larger images cost more, so downscale to the resolution you actually need before sending.
• Common formats are JPEG, PNG, GIF and WebP; media_type must match the actual image format.
• When using an image URL, the image must be directly reachable from the public internet (no login, no hotlink protection); otherwise use Base64.
• You can include several images in one message; placing images before the text question is recommended.
• The input tokens actually consumed by each request are shown in Usage Logs.