Documentation

What it is

VSArena is a browser stacking work-cell plus a WebSocket harness. Physics steps in Rapier compiled to WebAssembly at 60 Hz. You watch in the browser. A policy talks to the harness process. Scoring uses privileged poses internally. The VLA track does not send them to the policy. That track mirrors the OpenVLA / RT interface: pixels plus an instruction in, actions out.

Research

The systems report is public. Title: VSArena: A Browser-Native Public Arena for Vision-Language-Action Policy Evaluation. It describes the design, the hello / state / action / result protocol, the trust boundary, continuous Elo against a house rating of 1200 (full stack only), per-result provenance, and a published failure taxonomy. It is a draft. Hugging Face Paper Pages index arXiv IDs; the Space exists so the write-up can be read without that gate. The empirical study across seeds and learned VLA policies is in preparation.

How to cite

Cite the Space until a preprint ID exists. The report is an ONISCOR systems draft, not a peer-reviewed paper.

@techreport{vsarena2026,
  title       = {VSArena: A Browser-Native Public Arena for Vision-Language-Action Policy Evaluation},
  author      = {{ONISCOR}},
  institution = {ONISCOR},
  year        = {2026},
  type        = {Systems report},
  url         = {https://huggingface.co/spaces/AranKair/vsarena-paper},
  note        = {Draft. Empirical study in preparation},
}

Quickstart

Python 3.11+. From the repository root. Full Beginner / Researcher walkthrough: /submit.

pip install -e sdk/python
python -m vsarena

# live WebSocket client
pip install -e "sdk/python[live]"

import base64
from vsarena import Agent, run_match

class MyAgent(Agent):
    def act(self, state: dict) -> dict:
        instruction = state["instruction"]
        img = state["images"]["scene"]
        rgb = base64.b64decode(img["b64"])  # len = width * height * 3
        joints = state["scene"]["joint_states"]
        # your vision-language policy — do not expect state["scene"]["blocks"]
        return {"joint_targets": dict(joints), "gripper_state": "open"}
        # or: return {"ee_delta": {"dx": 0.01, "dy": 0.0, "dz": 0.0}, "gripper_state": "open"}

print(run_match(MyAgent(), dry_run=True, mode="vla"))

from vsarena import ColorSeek, run_match

print(run_match(ColorSeek(), dry_run=True, mode="vla"))

import os
from vsarena import Agent, run_match

class MyAgent(Agent):
    def act(self, state: dict) -> dict:
        joints = state["scene"]["joint_states"]
        _ = state["instruction"]
        _ = state["images"]["scene"]
        return {"joint_targets": dict(joints), "gripper_state": "open"}

run_match(
    MyAgent(),
    dry_run=False,
    mode="vla",
    api_key=os.environ["VSARENA_API_KEY"],
    agent_name=os.environ.get("VSARENA_AGENT_NAME", "MyAgent"),
)

VLA vs state

Physics stays at 60 Hz. Two observation tracks share that world. Scoring always uses privileged poses.

How ELO is written

Ratings are Elo against a fixed house rating of 1200. The outcome is binary: full stack = 1, anything else = 0 (partial towers do not move ELO). elo_delta is computed on ingest: POST /api/matches with header x-vsarena-ingest. Only the VLA track writes the board. The browser cannot. In-browser demos are for watching and debugging.

Protocol

Match loop: hello → state → action → … → result. JSON over WebSocket. VLA actions have a 2 s budget at 5 Hz; the state track is 150 ms at 20 Hz. A malformed action never touches the physics engine. Every official result carries provenance (product version, Rapier version, physics rate, git SHA, observation mode, latency budget, scene identifier, seed) and a dotted failure code.

{
  "type": "hello",
  "api_key": "…",
  "task": "block_stacking",
  "mode": "vla",
  "agent": "my-policy"
}

Demos

In Studio, Record demo captures the VLA camera at 5 Hz plus joint_targets, ee_delta, and gripper. Cube poses are not stored. Format vsarena-demo-v1.

If it fails

• Timeout. VLA actions have two seconds at 5 Hz. Return something every tick even if the policy is still thinking — hold the last joints.
• No ELO on the leaderboard. Dry-run never writes. Live needs sdk/python[live], VSARENA_API_KEY, VSARENA_HARNESS_URL, and HARNESS_INGEST_SECRET on app + harness. A browser POST is rejected on purpose.
• Hello rejected. Sign in, copy a fresh key, and register the agent name you pass as agent_name.
• Empty cubes in state. That is the VLA track. Do not parse scene.blocks. Use the image and the instruction.
• Product version on /health lags the local package. Redeploy the hosted harness after a version bump.

Related work

VSArena is not a replacement for those suites. RLBench, Meta-World, robosuite, ManiSkill2, CALVIN, LIBERO and Ravens are richer task sets; custody of the score stays with the contestant. SIMPLER measures real-to-sim transfer; VSArena makes no real-robot claim. Open X-Embodiment, OpenVLA, RT-1, RT-2 and LeRobot are why a shared VLA venue is feasible. MuJoCo and Isaac Gym are more faithful physics; Rapier is used because it runs in the tab. Elo, Bradley–Terry and Chatbot Arena are the rating precedent. Full bibliography, as cited in the systems report: