Chessarena.ai is an open-source platform for exploring and benchmarking how large language models (LLMs) perform in chess. Rather than focusing on simple win/loss results, Chessarena.ai measures move quality and game insight providing uniquely meaningful feedback on how much AI models truly "understand" chess.

See ChessArena AI in action - watch AI models battle it out with real-time move evaluation and scoring

Modern LLMs struggle to genuinely win at chess: most LLM-based games end in draws, and true chess mastery still eludes these models.

That's why we score move-by-move quality and insight rather than simply tracking wins!

Every single move played by an LLM is immediately:

• Evaluated by Stockfish, the strongest open-source chess engine.

• Compared to Stockfish's recommended best move.

• The difference ("move swing") is recorded in centipawns.

• If the move swing is >100 centipawns, we count it as a blunder.

This system produces a leaderboard rewarding the most insightful and accurate play, rather than luck or brute force.

Click the image below to watch the demo:

• LLM Chess Leaderboard: See how multiple language models compare, move-by-move.
• Real-Time Streaming: Built on Motia Streams, every move and score updates live.
• Open-Source, Event-Driven: Built with Motia for easy customization, real-time features, and code-first clarity.

• Node.js (v18 or higher)
• PNPM
• Python 3.x
• Stockfish Chess Engine

git clone https://github.com/MotiaDev/chessarena-ai.git
cd chessarena-ai
pnpm install

brew install stockfish

pnpm install-stockfish <platform>

Supported platforms:

• linux-x86
• mac-m1

Download directly from stockfishchess.org and install according to your platform's instructions.