Evaluating LLMs on their Wordle-solving capabilities
| Rank | Model | Provider | Success Rate | Avg Guesses | Avg Cost |
|---|---|---|---|---|---|
| 1 | gpt-5 | openai |
|
3.59 | $0.20 |
| 2 | claude-sonnet-4.6 | anthropic |
|
3.7 | $0.14 |
| 3 | gemini-3.8-flash |
|
3.45 | $0.04 | |
| 4 | gpt-5.4 | openai |
|
3.57 | $0.09 |
| 5 | claude-fable-5 | anthropic |
|
3.51 | $0.18 |
| 6 | gemini-3.7-flash |
|
3.53 | $0.01 | |
| 7 | gpt-5.5 | openai |
|
3.57 | $0.22 |
| 8 | gpt-5.6-sol | openai |
|
3.53 | $0.07 |
| 9 | qwen3.8-max | qwen |
|
3.55 | $0.25 |
| 10 | claude-opus-5 | anthropic |
|
3.6 | $0.11 |
| 11 | claude-sonnet-5 | anthropic |
|
3.74 | $0.04 |
| 12 | muse-spark-1.2 | meta |
|
3.62 | $0.08 |
| 13 | gpt-5.6-terra-pro | openai |
|
3.5 | $0.12 |
| 14 | qwen3.7-plus | qwen |
|
3.66 | $0.02 |
| 15 | claude-opus-4.6 | anthropic |
|
3.58 | $0.14 |
| 16 | gpt-5.1 | openai |
|
3.6 | $0.13 |
| 17 | grok-4.20-beta | x-ai |
|
3.68 | $0.07 |
| 18 | claude-opus-4.8 | anthropic |
|
3.51 | $0.08 |
| 19 | gemini-3-pro-preview |
|
3.53 | $0.14 | |
| 20 | gpt-5-mini | openai |
|
3.62 | $0.04 |
| 21 | gpt-5.6-luna | openai |
|
3.69 | $0.01 |
| 22 | gpt-5.6-sol-pro | openai |
|
3.5 | $0.34 |
| 23 | claude-opus-4.5 | anthropic |
|
3.62 | $0.29 |
| 24 | gemini-3.1-flash-lite-preview |
|
3.69 | $0.01 | |
| 25 | qwen3.7-max | qwen |
|
3.53 | $0.06 |
| 26 | grok-4.5 | x-ai |
|
3.62 | $0.10 |
| 27 | gpt-5.6-luna-pro | openai |
|
3.54 | $0.08 |
| 28 | gpt-5.6-terra | openai |
|
3.49 | $0.03 |
| 29 | deepseek-v4-pro-0813 | deepseek |
|
3.54 | $0.03 |
| 30 | qwen3.6-plus | qwen |
|
3.56 | $0.06 |
| 31 | qwen3.8-flash | qwen |
|
3.71 | $0.02 |
| 32 | inkling-small | thinkingmachines |
|
3.62 | $0.01 |
| 33 | gpt-5.4-mini | openai |
|
3.72 | $0.05 |
| 34 | grok-4.1-fast | x-ai |
|
3.73 | $0.01 |
| 35 | grok-4.6 | x-ai |
|
3.57 | $0.16 |
| 36 | claude-opus-4.7 | anthropic |
|
3.53 | $0.10 |
| 37 | claude-sonnet-4.5 | anthropic |
|
3.63 | $0.08 |
| 38 | gemma-4-31b-it |
|
3.56 | $0.00 | |
| 39 | gpt-5.2 | openai |
|
3.68 | $0.03 |
| 40 | gemini-3.6-flash |
|
3.53 | $0.07 | |
| 41 | kimi-k2.5 | moonshotai |
|
3.59 | $0.05 |
| 42 | gpt-5.4-nano | openai |
|
3.72 | $0.01 |
| 43 | qwen3-max-thinking | qwen |
|
3.74 | $0.24 |
| 44 | gpt-5-nano | openai |
|
3.69 | $0.01 |
| 45 | grok-4.3 | x-ai |
|
3.74 | $0.05 |
| 46 | mimo-v2-flash | xiaomi |
|
3.65 | $0.00 |
| 47 | mistral-medium-3-5 | mistralai |
|
3.87 | $0.30 |
| 48 | qwen3.8-2.4t-a95b | qwen |
|
3.5 | $0.22 |
| 49 | glm-5.3 | z-ai |
|
3.68 | $0.03 |
| 50 | qwen3-next-80b-a3b-thinking | qwen |
|
3.69 | $0.04 |
| 51 | kimi-k2.6 | moonshotai |
|
3.75 | $0.21 |
| 52 | mimo-v2-pro | xiaomi |
|
3.77 | $0.03 |
| 53 | claude-haiku-4.5 | anthropic |
|
3.71 | $0.08 |
| 54 | inkling | thinkingmachines |
|
3.5 | $0.09 |
| 55 | mimo-v2.5-pro | xiaomi |
|
3.76 | $0.04 |
| 56 | glm-5.2 | z-ai |
|
3.74 | $0.07 |
| 57 | deepseek-v4-flash-0731 | deepseek |
|
3.58 | $0.00 |
| 58 | muse-glimmer-30b | meta |
|
3.92 | $0.01 |
| 59 | gemini-3.1-pro-preview |
|
3.64 | $0.19 | |
| 60 | glm-5-turbo | z-ai |
|
3.73 | $0.03 |
| 61 | gemini-3.5-flash-lite |
|
3.58 | $0.02 | |
| 62 | muse-spark-1.3 | meta |
|
3.6 | $0.10 |
| 63 | gemini-2.5-pro |
|
3.6 | $0.16 | |
| 64 | gpt-oss-120b | openai |
|
3.74 | $0.01 |
| 65 | nemotron-3-ultra-550b-a55b | nvidia |
|
3.79 | $0.03 |
| 66 | qwen3.8-27b | qwen |
|
3.78 | $0.03 |
| 67 | gemini-3.5-flash |
|
3.44 | $0.06 | |
| 68 | glm-5.3-flash | z-ai |
|
3.77 | $0.00 |
| 69 | gemini-2.5-flash |
|
3.83 | $0.03 | |
| 70 | qwen3.5-397b-a17b | qwen |
|
3.63 | $0.08 |
| 71 | glm-5 | z-ai |
|
3.59 | $0.06 |
| 72 | mistral-small-2603 | mistralai |
|
3.98 | $0.01 |
| 73 | kimi-k3 | moonshotai |
|
3.57 | $0.41 |
| 74 | nemotron-3-nano-30b-a3b | nvidia |
|
3.66 | $0.01 |
| 75 | nemotron-3.5-lightning | nvidia |
|
3.73 | $0.01 |
| 76 | gpt-oss-20b | openai |
|
3.85 | $0.01 |
| 77 | qwen3.6-27b | qwen |
|
3.5 | $0.05 |
| 78 | qwen3.7-flash | qwen |
|
3.68 | $0.00 |
| 79 | deepseek-v4-flash | deepseek |
|
3.52 | $0.00 |
| 80 | gemini-3-flash-preview |
|
3.46 | $0.02 | |
| 81 | qwen3.5-122b-a10b | qwen |
|
3.56 | $0.14 |
| 82 | qwen3.5-27b | qwen |
|
3.61 | $0.06 |
| 83 | mimo-v2.5 | xiaomi |
|
3.67 | $0.02 |
| 84 | qwen3.6-flash | qwen |
|
3.76 | $0.06 |
| 85 | gemma-4-26b-a4b-it |
|
3.41 | $0.01 | |
| 86 | glm-5.1 | z-ai |
|
3.5 | $0.08 |
| 87 | glm-4.7 | z-ai |
|
3.67 | $0.07 |
| 88 | minimax-m2.5 | minimax |
|
4.24 | $0.01 |
| 89 | muse-spark-1.1 | meta |
|
3.66 | $0.05 |
| 90 | qwen3.5-35b-a3b | qwen |
|
3.79 | $0.04 |
| 91 | minimax-m3 | minimax |
|
3.77 | $0.05 |
| 92 | qwen3.6-35b-a3b | qwen |
|
3.49 | $0.07 |
| 93 | deepseek-v4-pro | deepseek |
|
3.08 | $0.08 |
| 94 | minimax-m2.7 | minimax |
|
4.14 | $0.01 |
| 95 | mistral-large-2512 | mistralai |
|
3.7 | $0.00 |
| 96 | glm-4.7-flash | z-ai |
|
4.0 | $0.01 |
| 97 | trinity-large-thinking | arcee-ai |
|
2.67 | $0.04 |
| Rank | Word |
|---|---|
| 1 | WAFER |
| 2 | EXPEL |
| 3 | EAGER |
| 4 | PIPER |
| 5 | BOOZY |
| 6 | GULLY |
| 7 | PROXY |
| 8 | HILLY |
| 9 | FIFTY |
| 10 | VALUE |
| Rank | Model |
|---|---|
| 1 | trinity-large-thinking |
| 2 | glm-4.7-flash |
| 3 | deepseek-v4-pro |
| 4 | muse-spark-1.1 |
| 5 | minimax-m3 |
| 6 | qwen3.6-35b-a3b |
| 7 | gemma-4-26b-a4b-it |
| 8 | gemini-3-flash-preview |
| 9 | glm-5.1 |
| 10 | deepseek-v4-flash |