The analysis uses a personal rolling benchmark built around private Chinese tasks. It tracks logic, math, coding, instruction following and human-intuition problems, then warns readers not to worship any leaderboard without testing models against their own needs.
This analysis is a long-running personal benchmark for large language models, focused on logic, mathematics, coding, instruction following and human-intuition style problems.
The benchmark is explicit about the limits: the benchmark is not official, not comprehensive and not meant to be treated as a universal authority. Its value is that it tracks model progress over time with a private Chinese task set. The test bank stays small, around 28 questions and fewer than 270 to 272 scoring points, but it is deliberately protected from public contamination. Each month, tasks can be added or removed, and questions whose average score rate rises above a saturation threshold are retired.
| Benchmark trait | Detail | Why it matters |
|---|---|---|
| Private task bank | Around 28 Chinese questions with roughly 272 scoring points. | Reduces the risk that models have memorized public benchmark examples. |
| Rolling monthly update | New questions enter and saturated questions leave. | Keeps the benchmark useful as frontier models solve older puzzles. |
| Personal methodology | The comparison states the test is not authoritative or complete. | Readers should treat it as a strong signal, not a final verdict. |
The April list refreshes around the newest frontier releases while removing models replaced by later versions or beyond the tracking window.
The analysis lists models that left the current table because newer versions arrived: GLM-5, Kimi-K2.5, several Qwen3.5 and Qwen3 variants, DeepSeek V3.2-1201, Tencent HY 2.0 Think, Ling-flash-2.0, MiMo-V2 Pro and Flash variants, and GPT-5.4. It also places older models outside the tracking window, including gpt-oss-120b and GLM-4.7 Flash. The point is that a model leaderboard in 2026 has a short half-life. A monthly benchmark has to prune aggressively or it becomes a museum.
This analysis scores each task by normalized points, requires valid reasoning and penalizes answers that arrive by guessing or brute-force coverage.
The scoring rules are strict. Models use official recommended parameters where possible, otherwise a low default temperature. Reasoning models can use a large thinking window, while non-reasoning models receive a separate output length limit. Each question has at least one scoring point, and every correct point contributes to a normalized score. But the final answer alone is not enough: the reasoning process must be right, guessed answers do not count, and brute-force attempts that merely cover the correct answer can lose points. If the prompt asks for no explanation, adding an explanation can also lose credit even when the answer is otherwise correct.
The April update adds an information-compression task and a character-matrix task, replacing two saturated older questions.
The information-compression task asks the model to recover original content from a compressed clue. It cannot be solved by exhausting all possibilities; the model has to notice the underlying pattern. GPT-5.4 and GPT-5.5, Claude Opus 4.6 and sometimes DeepSeek V4 Pro can reach full marks, while Kimi K2.6, GLM-5.1 and DeepSeek V4 Flash solve about half the cases after heavy reasoning. Qwen3.6, Seed 2.0 Pro and Gemini 3 are described as often only solving the easiest examples and hallucinating during the process. The character-matrix task is easier but loaded with negative constraints, testing whether the model can avoid forbidden actions while completing the required output.
For models with enough coding ability, this analysis adds multi-round engineering projects that ask a model to build near-production software from scratch.
The engineering benchmark is separate from the 28-question logic test. It contains desktop, mobile, backend, web and game projects, with different languages and knowledge areas. In each project, the model is prompted across multiple rounds until all requirements are met or the model gets stuck. April changes the scoring rule: instead of capping each round's combined mistakes at a fixed penalty, every unmet requirement or incorrect detail now costs points. The comparison notes this better reflects real user experience because a model that creates many small bugs forces more repair work, more verification and more agent time.
The April game project requires Godot plus C# and a custom physics engine, making it a test of coding knowledge, interaction design and physics reasoning at the same time.
GPT and Claude Opus handle this project more smoothly because they have broader programming knowledge. Their remaining errors are mostly small physics or shader mistakes that can be fixed in one round. Chinese models struggle more: DeepSeek V4 and GLM-5.1 can get stuck on presentation details or physical simulation behavior, with GLM-5.1 nearly failing before gradually recovering through small fixes. DeepSeek V4 is judged slightly better but still visually crude and only barely production-like. Other Chinese models often fail earlier because they lack Godot knowledge, misread errors or cannot implement and test a physics engine correctly.
The conclusion is balanced: Chinese frontier models have crossed a practical usability line, but still trail the newest overseas models in both intelligence and agentic detail.
The comparison argues that OpenAI has shifted some attention from raw intelligence toward hands-on execution, narrowing the gap with Claude. Chinese models, meanwhile, have to improve both intelligence and execution quality at the same time. In this view, the first tier of Kimi, GLM and the newly stronger DeepSeek has roughly reached the level of North American frontier models from the previous October, while agent and coding polish sits somewhere around the Sonnet 4.5 to Sonnet 4.6 zone. That still implies a gap of roughly half a year in intelligence and more in agent/coding detail, but this analysis also emphasizes a turning point: top Chinese models are now useful enough that teams that once avoided them are beginning to connect real business workflows, generating the data that can feed the next improvement loop.
Mistake
Treating one article as a final ranking
Why it hurts
Model releases, pricing, quotas and benchmark positions can change quickly.
Better move
Use the analysis as a shortlist, then run current checks against your own workload.
Mistake
Choosing by brand instead of task
Why it hurts
A strong chat model may still be weak for long documents, coding agents, multimodal work or low-latency routes.
Better move
Define the job first, then compare models with prompts, files or media that match that job.
Mistake
Copying claims without a current verification check
Why it hurts
Benchmark numbers, context windows, API names and prices may be dated or provider-specific.
Better move
Confirm high-impact details against official docs, model cards or live provider pages.
Use this page to understand the model family, the evaluation angle and the current conversation around it. Then choose one or two realistic prompts, documents or media tasks and test whether the model behaves well in your own workflow.
These questions reflect recurring reader concerns around Chinese model knowledge, evaluation and fast-moving model releases.
The analysis uses a personal rolling benchmark built around private Chinese tasks. It tracks logic, math, coding, instruction following and human-intuition problems, then warns readers not to worship any leaderboard without testing models against their own needs.
Use it as market and product context, then verify technical claims, pricing, quotas and release details against official pages or your own tests before making a decision.
The video gives a fast visual summary of the model story, while the written page carries the caveats, comparisons and practical checks.
SmarToken tracks public model releases, technical reports, product announcements and market signals to keep this catalog useful.
Technical claims need to be treated as dated unless they are confirmed by current official model cards, technical reports or provider announcements.
Pricing, quota, availability and benchmark details can change after the review date, so production decisions should use current vendor pages and direct workload tests.
