On September 22, 2026, I spent a day working inside Mistral's tools. I gave them three different jobs: build an n8n workflow, moderate text and images, and generate a web page. I also checked how much of my Pro subscription those tests used.
I wanted to see whether I could finish each job, what it cost, and where I had to step in. These are my own tests, with their methodological limits. They do not rank every model on the market or predict what will work in every company.
An n8n workflow with GLM 5.3 in two tools
For the first test, I asked for a simple lead-routing workflow in n8n. I used the same model, GLM 5.3 served by Mistral, through two tools: Mistral Vibe and Pi. GLM 5.3 is a Z.ai model hosted by Mistral; Mistral did not train it.
Vibe created the workflow in 2 minutes 48 seconds and published it after 4 minutes 15 seconds in total. It had nine nodes. Two errors found during testing were fixed, and it kept the requested webhook path. Pi created its version in 2 minutes 27 seconds and published it after 9 minutes 28 seconds in total, with eight nodes. It tested four branches. Its webhook path had to change because Vibe had already taken the intended address.
Vibe delivered faster in this run. The webhook conflict slowed Pi down. The tools did not follow identical paths to publication, and one workflow cannot establish their general reliability. This was an exploratory field test, not a controlled benchmark.
For a company, the time it takes to generate the workflow JSON is only part of the job. Someone still has to check the branches, error handling, data passed between nodes, and what happens when a real lead comes in. I discuss how to divide that work between a coding agent and a durable automation in our article on n8n and coding agents.
Shieldstral missed some text inside images
The second job involved Shieldstral 1.0, a moderation model I ran on a rented GPU. I prepared 40 text and image cases, each checked against five policies: violence, harassment, sexual or child-inappropriate content, personal information, and political persuasion. That made 200 decisions. The first test's main metrics cover 31 clear cases, or 155 decisions; nine borderline cases were excluded.
At the chosen threshold, macro F1 was 0.73, with seven false negatives and no false positives on those clear cases. The most visible weakness appeared in posters where the meaning depended on words printed in the image. An explicit election poster could fall below the detection threshold.
I then reran the image portion through two pipelines. The first sent each image directly to Shieldstral. The second sent it to OCR 4.1 and passed only the extracted text to Shieldstral. Both pipelines used the same 20 images and five policies. With four borderline images excluded, the main comparison covers 16 images, or 80 decisions per pipeline.
| On the 16 clear images | Image alone | OCR 4.1 then Shieldstral |
|---|---|---|
| Macro F1 | 0.40 | 0.96 |
| False negatives | 5 | 1 |
| False positives | 0 | 0 |
OCR recovered words that Shieldstral had struggled to use in the image. One important failure remained: a public Uncle Sam poster was still not identified as political persuasion after its text had been extracted. Reading the words does not always reveal their context.
These figures describe a small corpus that includes clean, deliberately constructed images. They are not a performance forecast for a company's documents or visual content. The threshold would need to be tested on a separate dataset before production use. For document extraction rather than image moderation, I have a separate guide to using Mistral OCR with n8n.
A usable landing page, after a correction
I also asked Mistral Medium 3.5 to generate a SaaS landing page from a visual reference obtained through the 21st.dev connector. Medium 3.5 produced a complete page and a useful starting point. The first pass did not set up the CSS pipeline correctly, though. A second pass was needed to get the expected rendering.
More seriously, the page made up marketing claims: "500 companies", SOC 2 certification, and a "guaranteed SLA". None of those appeared in the brief. Publishing them would have misled visitors.
This test does not measure Medium 3.5's visual quality against every newer model. My narrower assessment is that I could work from the page, but I could not publish it as written or trust it to invent its own commercial proof. On a company website, a plausible but false claim can cause more harm than a misaligned button.
Vibe: delivery and day-to-day usability
Mistral Vibe did get the n8n workflow published. Working with it still felt uneven. When I asked the agent to import an MCP configuration, it wrote a format that Vibe could not accept. Vibe failed to restart, and I used Pi to repair the configuration.
At first, I also thought Vibe ignored global instructions. Once I manually added my file to ~/.vibe/AGENTS.md, Vibe picked it up. The feature is documented. My criticism is about how hard it was to discover and set up, not about a missing capability.
Nor would it be fair to call Vibe abandoned: the repository had published another release on September 22. Regular releases do not spare the user from a configuration that stops the tool from starting. An agent changing its own environment should check the format before writing it and give the user a clear way back when something breaks.
What the day cost
My Mistral account showed a Pro subscription at €17.99 per month including tax. When I checked on September 22, the dashboard displayed two separate monthly allowances: €255 of Vibe Code usage and €25.50 of API and Studio usage. These are included usage amounts, not cash balances.
The usage report I opened at around 4:54 p.m. Paris time showed 276 requests and €4.96 in metered usage since midnight. That usage fell within the subscription allowances, so it was not an additional €4.96 charge. I checked before the day ended; this is not a final total for September 22.
I compared the dashboard with local Vibe and Pi logs. They contained just over 11 million cumulative tokens for calls to models served by Mistral. About 80% of input tokens were reads from cache. That does not mean 11 million newly written or generated tokens, and it does not establish a guaranteed monthly token allowance. Mistral's dashboard is the reference for the metered amount.
The GPU rented to run Shieldstral, and any subscription to a third-party tool, sit outside that figure. The cost of a full test also includes the time spent repairing a configuration, reviewing a page, and preparing a test corpus. I would consider all of that before choosing among an open model, a hosted API, and internal deployment.
What I would test before choosing a tool for a company
This day suggested a practical way for a small company to evaluate AI on its own work:
- Pick a bounded job with an output you can verify. A lead-routing workflow, a batch of documents, or a test page tells you more than an open-ended demo.
- Decide what counts as an error before the test. For a workflow, check branches and recovery; for a web page, public claims; for moderation, false negatives and borderline cases.
- Measure the whole chain. A capable model can be held back by a difficult tool. A cheap API call can become expensive once you add GPU time, operations, and human review.
- Keep an approval point before any visible or hard-to-reverse action. I would not have published the landing page with its invented figures. I would not deploy this moderation filter on the strength of 16 clear images either. Our guide to human approval in production AI agents explains where to put that check.
Mistral has useful models of its own and now hosts third-party open models such as GLM 5.3. In my view, its clearest opportunity is to make the working environment more coherent: a dependable CLI, understandable settings, and usage allowances people can follow. Vibe already runs across several interfaces; I want those interfaces to work together better, rather than another interface being announced.
If I were repeating these tests inside a company, I would start with one process whose possible errors and approval owner are known. That is also the starting point in our guide to choosing a first AI project and in Kirako's work on AI agents for businesses.
Sources and limits of these results
I published observations from the n8n workflow, Shieldstral, Shieldstral with OCR, and the landing page on the day of the tests. Both moderation tests also have local reports recording the corpus, settings, and decision-level results. The cost figures come from my account dashboard as viewed on September 22 and from local logs; they are not universal prices.
For product capabilities, I checked Mistral's official pages for GLM 5.3, Medium 3.5, and OCR 4.1, along with the Shieldstral cookbook and Vibe releases, on September 23, 2026. Judgments about usability and product direction are my own.
Also available: Read in French