On an AMD Ryzen 9 5950X I got:
VLM inference took 29.9s
Smaller LLMs are faster, but the current (Gemma4 E2B Q4)
is more reliable in its outputs and can correctly describe
what's on screen.
Allowing more than `--threads 1` makes it much faster,
but at the potential cost of nondeterministic
output of `llama-cli`.
The VLM screenshot analysis running in CI makes it slower,
but spending < 1 minute single-core seems worth it
given that it quite robustly removes the need for a human
to check if the GUI renders correctly.
Potential future improvement:
A bit benefit would be content-addressing of the screenshot:
Most nixpkgs update would likely not change the screenshot
pixels at all, reproducing it bit-identically.
In that case, there's no need to re-run the VLM at all.
Doing the VLM analysis in a separate derivation that takes
the screenshot as only input would make it ossible
to cache these results, while still being less maintenance
than having a "golden" screenshot that needs to be updated
by a human when screenshots actually change.
Niklas Hambüchen
09ddbaf6f8