METHODOLOGY / VERSION 1.0
How we test
A reproducible process before a publishable verdict.
This is our proposed protocol. No benchmark runs have been completed. Author identity and contact details must be supplied before editorial launch.
One source set, two kinds of work
Use three rights-cleared English videos: a single-speaker tutorial, a two-speaker interview, and a screen-sharing demonstration. Record each source’s duration, resolution, frame rate, transcript, license, and file hash. Submit the same files and settings to each tool. Run each source twice to expose variability; label reruns and their charges separately.
Define an accepted clip before seeing results
A usable clip must communicate a complete idea, preserve meaning, keep the relevant speaker or demonstration visible, and have no blocking export or caption defect. Record accepted, repairable, and rejected outputs separately. Count repair time; do not silently discard failures.
What the run record must contain
Save source ID, settings, plan, start and completion timestamps, job ID, credits before and after, refunds, output count, accepted count, export resolution, codec, frame rate, and caption errors. Retain original exports and redacted screenshots. Product screenshots will appear only after actual runs.
Our score, out of ten
| Dimension | Weight | Evidence |
|---|---|---|
| Clip quality | 30% | Coherent opening, complete idea, clean ending, and correct framing. |
| Caption accuracy | 20% | Word error rate against a manually checked transcript. |
| Automation | 20% | Completion, recovery, and verified delivery in the intended workflow. |
| Value | 20% | Subscription cost and repair time per accepted clip. |
| Usability | 10% | Time and interventions needed to reach an acceptable export. |
Final score = sum of each dimension’s score × its weight. Every dimension needs a published rationale and evidence. A missing dimension means no overall score, not a zero or an invented average.
Cost that reflects a real workflow
Report both allocated credit cost and actual cash cost per accepted clip. Include unused subscription capacity, repeated jobs, and editing labor. Never compare credits between vendors as though they were a shared unit.
Editorial independence & corrections
Scrutator is an independent research project. Product vendors do not set the score. Vendor statements, our calculations, and observed results are labeled distinctly. Corrections must retain a dated record of what changed and why.
Browse the research ↗