In my last post, I wrote about reaching a ceiling with Cypress and AI. The spike I mentioned there is now complete. This is what happened next.
Setting It Up
I still had an active Cursor subscription that had been my primary tool before I moved to Devin Desktop. Rather than let it sit idle, I gave it a job: analyze the test suite, analyze the skills that write and maintain those tests, and report back on the feasibility of migrating roughly 900 end-to-end tests from Cypress to Playwright.
For the analysis phase, I used Claude Opus as the model. It came back with a detailed plan. I reviewed it, refined a few things, then switched it to auto mode and let it run.
That was the last time I had to think about it for a while.
How the Migration Actually Ran
The first pass rewrote every test file and every supporting file from Cypress to Playwright. It also removed all Cypress dependencies from package.json and cleaned up every reference to the old framework. The goal for that pass wasn’t passing tests; it was zero compilation errors. Once TypeScript was clean, that phase was done.
From there, I set up a second plan focused on getting tests to pass. Most of the tests intercept API calls; a smaller group are smoke tests that hit a live backend with seeded data. I told it to focus on the larger group first.
It figured out on its own to work by functional area. It would identify the patterns that were causing failures, document its findings, then apply those findings across the remaining tests in that area before moving to the next one. Every hour or two, it would surface to report progress. By then the context window was usually close to exhaustion, so I would ask it to update the migration handoff document and give me a prompt I could paste into a new session to continue.
Eight sessions. Across five days. About two hours of my actual time.
What Made It Work
At no point did I need to open a failing test report to help the AI troubleshoot anything.
With Cypress, that had been a regular part of the cycle. The AI would hit a wall, I would run the test in the Cypress GUI, take a screenshot of what I was seeing, and feed context back in. With Playwright, it found the failures on its own, read the available output, and moved forward. Over hundreds of tests, that difference compounds fast.
I think it comes down to how Playwright reports what it’s doing. The error messages are structured in a way that’s useful to an automated reader, not just a human skimming a terminal. I haven’t had time to analyze and fully explain the gap, but the behavior difference was consistent enough across the entire migration that I’m confident it’s real.
The Numbers
The old Cypress suite took 30 to 35 minutes to run, on a machine doing nothing else. Early runs of the Playwright suite are finishing in about 25 minutes, on a machine that was busy with other things at the time. That’s not the main reason I did this, but it’s a welcome result.
Where This Goes Next
The migration itself is done. What remains is cleanup and forward motion:
-
Refining the E2E test skills now that they’re Playwright-based
-
Updating the hooks to enforce the same behaviors the old ones enforced
-
Writing guidance for migrating the Cypress tests still sitting in other active branches, so those can be converted before they get merged in
-
Documenting the full migration story from transcripts and the handoff document the AI kept updated throughout
The goal is that starting next sprint, every new test is Playwright. The Cypress chapter closes.
Two Hours
Two hours of my time to rewrite 900 tests; this is not the type of work I would have manually done a couple of years ago. It wouldn’t be worth it.
The AI managed the work. I showed up to hand off between sessions and answer the occasional question. The rest ran in the background while I worked on sprint features.
The earlier post was about recognizing when friction is structural and deciding it was time to act. This is what acting on it looked like.
Leave a Reply