Visual regression testing with PhantomCSS

Two YEARS!

On the 6th of November 2012 I finally decided to open source an insane idea I had about 10 months earlier, an idea that had seemed so obvious, yet so impossible.  To test the UI as a person would, by seeing. An automated visual regression test suite.

This, wasn’t a new idea; there had been many a boozy pub chat about how amazing it would be.  But at that point in time I did not know of any working implementations. I found out later that it had been tried successfully, at least twice, but the idea had never become popular.

I remember thinking, “this will never work, it’ll be too unstable” – but it did (kinda, mostly) work.

Continue reading

Contracts for automated UI testing

I’ve seen people write tests that assert that a button has a ‘disabled’ CSS classname to test if a button is disabled. I have then seen those same tests break because someone changed the styles so that ‘btn-disabled’ is now the classname of choice for styling that state, but of course nothing actually broke, the test failure was a false negative. At the same time the old styles for the ‘disabled’ class have been removed, but some buttons have been left with the old classname – tests still pass but it is (to the user) broken, a false positive. The example test below shows this kind of assert.

assert(page.myButton).hasClassName('.disabled', 'Should be disabled');

Continue reading

Asynchronous UI: When to lie

Let’s work on the premise that we are comfortable building user interfaces that lie. We won’t lie when something was successful, but we will lie when there is a temporary internet connection failure or an internal server error. The user might be annoyed when they find out (if they find out), they might lose trust in our application; or, they might try again, and it’ll probably be successful this time.

Continue reading