Continuous Integration Pt IV - Working with PHPUnit, Selenium, and SauceLabs

Ah, yes, the saga continues!

Today, I am going to try to get our PHP Selenium tests to run on SauceLabs' OnDemand service. SauceLabs is a cross-browser testing service. You can run your tests by hand - by opening an instance of a particular browser on their server via your own Web browser - or in a more automated fashion. I want to do the latter. What this entails is writing tests to tell the SauceLabs servers with which browser we wish to test on which operating system (e.g., IE7 on Windows). You can run the same test across multiple browsers, and any failed tests will be recorded for you to see exactly what went wrong. We're planning to automate the running of these tests, kicking them off when a developer pushes their code to the staging server.

The tests can be written in a number of languages for either Selenium RC or WebDriver. Because I'm using PHPUnit (since I code primarily in PHP), I'm stuck with Selenium RC, which is quite old but is supported by extensions in PHPUnit. I've successfully run tests on my local development machine, but now I'm going to adding one more layer of complexity, i.e. the SauceLabs servers. This requires one more set of extensions be added to the PHPUnit test framework. Here goes!

Install the SauceLabs SauceOnDemand packages from Pear:

sudo pear channel-update pear.php.net
sudo pear upgrade pear
sudo pear channel-discover pear.phpunit.de
sudo pear channel-update pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear channel-update components.ez.no
sudo pear channel-discover pear.symfony-project.com
sudo pear channel-update pear.symfony-project.com
sudo pear channel-discover saucelabs.github.com/pear
sudo pear channel-update saucelabs.github.com/pear

No problems there. I'd already done the "discovering" of pear.phpunit.de and pear.symfony-project.com, so I only needed to channel-update for those.

Next, install the SauceLabs Selenium PHPUnit extensions:

sudo pear install -a saucelabs/PHPUnit_Selenium_SauceOnDemand

Great! That worked for me with no problems.

The next steps requires a SauceLabs account, I think. It was very cryptic, at first, because, without being logged in, it just says to do the following:


sauce configure


But that didn't do anything but throw a strange error and then tell me I was ready to run saucy tests and that I feel hot and saucy (which I didn't, btw).


After creating a free account (which anyone can do) and logging in, returning to the instructions page revealed the same instructions to do "sauce configure," but that command was followed immediately by my username and a hash code. So, I ran that. No errors, and now I really am feeling hot and saucy.


I'm going to run a test of a test. (I guess I'm feeling saucy and testy.)

curl -s https://saucelabs.com/example/se1/php/private-JJHyCGMF | bash

Well, this threw up a big ugly error with stacktrace and junk. It looks like something is calling an undefined method:

undefined method PHP_CodeCoverage_Filter::getInstance()

That's no good. I'm wondering if I need to update my version of PHPUnit is old? Maybe I should just upgrade. I did this

pear upgrade

and suddenly faced all kinds of ugly. Upgrade failed. Doh! Oh, I forgot to use 'sudo':

sudo pear upgrade

Much better! Now I've got upgraded Pear stuff.

I ran the curl command from above again, and got exactly the same results. Hmmm...

Looking at my Pear channels and packages, I've noticed that I have two versions of PHPUnit installed, one in the traditional pear.phpunit.de channel, and one under saucelabs.github.com/pear. the saucelabs version is older than the other one. There is also a different version of PHPUnit_Selenium in that channel, though that's not the one that is active. I've emailed the SauceLabs people for some help on this. My concern is that it's due to conflicts in one version vs. another, and I'd like to nip that possibility in the bud.

In the meantime, I'm going to look at the problem code. The error message is:

Call to undefined method PHPUnit_Util_Test::getParallelismSettings() in /opt/local/lib/php/PHPUnit/Extensions/SeleniumTestCase.php on line 352

I won't write about that in this post, however. For the time being, this is a work in progress.