Now that I’ve finished re-building my house (almost) I’ve got two or three years of programming projects to catch up on. One of the most pressing is updating the Motif.org.au site with the new reading tests that have been developed and normed for ‘phase 2’.<\/p>\n

My problem is that I’ve been working on the site on-and-off over the last two years, there’s a version on staging which has completed UAT but it took so long that I’ve had to modify the live site with content updates. My old PC died so I’ve lost my old dev environment. I’ve switched from SVN to Git in one dependency. The live site has had a year of traffic too, so the staging database is way out of date. I feel extremely nervous about pulling all this together and re-launching the staging version, before going ahead with the queue of subsequent changes.<\/p>\n

Automation to the rescue!<\/p>\n

I’ve used Selenium IDE and RC before for full-stack UAT testing, and done a lot of unit testing and UAT scenario testing via PHPUnit so I thought it would be pretty easy to pull together some tests to ensure the site and the language tests are all still working properly while I finish off a list of integrations, package updates and refactoring I want to do before launch. The site isn’t huge and it’s well structured but the language tests have to work perfectly. It would be a pretty serious consequence to be mis-diagnosing children’s learning abilities, due to a software bug. This site really does affect people’s lives.<\/p>\n

So I started confidently pulling together a testing environment mostly copied from previous projects but decided to start writing the tests from scratch and use the latest patterns. I came very quickly unstuck, and found the Selenium 2 documentation for both the server, Web Driver and phpunit-selenium to be quite mysterious.<\/p>\n

I spent days feeling like I was missing something – I thought it was the proper documentation! In fact I had quite a few major misunderstandings about Selenium and phpunit-selenium. I was doing it wrong.<\/p>\n

So partly for my own memory, partly to help anyone else who is diving a little deeper into Selenium testing, I’m going to document my learnings here.<\/p>\n

The first main problem was Selenium v1 verses Selenium v2.<\/strong> There’s so much mixing of help and tutorials on the web, it’s easy to get the wrong advice. Even on Stack Overflow almost every question has a mix of right and wrong answers, the wrong ones often in v1 syntax when answering a v2 question. One of the underlying causes I believe is that there’s some helpfully intended but obscure backwards compatibility out there which some people (like me!) don’t know they’re using.<\/p>\n

e.g. on the phpunit-selenium readme it says:<\/p>\n

“This package contains a base Testcase Class that can be used to run end-to-end tests against Selenium 2 (using its Selenium 1 backward compatible Api).”<\/p><\/blockquote>\n

Now I’m not sure whether the “it’s” refers to this package or the server, but someone is providing a v1 API! (It’s actually both.) In the case of phpunit-selenium:<\/p>\n

\n
1. PHPUnit\/Extensions\/SeleniumTestCase.php<\/a>\u00a0– if you inherit from this you use Selenium1 (RC) API<\/li>\n
2. <\/a>PHPUnit\/Extensions<\/a>\/Selenium2TestCase.php<\/a>\u00a0– this implements the Selenium2 API\u00a0(Web Driver, via the Json wire protocol)<\/li>\n<\/ol>\n

   So that should be pretty clear right? Except for the fact the v1 test case is just called TestCase, (which is a mistake we all make when making the first version<\/em> of anything!) you are in control of which API you use when you make the inheritance choice. However I spent the last two weeks at work updating our hundreds of tests thinking I was using Selenium 2 – because a) we are using Selenium 2 server which is automatically started by Bamboo so I didn’t really get involved, b) all of our test cases extend from our common base class so you don’t see the inherited API class in your face, c) he Selenium HQ website’s attempts at explaining the differences between 1 and 2 make your head spin!<\/p>\n

   The best documentation for the Selenium 1 API<\/strong> doesn’t seem to be linked from the Selenium HQ website! Luckily Sebastian Bergmann provides it in his brief intro to phpunit-selenium.<\/p>\n

   \n
   • http:\/\/release.seleniumhq.org\/selenium-core\/1.0.1\/reference.html<\/a><\/li>\n
   • http:\/\/phpunit.de\/manual\/current\/en\/selenium.html#selenium.seleniumtestcase<\/a><\/li>\n<\/ul>\n

     The best documentation for the Selenium 2 API<\/strong> is also not on the HQ site! They’ve not finished the docs yet, but seem to be only writing guides and intros not documenting the API itself.<\/p>\n

     \n
     • https:\/\/code.google.com\/p\/selenium\/w\/list<\/a>\u00a0– index<\/li>\n
     • https:\/\/code.google.com\/p\/selenium\/wiki\/JsonWireProtocol<\/a>\u00a0– what phpunit-selenium implements<\/li>\n<\/ul>\n

       OK so that’s that cleared up. But I’m not talking REST to Selenium2, I’m a developer writing phpunit tests! I want to know which classes and methods to call when I’m trying to achieve test tasks.<\/em> I don’t want to know about REST and HTTP and Java types – it all sounds irrelevant to my interests, but in reality it’s just three steps of abstraction away from where I sit.<\/p>\n

       Indeed the wiki warns testers away and sends them back to HQ, presumably so they’ll find the client language SDK they’ll actually be coding against.\u00a0Sadly phpunit-selenium isn’t mentioned on the HQ site, but for PHP developers they point out several implementations of the WebDriver bindings, including the Facebook one, which again confused me to begin with. Although it’s related and implementing the same upstream API, the code samples around the web don’t directly help me write my PHPUnit tests!<\/p>\n

       So back to phpunit-selenium: where is the best documentation for fighting with Selenium v2 from this class-base?<\/p>\n

       \n
       1. Firstly check out the PHPUnit site’s page on Selenium<\/a>\u00a0(already mentioned), but ignore the section on\u00a0<\/span><\/span>PHPUnit_Extensions_SeleniumTestCase which is v1. It’s easy to scroll down the page in a hurry and start reading the wrong stuff!<\/li>\n
       2. This links you off to the Test Cases<\/a>\u00a0for phpunit-selenium itself as the main reference.\u00a0At first this confused the hell out of me, and my head almost imploded in a recursion of unit testing. But I can see why Sebastian and Giorgio are doing this – they’re still working diligently on implementing the v2 API and in true TDD style, the tests of a system are the best functional spec<\/em>!<\/li>\n
       3. Thirdly the code documentation in the TestCase2 class<\/a>, these are the droids you are looking for. Well this is where things aren’t really finished, and are partly scuppered by the generalisation of the calling process (using _call for most API commands). There’s a bunch of @method statements in the main classes intro, some of which don’t compile in some documentors (they don’t work for me in PHPStorm for example) and some aren’t linked to the sub-classes which perform the actual work.<\/li>\n
       4. There’s also some relevant API command documentation in other classes such as:\n
          \n
          1. The Session class<\/a> where you can setup global stuff rather than test-specific.<\/li>\n
          2. The Element class<\/a> which has specifics on querying DOM elements.<\/li>\n
          3. The Select class<\/a> which has been split out for drop-down specific stuff.<\/li>\n
          4. A few other places. Explore!<\/li>\n<\/ol>\n<\/li>\n
          5. [UPDATE] Someone has kindly run apigen and hosted\u00a0the resulting documentation for phpunit-selenium<\/a>, although I’m not sure what version it is.<\/li>\n
          6. [UPDATE] Check SystemicPlural’s comment at\u00a0 this tutsplus.com tutorial how-to-use-selenium-2-with-phpunit<\/a><\/li>\n<\/ol>\n

             So at the end of the day, there IS plenty of documentation for using phpunit-selenium with Selenium 2 Server, but it’s just not all been tied together yet. The phpunit-selenium codebase is organised in a very different way from the actual Selenium 2 WebDriver API so it’s really hard to find a one-to-one mapping when using the latter as a reference while coding in the former.<\/p>\n

             I’m hoping that once I understand it all myself, I will get some time to help out writing some noob-developer-friendly documentation and actually contribute to the huge effort these teams have been putting into providing these amazing tools.<\/p>\n

             The only problem is, once I’ve learned it all I may forget what it was I didn’t know!<\/p>\n","protected":false},"excerpt":{"rendered":"

             Now that I’ve finished re-building my house (almost) I’ve got two or three years of programming projects to catch up on. One of the most pressing is updating the Motif.org.au site with the new reading tests that have been developed … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[26,42,12],"class_list":["post-88","post","type-post","status-publish","format-standard","hentry","category-articles","tag-php","tag-software-engineering","tag-unit-testing"],"_links":{"self":[{"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/posts\/88","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/comments?post=88"}],"version-history":[{"count":7,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/posts\/88\/revisions"}],"predecessor-version":[{"id":404,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/posts\/88\/revisions\/404"}],"wp:attachment":[{"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/media?parent=88"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/categories?post=88"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scipilot.org\/blog\/wp-json\/wp\/v2\/tags?post=88"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}