Wednesday, December 17, 2014

Cucumber Testing Framework

Cucumber is BDD (behaviour driven development) framework. In contrast to other BDD frameworks (like RSpec) the specification is written in a natural human readable language.

Rubocop-yast

I wanted to write tests for the new Rubocop-yast plugin in some nice way. I started with RSpec, but the tests looked ugly (writing multiline indented Ruby code in a string literal requires extra escaping which makes it quite hard to read...).

Then I looked at the original Zombie Killer tests. They are written in Markdown so they are better readable, you can write some additional comments to the test etc... And how to run the Markdown tests? There is a custom Markdown renderer which converts the Markdown specification into a RSpec test.

Looks nice, but having a custom renderer makes it difficult, we have to maintain it and the Markdown format in specific, there is nothing else similar to it...

Then Cucumber come to my mind! It exactly fits our need! The specification allows to write extra comments and notes, it's readable almost like our Markdown and is a standard tool. The killer feature is multiline docstring parameter. It allows to write indented Ruby code directly without any extra escaping.

You can check how the tests looks here, check the *.feature files. Here is the code which converts the specification into the testing code.

Pros & Cons

Here is the summary of pros & cons I found when starting with Cucumber:

Advantages
  • Specification in natural language, readable tests and user stories
  • Allows to use other testing framework for running the real tests (like RSpec)
Disadvantages
  • Extra code for converting the textual specification to Ruby code
  • One more layer between test description and the code (you need to make sure the code really matches the description)
  • Test descriptions should describe the high level features (usually the user interaction), they should not describe the low-level implementation details
Ideally the features should be written by managers, designers or other non-technical people. They can describe the required features without any programming skills. That's probably the most important Cucumber feature.

Suitable for Yast?

In my opinion not. Why? We are usually focused on low-level features and we would probably need too much code for converting the specifications to tests. And the tests are usually too different, we would need to write extra conversion for each test and maintain it. The overall benefit would be small in my opinion and would not be worth of doing it.

Moreover the feature descriptions we usually get are hard to convert to a testing code, they are usually too generic or cannot be tested in unit tests (e.g. the installer features).

Links


4 comments:

Leslie Lim said...



I simply want to say I am new to weblog and truly savored you’re website. More than likely I’m want to bookmark your blog . You actually have excellent writings. Thanks a bunch for sharing your web site.

Jorcel
www.imarksweb.org

Sherlin Jones said...

The biggest change in Selenium recently has been the inclusion of the WebDriver API. Driving a browser natively as a user would either locally or on a remote machine using the Selenium Server it marks a leap forward in terms of browser automation.

Selenium WebDriver fits in the same role as RC did, and has incorporated the original 1.x bindings. It refers to both the language bindings and the implementations of the individual browser controlling code. This is commonly referred to as just "WebDriver" or sometimes as Selenium 2.
Selenium Training Institute in Chennai

Jones Sathya said...

A nice article here with some useful tips for those who are not used-to comment that frequently. Thanks for this helpful information I agree with all points you have given to us. I will follow all of them.


Selenium Training in Velachery
Selenium Training Center in Chennai

Nancy 12 said...

Thank you for sharing such a nice and interesting blog with us. Hope it might be much useful for us. keep on updating...!! Selenium Training in Chennai