Monday, May 30, 2016

Ruby/Cucumber and vim in multi-project environments...

I'm currently working as a Senior QA Automation Engineer for a major bricks-and-mortar retailer that also has a large online presence. They offer a mobile app, a mobile-website experience, and special support for tablet, Android, and iOS devices. Needless to say, there are a number of different top-level projects, each providing a similar but customized experience for the channel their customers use.

QA automation (QAA) uses ruby/cucumber for automated testing, and the QAA environment provides the 'standard' cucumber structure with features/, features/step_defintions/ for feature files and step definitions. Each top-level project has their own sub-directories so, for example, the mobile-website project uses:
features/mobweb/
features/step_definitions/mobweb
while the tablet project uses:
features/tablet/
features/step_definitions/tablet/
I use vim/tmux for my IDE and depend on vim-cucumber (thanks to Tim Pope for a great plugin) to find relevant step definitions from the feature files. The problem I ran into is that -- because of the multiple project sub-folders -- I was constantly getting the error: "Multiple matching steps found" and thus had to examine each alternative step individually to find the one that related to my project.

This was seriously irritating (not to mention time-consuming) so this weekend I dug into the vim-cucumber code to see what could be done. It turned out to be pretty straight-forward (see my fork of vim-cumber here; I've submitted a pull request and hope it gets merged).

Tim's approach is to first find the top-level cucumber feature file directory (this is in the ftplugin/cucumber.vim file); that could be either features/ or stories/.
let b:cucumber_root = expand('%:p:h:s?.*[\/]\%(features\|stories\)\zs[\/].*??')
That is then saved as a globbable name (e.g., features/**/*.rb):
let b:cucumber_steps_glob = b:cucumber_root.'/**/*.rb'
and later used to find all the relevant ruby files through the use of vim's glob statement:
for file in split(glob(b:cucumber_steps_glob),"\n")
and then iterate through the list looking for the step you are searching for. Nice stuff.

Of course, with the multi-project directory structure we have, there are many 'duplicate' step definitions: we're all tapping on buttons or checking the status of similar things. Think
When(/^I tap on the "(.*)" button$/) do... 
Then(/^I should see the "(.*)" button is (enabled|disabled)$/) do...
as simple examples).

I now have a working solution with a small footprint on the basic plugin. I looked at different ways of doing this, but then hit upon this solution. Export a glob spec to the shell environment variable CUKEFILES. When setting the glob spec, vim-cucumber looks for the existence of this variable and uses it instead of the default value.
if !exists("b:cucumber_steps_glob")
  if empty($CUKEFILES)
    echom "Using default definition for b:cucumber_steps_glob"
    let b:cucumber_steps_glob = b:cucumber_root.'/**/*.rb'
  else
    echom 'Using CUKEFILES environment variable for b:cucumber_steps_glob'
    let b:cucumber_steps_glob = $CUKEFILES
  endif
endif
Once that's set, everything works just as it did before, except with a more precise set of files. Note the slight difference when using CUKEFILES: it assumes that the environment variable already has the /**/*.rb set, so that you could define multiple directories in your glob spec.

The CUKEFILES setting I'm using is:
export CUKEFILES=./features/step_definitions/mobweb/**/*.rb
Now my multiple matches truly indicate duplicate step definitions that I need to examine and probably correct.

Finding a working glob statement was it's own challenge. I wanted to have the option of including more than one directory in the glob spec by using the "|" separator but that's another story

No comments:

Post a Comment