Ruby/Cucumber and vim in multi-project environments...

I'm currently working as a Senior QA Automation Engineer for a major bricks-and-mortar retailer that also has a large online presence. They offer a mobile app, a mobile-website experience, and special support for tablet, Android, and iOS devices. Needless to say, there are a number of different top-level projects, each providing a similar but customized experience for the channel their customers use.

QA automation (QAA) uses ruby/cucumber for automated testing, and the QAA environment provides the 'standard' cucumber structure with features/, features/step_defintions/ for feature files and step definitions. Each top-level project has their own sub-directories so, for example, the mobile-website project uses:

features/mobweb/
features/step_definitions/mobweb

while the tablet project uses:

features/tablet/
features/step_definitions/tablet/

I use vim/tmux for my IDE and depend on vim-cucumber (thanks to Tim Pope for a great plugin) to find relevant step definitions from the feature files. The problem I ran into is that -- because of the multiple project sub-folders -- I was constantly getting the error: "Multiple matching steps found" and thus had to examine each alternative step individually to find the one that related to my project.

This was seriously irritating (not to mention time-consuming) so this weekend I dug into the vim-cucumber code to see what could be done. It turned out to be pretty straight-forward (see my fork of vim-cumber here; I've submitted a pull request and hope it gets merged).

Tim's approach is to first find the top-level cucumber feature file directory (this is in the ftplugin/cucumber.vim file); that could be either features/ or stories/.

let b:cucumber_root = expand('%:p:h:s?.*[\/]\%(features\|stories\)\zs[\/].*??')

That is then saved as a globbable name (e.g., features/**/*.rb):

let b:cucumber_steps_glob = b:cucumber_root.'/**/*.rb'

and later used to find all the relevant ruby files through the use of vim's glob statement:

for file in split(glob(b:cucumber_steps_glob),"\n")

and then iterate through the list looking for the step you are searching for. Nice stuff.

Of course, with the multi-project directory structure we have, there are many 'duplicate' step definitions: we're all tapping on buttons or checking the status of similar things. Think

When(/^I tap on the "(.*)" button$/) do... 
Then(/^I should see the "(.*)" button is (enabled|disabled)$/) do...

as simple examples).

I now have a working solution with a small footprint on the basic plugin. I looked at different ways of doing this, but then hit upon this solution. Export a glob spec to the shell environment variable CUKEFILES. When setting the glob spec, vim-cucumber looks for the existence of this variable and uses it instead of the default value.

if !exists("b:cucumber_steps_glob")
  if empty($CUKEFILES)
    echom "Using default definition for b:cucumber_steps_glob"
    let b:cucumber_steps_glob = b:cucumber_root.'/**/*.rb'
  else
    echom 'Using CUKEFILES environment variable for b:cucumber_steps_glob'
    let b:cucumber_steps_glob = $CUKEFILES
  endif
endif

Once that's set, everything works just as it did before, except with a more precise set of files. Note the slight difference when using CUKEFILES: it assumes that the environment variable already has the /**/*.rb set, so that you could define multiple directories in your glob spec.

The CUKEFILES setting I'm using is:

export CUKEFILES=./features/step_definitions/mobweb/**/*.rb

Now my multiple matches truly indicate duplicate step definitions that I need to examine and probably correct.

Finding a working glob statement was it's own challenge. I wanted to have the option of including more than one directory in the glob spec by using the "|" separator but that's another story

ES6 highlighting and lint checking in vim

I am a long-time "vimmie" (that is, I use vim for my IDE) and I love the plugin ecosystem which lets me do just about anything I want.

For example, I recently started using ReactJS (wish I hadn't wasted to so much time trying to get Angular to work) and one thing I love about it is how straight-forward it is to setup and start writing TDD/BDD for development (but that's another story).  My configuration uses Babel and es6, so I wanted to get syntax checking and indentation/highlighting working right off the bat.

Here's how -- it's pretty easy. I'm using Vundle as my plugin manager so that's the examples given. If you're using pathogen or vim-plug as your plugin manager, it should be pretty easy to modify and get it running. I've used pathogen and it's excellent, but have no experience with vim-plug.

1. If this is the first time you've used a plugin, go to the Vundle website and follow the instructions there.
2. Install the vim-javascript plugin by pangloss: vim-pangloss. This will give you basic syntax-highlighting / indentation using vim's built-in syntax support.
3. Install the syntastic plugin by scrooloose: syntastic (he's the author of the excellent NERDTree plugin which gives you a hierarchical tree menu for system files). This will allow you to use external syntax checkers with vim, in addition to the built-in syntax checking already available. I use the eslint program for this. The options I use in vim for syntastic are:
   

   """"""""""""""""""""" Syntastic """"""""""""""""""""""""
   " From: http://usevim.com/2016/03/07/linting/
   set statusline+=%#warningmsg#
   set statusline+=%{SyntasticStatuslineFlag()}
   set statusline+=%*
   let g:syntastic_always_populate_loc_list = 1
   let g:syntastic_loc_list_height = 5
   let g:syntastic_auto_loc_list = 0
   let g:syntastic_check_on_open = 1
   let g:syntastic_check_on_wq = 1
   let g:syntastic_javascript_checkers = ['eslint']
   let g:syntastic_error_symbol = '❌'
   let g:syntastic_style_error_symbol = '⁉️'
   let g:syntastic_warning_symbol = '⚠️'
   let g:syntastic_style_warning_symbol = '💩'
   highlight link SyntasticErrorSign SignColumn
   highlight link SyntasticWarningSign SignColumn
   highlight link SyntasticStyleErrorSign SignColumn
   highlight link SyntasticStyleWarningSign SignColumn
   """""""""""""""""""""""""""""""""""""""""""""""""""""""
   

4. Install the eslint program. This is a node module installed with the 'global' option so that it's available everywhere as an executable. This is just
   

   npm install -g eslint
   

5. You would think this would work (I sure did) but it's not yet fully baked. With this setup, the first import statement in some of your .js files will be flagged as an error. To solve this issue, you have to give eslint a few parameters, in the file ~/.eslintrc.json
   

   // From: http://eslint.org/docs/user-guide/configuring
   {
       "parserOptions": {
           "ecmaVersion": 6,
           "sourceType": "module",
           "ecmaFeatures": {
               "jsx": true
           }
       },
       "rules": {
           "semi": 2
       }
   }
   

Now you've got full es6 syntax checking and error notifications.

And I am so done with this, too...

Before I get started, let me just say that I like a lot of things about AngularJS.

The basic approach of using directives in the HTML is very appealing and seems natural; filters are easy and intuitive; custom directives provide great options to adding function with little fanfare; data models fit right in to JavaScript; and the ease with which data flows between the view and the data model is pretty much seamless. Google being behind it in a big way is also a major plus.

But what really excited me about AngularJS was that it had "Testability Built-in" [that's a quote from the AngularJS home page]. I'm a believer in TDD/BDD so that was a big selling point for me.

However, my experience trying to get TDD actually working led me to a different conclusion. Here are some of my experiences.

1. Creating a logger requires what? Logging is simple... right? In Ruby it's just:

logger = Logger.new("my_log.log")  # or STDOUT if you want to make it really simple
logger.error "This is my error"

In the Javascript world, it's even simpler:

console.log("This is my error")

Sorry, Angular... you make it really ridiculous:

describe('basic test', function(){
    var log;
    beforeEach(inject(function(_$log_){
        log = _$log_;
    }));

    it('should just work', function(){
        log.info('it worked!');
        expect(log.info.logs).toContain(['it worked!']);
    });
});

And notice the crazy little "trick" where you say _$log_ to get this to work. Yeah, it's "documented" but buried away and not exactly readily available.

2. "injector already created. can not register a module"
Hmmm... Angular knows that we want to register a module, they know that the injector has already been created, so why can't they Just Fix It? They do all kinds of other magic behind the scenes; this sounds like an easy one. If they can't Just Fix It, then how about a meaningful error message and documentation explaining why that really happens. This is one of those cases where you have to know the innards of Angular to be able to figure things out.
See SO: http://stackoverflow.com/questions/24900067/injector-already-created-can-not-register-a-module

3. How many different ways are there to 'inject' stuff into your test.. (I sure can't count 'em). In fact, why do you even have to play that game at all? If you look around, you'll find countless ways to inject things...

from: http://andyshora.com/unit-testing-best-practices-angularjs.html

describe("Unit Testing Examples", function() {

  beforeEach(angular.mock.module('App'));

  it('should have a LoginCtrl controller', function() {
    expect(App.LoginCtrl).toBeDefined();
  });

  it('should have a working LoginService service', inject(['LoginService',
    function(LoginService) {
      expect(LoginService.isValidEmail).not.to.equal(null);

      // test cases - testing for success
      var validEmails = [
        'test@test.com',
        'test@test.co.uk',
        'test734ltylytkliytkryety9ef@jb-fe.com'
      ];

      // test cases - testing for failure
      var invalidEmails = [
        'test@testcom',
        'test@ test.co.uk',
        'ghgf@fe.com.co.',
        'tes@t@test.com',
        ''
      ];

      // you can loop through arrays of test cases like this
      for (var i in validEmails) {
        var valid = LoginService.isValidEmail(validEmails[i]);
        expect(valid).toBeTruthy();
      }
      for (var i in invalidEmails) {
        var valid = LoginService.isValidEmail(invalidEmails[i]);
        expect(valid).toBeFalsy();
      }

    }])
  );
});

from: http://stackoverflow.com/questions/12758157/how-to-inject-dynamically-dependence-in-a-controller

var algoController = function($scope, $injector) {
    $scope.base64 = $injector.get('base64');
};

But then there's the 'dynamic' alternative

var algoController = function($scope, base64) {
    $scope.base64 = base64;
};

Or how about this, from: https://www.airpair.com/angularjs/posts/testing-angular-with-karma

function MyController ($scope) {
  $scope.property = 'value';
}
MyController.$inject = ['$scope'];

But then there's this (which won't work with minified code)

function MyController ($scope) {
  $scope.property = 'value';
}
angular.module('myApp', [])
  .controller('MyController', [
    '$scope',
  myController
]);

I'm sure that all these methods work just fine for the folks who documented them (nothing against them; I certainly respect what they can do). But trying to figure out a sane way to do things when you're getting started is... interesting.

On http://stackoverflow.com/questions/31977313/tdd-with-angularjs-and-protractor.
A question I asked with no response...

I could go on and on with the attempts I made to get a reliable, stable approach to injecting code for testing. In the end, I gave up. I'm sure that Angular supporters will scoff and say: "Hey! It's really easy... just do this".

4. So I asked friends who were working with Angular for some help, I got comments commiserating with me about how it's all not very clear, confusing, overly-complicated, difficult, etc. Some folks even pointed to the fact that you have to know a lot about how Angular works internally before you can easily create real tests.

However, the kicker for me was when I asked a member of the Angular core team about testing and his response was a shrug, a pained expression and the comment: "Yeah; we need to work on that."

That's when I said: "Yeah; I need to move on to something else..."