Adding Protractor Tests and | The JHipster Mini-Book

Earlier this year, I integrated end-to-end tests into a client's project. Later, I figured out how to get them running on Jenkins. This past weekend, I decided to take my learnings and apply them to the JHipster Mini-Book's example project: 21-Points Health.

This article explains how to create end-to-end tests with Protractor, configure them to work in a JHipster project, and run them in Jenkins. As an added bonus, I'll show you how to take screenshots for every test and run an analysis on your JavaScript source code.

Testing with Protractor

I started by adding some new dependencies in package.json for running Protractor and capturing screenshots.

npm install bower --save
npm install protractor --save
npm install grunt-protractor-runner --save-dev
npm install protractor-jasmine2-screenshot-reporter --save-dev

I added a post install script to package.json so it'd run Bower and download the necessary webdriver files.

"scripts": {
  "postinstall": "bower install && webdriver-manager update"
}

Next, I created src/test/javascript/protractor.conf.js and configured JUnitXmlReporter and HtmlScreenshotReporter.

var HtmlScreenshotReporter = require("protractor-jasmine2-screenshot-reporter");
var JasmineReporters = require('jasmine-reporters');

exports.config = {
    seleniumServerJar: '../../../node_modules/protractor/selenium/selenium-server-standalone-2.47.1.jar',
    chromeDriver: '../../../node_modules/protractor/selenium/chromedriver',
    allScriptsTimeout: 20000,

    specs: [
        'e2e/*.js'
    ],

    capabilities: {
        'browserName': 'chrome',
        'phantomjs.binary.path': require('phantomjs').path,
        'phantomjs.ghostdriver.cli.args': ['--loglevel=DEBUG']
    },

    baseUrl: 'http://localhost:8080/',

    framework: 'jasmine2',

    jasmineNodeOpts: {
        showColors: true,
        defaultTimeoutInterval: 30000
    },

    onPrepare: function() {
        browser.driver.manage().window().setSize(1280, 1024);
        jasmine.getEnv().addReporter(new JasmineReporters.JUnitXmlReporter({
            savePath: 'build/reports/e2e',
            consolidateAll: false
        }));
        jasmine.getEnv().addReporter(new HtmlScreenshotReporter({
            dest: "build/reports/e2e/screenshots"
        }));
    }
};

I added a protractor task in Gruntfile.js:

protractor: {
    options: {
        // Location of your protractor config file
        configFile: 'src/test/javascript/protractor.conf.js',

        // Do you want the output to use fun colors?
        noColor: true,

        // Set to true if you would like to use the Protractor command line debugging tool
        // debug: true,

        // Additional arguments that are passed to the webdriver command
        args: {}
    },
    e2e: {
        options: {
            // Stops Grunt process if a test fails
            keepAlive: false
        }
    },
    continuous: {
        options: {
            keepAlive: true
        }
    }
}

I created an alias to this task so I could run the tests using "grunt itest".

grunt.registerTask('itest', ['protractor:continuous']);

I had to make some adjustments in src/test/javascript/karma.conf.js to exclude protractor.conf.js and its tests from being processed.

-            'test/javascript/**/!(karma.conf).js'
+            'test/javascript/**/!(karma.conf|protractor.conf).js'
         ],


         // list of files / patterns to exclude
-        exclude: [],
+        exclude: [
+            'test/javascript/e2e/**'
+        ],

Finally, I created src/test/javascript/e2e/account.js and wrote tests to verify authentication works. I added ids on menu links to make elements easier to find. I also found Protractor's element explorer offers a handy way to debug tests and execute them line-by-line.

'use strict';

describe('account', function () {

    beforeAll(function () {
        browser.get('/');
        browser.driver.wait(protractor.until.elementIsVisible(element(by.css('h1'))));
    });

    it('should fail to login with bad password', function () {
        expect(element.all(by.css('h1')).first().getText()).toMatch(/Welcome!/);
        element.all(by.css('[ui-sref="login"]')).get(1).click();

        element(by.model('username')).sendKeys('admin');
        element(by.model('password')).sendKeys('foo');
        element(by.css('button[type=submit]')).click();

        var error = $('.alert-danger').getText();
        expect(error).toMatch(/Authentication failed!/);
    });

    it('should login successfully with admin account', function () {
        expect(element.all(by.css('h1')).first().getText()).toMatch(/Authentication/);

        element(by.model('username')).clear().sendKeys('admin');
        element(by.model('password')).clear().sendKeys('admin');
        element(by.css('button[type=submit]')).click();

        expect(element.all(by.css('h1')).first().getText()).toMatch(/Hello, Administrator!/);
    });

    it('should be able to update settings', function () {
        element(by.id('account-menu')).click();
        element(by.css('[ui-sref="settings"]')).click();

        expect(element(by.css('h2')).getText()).toMatch(/User settings for \[admin\]/);
        element(by.css('button[type=submit]')).click();

        var message = $('.alert-success').getText();
        expect(message).toMatch(/Settings saved!/);
    });

    afterAll(function () {
        element(by.id('account-menu')).click();
        element(by.id('logout')).click();
    });
});

After creating these files and configuring Grunt, I was able to start Spring Boot in one terminal (gradlew bootRun) and run the tests (grunt itest) in a second one. I was able to verify screenshots and a report were created at build/reports/e2e/screenshots. I moved onto trying to get it all working in a continuous integration environment.

Running in Jenkins

I recently wrote about how to build and test a JHipster project in Jenkins. These instructions are now included in the README.md of new JHipster projects. This section builds on those instructions and only details what needs to be changed.

I configured an Execute Shell command to take place after the Invoke Gradle script in the Build section.

./gradlew bootRun &
bootPid=$!
sleep 30s
grunt itest
kill $bootPid

This was enough to get my tests passing, but I wanted more!

Publishing HTML reports

To publish the screenshots and their associated HTML report, I added the HTML Publisher Plugin. I configured it as a post-build action, and specified the screenshot reports directory.

Post-build Actions

Publish HTML Reports
- HTML directory to archive: build/reports/e2e/screenshots
- Index page[s]: report.html
- Report title: Test Screenshots

Console log colors

One of the issues I encountered with Grunt was that Jenkins doesn't render its logs in color. To see colors in your console logs, you have to install the AnsiColor Plugin. After installing, you'll have to configure your job to use it in the Build Environment / Color ANSI Console Output section.

Xvfb

I only run the 21-Points project in Jenkins on my laptop, so I haven't had to configure Xvfb myself. I believe installing Xvfb is the hardest part of getting Protractor tests running on a headless environment. Once you have Xvfb installed and configured on your server, you can install the Xvfb Plugin for Jenkins. Enable it in your build and you'll be taking screenshots in no time!

Plato Metrics

Another tool I found useful is plato. This is an analysis tool that scans your code and analyses it according to your .jshintrc file. To generate reports using Grunt, I added grunt-plato as a dependency:

npm install grunt-plato --save-dev

Then I configured a plato task in to Gruntfile.js:

plato: {
    options: {
        title: '21-Points',
        jshint: grunt.file.readJSON('.jshintrc')
    },
    metrics: {
        files: {
            'build/reports/metrics': ['src/main/webapp/scripts/**/*.js']
        }
    }
}

I added a jenkins alias too:

grunt.registerTask('jenkins', ['itest', 'plato']);

I configured the Jenkins job to use this new task as well.

./gradlew bootRun &
bootPid=$!
sleep 30s
grunt jenkins
kill $bootPid

To publish plato's reports in Jenkins, I added another report to the Publish HTML Reports section.

Post-build Actions

Publish HTML Reports
- HTML directory to archive: build/reports/metrics
- Index page[s]: index.html
- Report title: Code Metrics

I noticed there were additional options that allowed archiving results with the build.

After adding this report, the jobs menu has has a Code Metrics link, in addition to Test Screenshots.

I also added the Protractor results to the Publish JUnit test result report section.

Post-build Actions
- Publish JUnit test result report / Test Report XMLs: build/test-results/*.xml,build/reports/e2e/*.xml

Summary

I hope this helps you start using Protractor to write and run end-to-end tests in your JHipster project. I'd be interested in seeing how this same setup works on Travis CI. It'd also be cool to have a Docker image that comes pre-configured with JHipster (Node, Java, etc.), Jenkins, and Xfvb. This would allow JHipster projects to be automated right away. Please let me know if something like this already exists!

Adding Protractor Tests and Automating with Jenkins