Software Quality Assurance: the road to Zero Defects

Cliquer ici pour la version française.

Continuous Integration has been in place in Isogeo for more than two years now, which has been an excellent way to release fairly consistent versions of our backend components (our API for the main part) over time. Our main achievement in 2014 in this domain was to be able to integrate our frontend components (our main application) in the same process. This was a bit challenging because of technology differences (Javascript vs .NET) but we are pretty pleased by the results.

Our goal in 2015 is to go further down that road and improve our Deployment Pipeline to ensure an overall better quality and a better availability of our platform. We want to be able to test more, more consistently and get us closer to a Continuous Delivery platform.

State of the process

Obviously (?) the first and most important part of a Continous Integration is the Revision Control component: all our projects are checked in in as many hosted Git repositories. Every repository is monitored by a build server that is in charge of ensuring that the developers commit consistent code (Continuous Build) and that creates complete deployable packages every night (Nightly Build).

Continuous Build

The Continuous Build checks that, for every commit:

  • the source code is consistent, ie does not break the build. This ensures that other developers can retrieve these changes locally and still be able to compile the projects.
  • the resulting software meets our most basic quality criteria:
    • it must pass all of our unit tests.
    • it must adhere to our internal guidelines, which are enforced by a set of dedicated tools (FxCop or JSHint).

Every developer must monitor the state of the build at all times, and a broken build must be fixed immediately.

Continuous Build

The build server configuration can be kept fairly simply simple thanks to the use of:

  • conventions that define the way projects are organized (certain folders of files are here for certain purposes). This is also very important for developers, who can jump from one project another and quickly understand where everything is.
  • package managers that handle dependencies in a simple way (NuGet or npm). One of these dependencies is a set of build scripts that work on every project that adheres to the aforementionned conventions. You can find the source code for these scripts on GitHub.

Nightly Build

The Nightly Build is basically an extension of the previous build. It is triggered every night and it does everything the Continuous Build does and:

  • it creates a deployable package for the project, deployable meaning that we will ultimately be able to deploy it on all our platforms (we currently have Integration, QualityAssurance and Production) with only a reconfiguration. The technology we use to create such packages is Microsoft Web Deploy which is very suited to the task.
  • it automatically deploys the package on our Integration platform. This platform is self-hosted inside Isogeo and can be used by developers and testers alike to ensure that all the components put together still form a coherent platform.

We also have a QualityAssurance platform that is hosted on Windows Azure (as is our Production platform). It is used to test the platform in a realistic environment before the release of a new version, every three months. Most of our components can be deployed on Windows Azure using the same Web Deploy technology, but our API requires to be packaged in a specific Azure package format. We use another set of scripts to manually create such packages from the build server packages.

Nightly Build

The tests that are carried out on these platforms are manual at the moment.

Once deployed our platforms are monitored:

The road ahead

Our current process makes the deployment of a new version a breeze, and we are fairly confident in doing it. But there are loads of improvements to be made in certain areas to greatly raise the bar in the quality of our platform: we need more automated tests, and we need to get rid of the few remaining manual tasks required in the deployement process.

Automated tests

The main goal for 2015 is to improve the quality and the coverage of our current tests:

  • we need more unit tests on all our components. This is an evergoing process.
  • we need automated integration tests for our API. The main effort resides in the creation of a completely managed test database that can be used to test for all our use cases. And we are currently evaluating the use of RunScope as our test platform.
  • we need automated integration tests for our applications. This only requires the same test database as above. Once this is done, tools like Selenium can be used for that purpose.

These tests will not be part of the Continuous Integration process because they have too many dependencies: the fact that for any reason the test database server is unavailable or a web server is down should not prevent us to create a new package and deploy a hotfix for instance.

Automated deployment

We also need a Deployment Server that can automatically mix a set of configurations with a set of packages and deploy a new version of our software on any given platform (Integration, QualityAssurance or Production). The main benefits of such a platform would be to:

  • completely automate the deployment of our software so that it would not require a high technical expertise anymore.
  • allow to automatically run part or all of our automated integration tests on any platform.

Continuous Deployment

The road is straight, but the slope is steep! Anyway we are looking forward to develop these new processes and tools so that our users can enjoy their experience on an evermore reliable platform: Isogeo.

Soyez le premier à commenter

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *

Unable to load the Are You a Human PlayThru™. Please contact the site owner to report the problem.