Quality Infrastructure

From wiki.gpii
Revision as of 22:52, 7 May 2015 by Avtar (talk | contribs) (First draft)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

The Developer Space Quality Infrastructure Illustrated

Error creating thumbnail: Unable to save thumbnail to destination


The objective of the Quality Infrastructure is to assist the Developer Space in providing high quality components. The QI will help provide developers with an idea of how well D-Space components are maintained, whether automated tests are used, how often those tests pass or fail, and the general health of projects among other aspects. In order to accomplish this the project will employ a great deal of automation in the form of continuous integration, configuration management, tools, and deployment strategies. For example it will automatically perform the following actions when changes are committed to code repositories:

  • Run unit and acceptance tests
  • Build and package the code
  • Publish documentation
  • Notify communities when failures occur

The following is an illustration of the series of events that will take place when a change is made in a participating Github repository:

Error creating thumbnail: Unable to save thumbnail to destination

Areas of Work

Given the broad set of features the QI has to offer it will need to be comprised of several different services, some self-hosted and others hosted by third party organizations. The list of services will evolve as we perform more research and work through early prototypes. The following are some areas of work that will help us direct our efforts and further refine a list of chosen technologies:

  • The D-Space dashboard
  • A backend that:
    • the D-Space dashboard communicates with
    • offers administrators the ability to create developer accounts
    • could be used to manage information about external code repositories and webhook integrations
  • Continuous Integration service (Jenkins for now)
  • Project manifests that developers would maintain
  • Identify third party services that we would like to integrate with (code coverage, etc.)
    • Hosted (coveralls.io)
    • Self-hosted (quail.js)
  • Security (SELinux on build VMs, continuous integration ACLs, repostiory manifest limits, etc.)
  • Ansible roles that would take project manifests and create Jenkins jobs
  • Physical infrastructure where build VMs would be hosted

Proposed Timelines

  • July 2015: Deliverable D201.1 - specification of the QI (Phase 1)
  • November 2017: Prototype required by the DoW
  • February 2016: Early prototype ready for the next review meeting

Phase 1


  1. Produce a pretty good, comprehensive architecture plan for the QI
  2. Produce an early working prototype of the QI Dashboard (UI and backend)
  • Research and architect the parts of the whole solution
    • Diagrams
    • Suggested technologies
    • Risk areas, gaps
    • Design activities:
      • Comparative analysis of similar tools and UIs
        • Github Pulse
        • Other tools
      • Personas US&C for developer space users
        • Developers of components
        • Implementers/users of components
      • Design a) at-a-glance and b) detailed views for each of our category 1 metrics
  • Make a giant list of metrics
    • Split them up into three categories:
      • 1: Metrics we can easily gather now using existing tools
        • e.g. number of committers, repository activity, frequency of releases (commits?)
        • number of unit tests, pass/fail over time
      • 2: Metrics we can gather by pulling together existing tools
      • 3: Metrics we want to gather, but might be very hard
  • Design the dashboard UI for category 1 metrics
  • Implement a simple Node.js backend that will aggregate category 1 metrics from:
    • Github/Bitbucket
    • Jenkins or other CI
    • xUnit test reports