Two standard workflows embraced by GitHub users are the Fork & Pull Workflow, with developers working on their personal copy of a repository (common for open source), and the Branching Workflow, with developers sharing repositories using multiple branches.
For software teams following the branching workflow, this article explains how Testspace helps manage the quality of their software development process. (Fork & Pull will be covered in another article)
By collecting and analyzing data from the Continuous Integration (CI) Workflow, Testspace provides valuable insights into the CI workflow efficiency and test effectiveness.
The keys to the Analysis and why it Works
The keys to how, and why, Testspace Analysis provides effective results:
- All data from all sources -- test results, code coverage, static analysis, custom metrics, code churn, and defects -- are combined into a single record of health.
- Health records are maintained and charted for every branch -- master, feature, and topic -- across every repository over time.
- Development teams simply follow their normal workflow
By connecting your Testspace account to your GitHub Organizations, collecting data from every branch, for their entire lifecycle, becomes automatic.
Adding Testspace to the CI process is trivial and once setup:
- Developers receive alerts when branches regress, from any data source or metric.
- Leads set and monitor quality objectives for a development branch.
- Managers review and assess overall quality effectiveness spanning multiple repositories.
Connecting to the GitHub Service and Repositories
With Testspace Connected Services, there are no plugins or extensions to install, the Testspace account owner -- logged into Testspace with their GitHub credentials -- simply connects to their GitHub Service.
Creating a new, connected Project in Testspace involves selecting a repository from a GitHub Organization.
Adding Testspace to your Online CI
Adding Testspace to online CI is merely adding a client utility to the install process. Using Travis-CI, as an example, the addition to the yaml configuration file is simply
install: - mkdir -p $HOME/bin - curl -fsSL https://testspace-client.s3.amazonaws.com/testspace-linux.tgz | tar -zxvf- -C $HOME/bin - testspace config url access-token@MY-ORG-NAME.testspace.com script: .. after_script: - testspace results*.xml coverage.xml ...
Branching and Adding Commits
At the heart of the Git Branching Workflow are Topic (or Feature) branches; short term branches used for a single feature or defect. Topic branches are used by developers to create, work on, merge, and delete often several times a day. For this workflow to be both effective and efficient
- The complete definition of branch health (criteria, thresholds, etc.), using all metrics (test, code coverage, etc.), must be applied to each topic branch to ensure base branches remain in a state of healthy.
- All regressions of health on topic branches, from any source, must provide notification, with quick access to actionable details.
New spaces are automatically created the first time data is pushed on a new branch.
Metric charts for every data source -- test, analysis, coverage, etc. -- are created automatically for every new branch.
Each new Space is given a copy of the most recent health record from its base Space at the moment the branch occurs. All metric deltas and trends monitored over time are based on the first health record.
Each time a commit is added -- triggering CI -- new data is pushed to the Space, and all criteria and thresholds are applied. All resulting metrics from all data sources are aggregated into a single indication of health for each commit. Each new health record links back to the commit details in GitHub
The single health indicator provides GitHub with a single check, encapsulating all definitions of health, and with all
Details accessible from the one link.
Pull Requests, Reviews, and Merging
After opening a pull request, and all metrics from all sources have met their health criteria, trends provide valuable data and insights when discussing changes that have occurred on the branch.
Development and master branch leads can make informed decisions before merging back to the base branch
All trends, code churn, health statistics, and all rates of change for each branch are maintained by the Testspace Project regardless if the branch becomes deleted.
The collection of all data, from every branch, across all repositories over time, provides valuable Insights into the
workflow efficiency and
test effectiveness for the entire development process
...the subject of a series of articles coming soon!