Exploring the value of open-source practices and communities so that local authorities may benefit from shared, reusable technical capabilities
Open source communities and practices in technology have been around since the early 1960s with many academics distributing free and open software to encourage and support openness and co-operation. As many will already be aware, some of the most well known software and organisations on the planet are founded on the very principle of open-source and communities to support the collaborative development of technologies that are then distributed freely amongst those that need it.
In a similar vain, a search for ‘localgov’ on the popular technology social network GitHub yields 52 results. The https://government.github.com/community/ page lists 58 central government agencies making their code and technologies readily available as open source projects, however, there are only 29/408 UK Councils listed.
Equally, over the past few years GDS have facilitated and promoted the sharing, collaboration and reuse of many of the assets they’ve created.
There are 408 UK councils with common services and common problems, these councils consume software by common vendors for common purposes and yet many of them are creating common workarounds, common integrations for common use cases without creating common code bases.
We believe there are a number of reasons for this, based on interviews with some technical architects and technology professionals across 6 different local authorities and our own experiences.
Perceived risk is potentially a big factor in the reluctance to both share and reuse of in-house developed software. The risk of sharing the source of software openly with others has a perceived security risk which we suspect is linked to confidence in security and modern software development practices like Dev(Sec)Ops. It stands to reason that the potential risk of reusing and open source project is equally linked to similar foundational problems, confidence in testing, assurance, and quality.
Ongoing support is another reason that organisations may avoid investing in or reusing open-source products, when buying from a vendor local authorities often sign a contract for the vendor to provide annual support and maintenance. With an open-source product, that support is potentially provided in-house, by a community, or by a separate support contract by a third party.
Time to value has been another common response from users to date. It’s fairly trivial to take most open-source projects and get them running on a laptop, however, the expertise, assurance, design and cross-team understanding needed to deploy that project in a production environment where it can deliver value to users is significantly more and potentially a barrier to reuse.
Although service design has been given a huge amount of support and endorsement across government and we’re beginning to understand users needs and how to design better services for people better than ever, there are still vast amounts of duplicated efforts going into technology implementations.
As a service is designed for the better, making systems talk to each other, making data work, augmenting and extending off the shelf software to ‘make it fit’ that newly designed service is still a huge undertaking, one that’s once again common up and down the UK. Vendors of business software aren’t going away any time soon and neither should they, yet the software local authorities buy from vendors is still stifling digital momentum. Poor or costly APIs, little or no data standards and structure or archaic architectures that make delivering on our newly designed service incredibly costly.
We see open-source communities and practices as an opportunity to reduce wasted and duplicated effort, taking advantage of not just open-source code but using the now readily available automation technologies to create open-source projects that can be deployed automatically for local authorities to start generating value sooner with minimal effort.
There are many authorities and agencies open-sourcing their code and products which does meet an internal need to be open. It also creates the opportunity to take advantage of the community aspects of open-source projects like cross-sector collaboration, crowd-sourcing ever-improving solutions, however, this in many cases appears to be left as missed opportunities.
After a short scan of existing GitHub repositories, there is little evidence of collaboration outside an organisation of contributing to projects that have been made open-source, there is equally little evidence of attempts to create a community around those open source projects in an effort to draw in outside contributions to create more robust solutions.
Whilst there are some examples of code-base reuse; Essex using Stockport’s CMS open source integrations, Croydon and Adur and Worthing and Brighton and Hove working from similar project sources on using Drupal 8 and the GovIntranet WordPress boilerplate used by multiple authorities across the UK, there is generally little evidence of open source projects being used in a production environment by another organisation or individual beyond the creators.
Some assumptions for this are:
- The open-source project doesn’t actually meet user needs
- Paying lip-service to openness without the actual intent for collaboration
- opening up the source code because of the implied positive perception rather than intent to share, cynical but a possibility
- Not having a mechanism in place to audit, shape and define contributions from other parties
- Publishing source code without publishing the product roadmap, making it difficult for outside contributors to know what to work on
- Little to no documentation on how to make use of the code creating additional barriers to outside contributors or consumers being able to run a working version of the project
- Difficult to quickly demonstrate value to stakeholders – it’s often simple to have a project running on a developer laptop but potentially much more difficult to deploy it to production where it can deliver value
- Lack of skills or awareness in open-source community building where open sourcing code alone does not stimulate a thriving community of shared development and reuse
- Lack of skills or awareness in modern software development technologies like CI/CD pipelines, automated testing, infrastructure as code and operational automation technologies to enable fast, repeatable, reliable deployment – this is based on the little evidence of those technologies being visible in existing local authority “open-source” projects.
- Perceived risk to organisations re-using software published by another organisation based on trust, security and assurance of that software
- Local authorities attempting to commercialise software they’ve built in an effort to generate income rather than open it up for others to use. Potentially due to decreasing budgets and therefore any bespoke development is considered an ‘investment’ with an ROI attached.
- For those organisations that haven’t published anything at all, we assume this is either a perceived risk, lack of skills or lack of incentive to do so – or a combination.
In an effort to research these assumptions and considerations, we’ll explore through;
- user research in the organisations that have currently published open-source projects to understand incentives
- to include interviews, shadowing, pair programming and process mapping
- user research and shadowing in those that have yet to publish projects openly
- to include interviews, shadowing, process mapping and exploring any existing un-shared code bases
- user research with common local government vendors to explore their incentives and behaviours regarding re-use of their technology and existing user groups and communities
- to include interviews, shadowing and attendance of events
- user research with existing open-source communities to explore how successful and unsuccessful initiatives
- user research with vendors that have open-source initiatives to explore how
- quantitative data gathering and analysis on the state of technology use for software development across local government
- quantitative data gathering, analysis and user research interviews to explore the state of skills and continuous learning culture within local authorities
- quantitive data gathering, analysis and user research to explore the performance metrics associated with in-house service design and digital teams within local authorities
We’ll be conducting this research across authorities using a variety of tools for remote collaboration (video calls, surveys, social media, Pipeline and GitHub etc) whilst also attending existing events and facilitating a series of small workshops to bring authorities and vendors together. We may also create a small scale prototype project to explore what barriers exist to organisations contributing to, adopting and re-deploying an open-source project that meets needs.
This problem of repetition in technology particularly is a national one, it applies from parish and town councils right up to the biggest of Counties and Unitaries.
The biggest cost is estimated to be payment to technology vendors themselves for repeated work and implementations of the same, that aside, a conservative estimate might look like…
Average development costs for the partners of this project on small scale projects (one simple system integration for example) = ~£400pd * 30 days * 1 developer = £12,000. If we add a very conservative 50% for technical design, QA, testing and deployment processes we get to £18,000.
We then estimate that the problem that’s just been solved by this authority, integrating to this system or data set is potentially reuseable by 1/3 (136) of the authorities in the UK since not all authorities run the same services. Of that 1/3, 1/4 (34) use the same vendor system since for a majority of main services have common vendors across the sector.
After all of that, if we’re continually more conservative and say a common solution was reused 15 times, 18000 * 15 = £270,000 for one reused solution.
Council’s run an inordinate number of services with a vast number of data sets, integrations and solutions, therefore, anecdotally, a scaling up of this problem-solution might look like;
If a quarter of the authorities in the country (102) created one reusable solution with sufficient support, risk mitigation and value proposition and each one of those solutions was reused by 15 of its counterparts, that’s 1530 (102 * 15) implementations. Take the original 18k investment in each one of those solutions and that’s 1530 * £18,000 = £27,540,000.
Although this is wildly unlikely to be realised, we believe the potential opportunity to reduce the cost of digital change in local government by creating a sustainable open-source, collaborative culture is one that is still going vastly unrealised.
Our governance structure will include a nominated product owner to own the direction and focus for the research and any steps beyond the Discovery research. The multi-disciplinary team will then work in 2 week cycles with output being published continuously but formalised and reviewed every two weeks to ensure we’re taking account of any learning and whether we need to pivot to incorporate any new information as we go. Bi-weekly calls with an open invite will also be booked so that anyone with an interest on the teams progress can hear about what we’ve learned first hand. This also means that stakeholders that aren’t formally engaged or part of the research can still take part, learn and contribute.
We’ll also complete any reporting back to MHCLG as required, however, we hope that the openness of the research will support the minimal amount of specific reporting back as all information will be as open and freely available as possible.
We’ve already set up a shared document sharing space in our shared storage tool, Huddle. All partners and stakeholders will have an invite to the Huddle space to have full access to all the outputs from research as they happen.
We’ll also be using;
- GitHub as a project backlog for exploring research topics – this is so that our research can be version controlled and we establish open-source principles ourselves from the very beginning
- Slack (LocalGovDigital) for day to day messaging communication
- Skype for Business / Whereby for video calls – uploading video call recordings to YouTube for transparency
- GitHub pages / Jekyll on GitHub pages for documenting research and findings openly and version control for transparency
- Medium / Pipeline for weeknotes and documenting project learnings and updates
- Calendly and Doodle for meeting arrangements
- Eventbrite for event management and attendance
- Harvest for time recording and forecasting spend against budget
- AzureDevOps / GitHub for any prototyping or experiments with existing open-source projects
- We’ll also be travelling to events or be on site with a few stakeholders so that they can take part in the most direct way.
We may need some support with contacts within other open-source communities like the Mozilla Foundation, or industry experts who are already running similar communities of practice, although we have some ties and network connections in some of these organisations, the broader our research candidates can be, the better we’ll understand the context and the more we’ll be able to share with the sector.
Within the team, we have already procured from the Digital Marketplace and attended many LocalDigital training sessions or received formal training and as a shared service across many organisations day to day, we’re fairly confident of our ability to bring the team together across multiple locations and organisations. We’re solely focused on delivering valuable learning and outputs that the whole sector can benefit from.