Learning how to make code-sharing between councils work in practice
There’s no pan-local government ‘playbook’ describing how to make code sharing work in practice. We believe this discourages collaboration, and leads to painful reinventions of the wheel in councils where it’s tried.
Our Discovery project will work through the issues that make shared development of code difficult, including:
- issues of fairness in investment, workload, benefits and ownership of IP
- resource implications initially and over time, including protecting resource against competing pressures
- varied skills, development approaches and technologies in each council (agile vs other methods, tools, testing, code review, governance, support and maintenance, security)
- developing a shared product vision and roadmap, with enough flexibility to create features both jointly and alone
- longer term planning (exploring common user needs across authorities, councils joining and exiting, licensing, growing a vibrant and sustainable community)
An opportunity has arisen from a project recently started by 2 of our councils. Brighton & Hove City Council and the London Borough of Croydon are jointly developing a Drupal 8 codebase for use by their separate websites, and have begun to work through some of the issues listed above.
We have 2 partners in Bracknell Forest Council and Oxford City Council who’re interested in adopting the codebase in the future, and are willing to be research participants, feeding in ideas and testing guidance as needed.
A successful LDD fund bid will pay for a separate Discovery team to sit alongside our existing Drupal 8 development staff. They will see first hand the blockers that exist in shared codebase projects, and start to explore solutions that will scale to the wider local gov digital community.
What do you think is causing the problem?
Experience shows us that code collaborations with other LAs are rewarding, but hard to do well. Officers are stretched enough attending to their own demands, let alone considering others.
Councils also tend to think their users are special, and so need custom features. Breaking down silos to focus on common needs requires careful detailed work.
We believe hard lessons are being repeatedly learned in LAs up and down the country. By encouraging collaboration – which is necessary and great to see – it’s possible that the Local Digital Declaration is making the problem more common.
It’s a nice problem to have, but it still needs to be fixed. Our bid will explore the practical guidance needed to make this kind of collaboration straightforward.
What are the dependencies and stakeholders related to the problem e.g suppliers, central government?
The problem we’re tackling lies wholly within local authorities that choose to build digital products together, and relate to differences in:
- digital maturity (specifically an understanding of user needs and agile practices, attitude to open communication)
- user needs in each location
- tools, technology and governance
- team skills and development approaches
- competing priorities
Our Discovery project will address each of these areas in the lead authority and all partners. We will identify blockers to collaboration and test various approaches for overcoming them.
The bid will pay for a new short-term team to work alongside our existing Drupal 8 product teams. They will research the issues above, explore blockers and help us overcome practical problems in real time. We have no dependencies here as the software is open source and both Brighton & Hove and Croydon have necessary development skills in house.
With supportive narrative, please link to any relevant outputs from your previous discovery phase, either with an accessible url or page reference in an attached pdf.
The announcement of BHCC and LBC code sharing can be found on the Croydon Digital blog: https://croydon.digital/2019/07/15/croydon-council-is-sharing-a-publishing-platform-with-brighton-hove/
We aim to test the following hypotheses:
- it’s possible, cost effective and worthwhile for LAs to share development work in pursuit of a common goal
- the problem is about people and how they interact, as well as the technology being used
- there can be a level of collaboration which results in digital solutions which solve shared problems, but allow enough personalisation and configuration for each authority to make it ‘their own’
- the lessons of collaboration are mostly undocumented, leading to authorities reinventing the wheel, costing time and money
- meaningful guidance can be created to help the 4 councils participating in this Discovery, based on their experience of working through the identified issues that make collaboration hard in practice
- whether a common platform (ie Drupal) would be the most effective approach
Our assumptions are as follows:
- that all signatories of the Local Digital Declaration support agile methods
- that it’s possible to resolve the barriers to collaborating on shared code, resulting in ‘plays’ that can be repeated by others facing these barriers
- that the model we’re exploring is about creating a ‘store cupboard’ of code that can be used by all the partners, not a shared product roadmap where teams work together all the time, and every partner has to use everything
- there’s a large pool of LAs that are interested in writing code together, who would benefit from this collaboration
How are current tools, products and other projects addressing the user needs in the problem space?
There are countless tools and products to aid collaboration – many of us have experienced the problem of ‘too many channels’. We think the problem lies in:
- choosing the most appropriate channels to communicate and share
- the technology and cultural constraints in our organisations
- underlining the importance of collaboration, and giving it the time it deserves
- defining lightweight methods of collaborative development that work across LA boundaries
- overcoming concerns about longevity and ongoing support, and showing that it’s worth investing time
- dealing with functional and security flaws
This isn’t as simple as open sourcing our code and accepting pull requests on Github. We know of previous projects that have done this, and failed to be sustainable long term. Our belief is that collaboration is as much about challenging attitudes and ways of working as it is ‘being open’ or fixing technology issues.
The Local Digital Fund has been very successful in encouraging collaboration between LAs, so the user needs around this are being addressed. Project findings are being shared, but they don’t necessarily include ‘what works’ when it comes to collaboration. Our project addresses this directly.
What kind of research methods do you plan to use?
There are two parts to this work:
1) Collating and better understanding what’s already known
LAs have collaborated on countless software projects down the years, and we need to better understand what’s been tried. We will hire a researcher to:
- undertake desk–based research of government blogs, related pages and existing collaboration spaces (e.g. the Local Gov Slack channel)
- conduct phone or Skype interviews with leading collaborators in local government digital
- audit Croydon, Brighton & Hove and the partners’ efforts at collaboration on digital projects (including the Drupal 8 work we’ve already started)
- run group discussions internally and externally as needed to further explore collaborative efforts, and identify best practice
2) Focusing in on what we don’t know
As we collate what we know, we anticipate finding many gaps in knowledge and guidance. These could be vast, so we intend to use agile build-measure-iterate approaches in areas that will deliver against the most common and urgent needs of our teams.
We know already there are knowledge gaps in the following areas:
- Engagement with other LAs
- Setting up a project so commitments are clear and fair
- Day to day delivery
- Issues around exiting a project (what are the indicators a partner should exit? How to leave cleanly with re-useable code?)
As described above, the bid will pay for a new short term team to site alongside our existing Drupal people to explore issues first hand. We intend to hire an agency well established in agile principles to supply product, delivery and technical know–how, with the flexibility to provide skills suited to the challenges we find.
How will this research be undertaken on a cross council basis?
All partners in the project have agreed to:
- set aside the necessary time and people to be involved in the activities above
- be actively involved in supplier selection
- discuss past collaborative efforts
- ensure timely introductions to the relevant people in their organisation
With this in place, our new Discovery team will have autonomy to support collaborative efforts, help remove blockers, collect learnings and, if useful and practicable, produce guidance for the partners and the wider local digital community.
The cost of the problem is well illustrated by the Drupal 8 project that Brighton & Hove and Croydon have begun to collaborate on. We estimate Croydon will immediately save approximately £0.5m by sharing Brighton & Hove’s work, rather than starting from scratch. In time both LAs will share the load of development, leading to further cost savings.
Both Bracknell Forest and Oxford have Drupal 7 corporate publishing systems that are reaching end of life. They’re currently considering options, with joining the shared Drupal 8 project as a leading contender. If all 4 councils adopt our shared platform, the total savings could reach £1.5m. A favourable response from MHCLG to this bid will enable more effective collaboration between the partners, so will make this much more likely to happen.
Rather than reduce dev effort, Brighton & Hove and Croydon have been able to go further faster, giving us the ability to develop more sophisticated solutions in response to user needs. Previously we would have spent the same amount of money delivering the basics.
Describe the national scale of the problem – does this apply in all districts and authorities?
Anecdotal evidence suggests collaboration issues in LAs incur costs in 2 principal ways:
- collaboration is not considered at all, or is discussed and rejected for being ‘too hard’ For example, the author of this bid has been involved with 5 separate website build projects in central and local government totalling approximately £3.5m, all of which could have taken a collaborative approach
- collaboration is attempted, but the project is not successful/ overruns Again, the author has witnessed collaborative projects that didn’t deliver the expected benefits for users in time or to budget. These failed in part due to the challenges outlined elsewhere in this bid: lack of shared understanding of needs, poor tooling and inappropriate project management
Taking publishing alone – a relatively straightforward operation – all LAs have websites, and every organisation we’ve collectively worked for has chosen to build them from scratch. This suggests the collaboration problem is widespread, if not universal.
Where possible, quantify the size or scale of the costs described, to partners or nationally. This will be indicative only, as more robust analysis is expected as part of the Discovery project.
If all authorities collaborated on the creation and support digital publishing tools, the annual saving would be approximately £11m. This is based on the following assumptions:
- 326 local authorities in the UK
- Their digital publishing platforms are replaced every 3 years
- The average cost of replacement is £100,000 (we’ve seen much larger and smaller sums)
Estimating the possible savings for LAs in collaborating on transactional design and technologies is harder, so we’re not going to attempt it here. However, we feel that meaningful guidance on collaboration could start to make a difference.
This is an interesting question, because our project is about finding the right tools and governance structures that can work for us and our partners, and the wider local government digital community.
Brighton & Hove and Croydon have already started collaborating on the Drupal 8 work. Here’s a sample of the tools we’ve tried so far, together with the challenges they’ve created:
Product management and sequencing of work
Both teams use Trello and have access to each other’s Trello boards and upcoming work. We need to do more on this to get the necessary visibility of each other’s roadmaps and priorities in order to get the maximum value from our shared development efforts.
Both teams use Github and we’ve managed to share some code. We need to better understand the nuances of the Drupal platform to focus on what specifically is being shared. We’ve not merged any code as yet, but have begun discussions about the best way to do this.
Day to day communication
This has been challenging. We used a private channel on the Local Gov Slack to agree a collaboration, but access on both sides is limited due to governance restrictions. We’ve resorted to email for essential communications which isn’t ideal. Both Brighton & Hove and Croydon use Sharepoint and Microsoft Teams so this is one of the avenues we intend to explore.
Access to video calling facilities for groups is limited, and there’s no substitute for face to face communication.
Wider collaboration with Bracknell Forest and Oxford will involve discussions about the above. The bid would allow us to explore many more options, and offer useful guidance to the wider local government digital community.
What governance structures will you use to allow the project partners to work together effectively throughout the project?
Again, the purpose of the bid is to experiment with governance to establish a project that works for multiple partners long term.
We’re at the beginning of this journey. As Brighton & Hove and Croydon have begun to understand each others’ roadmaps, we’ve started to explore ways in which we can build product features that benefit all users in a timely and effective way.
Governance conversations with Bracknell Forest and Oxford have yet to begin. It may be that one of the partners is particularly active, and the other less so – a pattern that is likely to be repeated across the wider local government landscape. We should have governance that caters for all.
It helps that all LAs understand agile ways of working and the importance of active engagement with users, both within our organisations and outside. We’ve not yet had a difference of opinion, but we believe our shared approach will stand us in good stead.
Looking further into the future, a public organisation with a less agile approach may wish to share our work. Any governance arrangements should consider this carefully.
The success of this project depends in part on engaging seasoned collaborators from the local digital community. A successful bid will go a long way to encouraging contributions from them. Additionally, we would welcome MHCLG introductions to key individuals and local authorities.
We would appreciate MHCLG support in procuring a specialist agency via the Digital Marketplace.
If our Discovery work leads to the development of playbook style content, it needs to be hosted somewhere. The MHCLG Local Gov Digital pages are an ideal location, and it would be good to discuss this.