You get what you measure
By
Marcelo FernandesView All Posts
While not surfing, a computer scientist that loves to work out of the comfort zone, pushing the boundaries of knowledge. A Hublot Eyewear spectator.
Top Stories
1 - You get what you measure
Virtually everybody knows that "If you cannot measure it, you cannot improve it" (by Lord Kelvin). However, let's be honest, metrics are a very touchy subject.
You have probably experienced both of the following types of projects:
- No data is available. With no metrics, you need to be guided by gut instinct only;
- Dozens of metrics with an evidence-driven approach. Metrics are the end goal.
And, I believe your life was neither easy nor happy in either of them, right?!
Lord Kelvin, a Scottish-Irish mathematical physicist and engineer born in 1824, also said that "when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.'' Therefore, you need some way to measure your advancements (or regressions) if you want to know if you are progressing in a specific area.
Metrics in software engineering projects have been studied for decades. Over 1.5 million books, blog posts and academic papers have been published so far on the subject.
For agile software engineering teams, the right interpretation of metrics can be extremely useful. Not only does it provide an awareness of how the team works together, but it is also favourable to continuous delivery.
Those are probably the two main motivations for each agile method, to come with a pre-established set of metrics to be applied out-of-the-box. In other words, metrics are an intrinsic part of any agile method.
For example, eXtreme Programming uses the Team Velocity and the Yesterday's Weather concepts to guide teams to commit to the amount of work to be performed in the next iteration in a sustained way. On the other hand, Kanban practitioners are strong advocates of measuring the Lead Time and the Cycle time.
There are at least 4 areas where metrics are considered at Farfetch globally: team health, customer satisfaction, delivery and cross-team dependencies.
Team health
Team health
We really care about our people at Farfetch.
From giving each other frequent feedback and keeping goals clear and focused, to not being afraid to delegate and develop skills. We are always finding ways to improve Farfetchers productivity and happiness.
Be Human is a Farfetch value about being respectful and kind to yourself and the people around you. Trust each other’s intent, resolve problems with empathy and welcome mistakes as learning opportunities.
Additionally, the Farfetch leadership group works hard every day to make Farfetchers happy and unlock their potential to be able to perform at their very best.
In addition to our focus on people and culture initiatives, we continue to follow Lord Kelvin's advice. That's why twice a year all Farfetchers are invited to answer a survey from which we derive metrics which aims to answer the following questions:
- How happy is our team?
- Do we plan to stay?
- Is our vision, goals and roles clear?
- How do Farfetchers feel about our culture and team dynamics?
- How agile (self-organisation, sustainable pace, size & skills, workspace, etc.) are we?
- How confident are we about our performance?
- How is leadership doing (process improvement, people development, technical expertise, effective facilitation, etc.)?
From the responses to the survey, we build high-level enterprise views of the health of our teams from both a quantitative and qualitative perspective. This allows every Farfetcher to understand their team, cluster, domain and organisation’s health.
Delivery metrics
Our Todos Juntos value advocates that every Farfetcher should act for the greater good of all teams, even when it’s beyond your area of expertise. Foster positive actions in those around you and celebrate our shared achievements.
A survey is a tool that helps us to grow the team health, to increase performance and to improve alignment to business outcomes. Actually, the survey is used as a way to get actionable insights which are then translated into tangible actions that compose, what we call, the team's growth plan.
While some actions may be translated as part of the next Objectives and Key Results cycle, other actions are handled immediately. Around 6 months later, we iterate, collecting the metrics again and set about working to improve even more.
At Farfetch, we have the "Amaze Customers" value. We love and understand our customers, keeping them front of mind and exceeding their expectations by delivering unforgettable experiences.
Cross-team dependency metrics
We believe metrics should be treated as part of an engineering discipline. They should be used as a way of estimating how much teams can get done in a given period of time, and as a tool to measure progress either in terms of project or product delivery.
Actually, this is an agile mindset that is at the heart of our global roadmap planning, as well as the local sprint planning of the teams that have adopted an iteration-driven approach.
Moreover, we always emphasise that defining and tracking metrics should be something immediate and straightforward to do. Otherwise, complexity prevents them from being something they can readily benefit from.
Overall, the key metrics every software engineering team tracks are:
- The number of user stories, tasks and bugs delivered per period of time (e.g. sprint or month), together with their trend and evolution analysis;
- The total story points delivered per period of time;
- The delivery focus (e.g. how much teams have been working on maintenance, roadmap, non-roadmap or cross domain initiatives);
- Cycle time, sometimes with a subsequent root cause analysis of the long-lasting deliveries.
At this moment you may be thinking "Hey! Why are you not using <add a metric here> which is as important as the ones you said?". Well, our metrics evolve over time. We may be about to substitute one of them for the very metric you are thinking of.
The most important concern is always to follow the aforementioned mindset and culture. What gives us huge flexibility is the fact that we have data automatically extracted from the (commercial) agile management tool we use.
Then, after processing such data, metrics are computed and reported to the stakeholders of each team's cluster through an in-house developed tool.
Be Revolutionary is a value that drives Farfetchers to disrupt the status quo and embrace innovation with a curious mindset. Make courageous decisions, searching for new ways to shape the future.
So, it's kind of easy for us to pivot and start computing any other metric (once data is available, of course).
At Farfetch, we are building applications based on a geo-distributed microservices architectural style. There are thousands of loosely coupled applications organised around business capabilities.
System complexity, conflicting priorities, dynamic product scopes, and overlapping projects and product release cycles are just some reasons why dependency management is so difficult in such an environment. Missed deadlines, non-stop meetings and chaotic context switching probably are the most painful consequences of poor cross-team dependency management.
As a result, a challenging area that is of paramount importance for us is the management of dependencies between the software engineering teams that are building and maintaining our microservices architecture. We look to reduce cross-team dependencies as much as possible, something that could prevent the continuous delivery of value. It's really a challenging process while keeping an agile mindset.
Conclusion
Think Global is a Farfetch value that reflects our global community, working collectively as a united force. We welcome and respect everyone, creating an inclusive environment for all.
From a project and portfolio management perspective, we seek to anticipate as much as possible (in an agile way) the rise of dependencies so that alignment and clear communication between teams is guaranteed throughout the project lifecycle as well as the product roadmap process.
From an application architecture angle, we manage to merge components and/or slice and scatter to break up bottlenecks and minimise cross-team blocking chances.
Finally, from a metrics point of view, we track the number of dependencies between teams and clusters of teams. Also, the status of each dependency - as well as when it needs to be fulfilled to prevent teams from getting blocked - is a key aspect of dependency management.
Again, we make use of our in-house developed tool to deal with cross-team dependency management and track such metrics.
Conclusion
Metrics are only one part of the software engineering equation.
Metrics should always help teams to achieve their best in a sustainable way. But we should always pay attention if we are getting too obsessed with them. And if you think they are your end goal, you are most likely driving in the wrong direction.
As engineering leaders, we should always strive to get all the contextual information out of the metrics that we can and try to understand the relationship between them. This is key to making better and effective decisions.
Be Brilliant is a Farfetch value that encourages you to be unrivalled in everything you do, with passion and ambition. Ambiguity and complexity never stop you from going the extra mile.
We believe we should always be aware when it's time to retire the current metrics and start thinking of new and more useful ones. All in all, you will get what you measure.
