How to Host a Completely Virtual Data Hackathon

FollowingFollow

Growing great data teams, advocating science-based thinking, with cats, board games and a Barbour jacket.

Part 1: Pitching your idea

1/ Gather your initial guiding coalition

Who is enthusiastic about the topic? Who has participated in hackathons or hack days before? Do you need any initial advice from key players?

In our case, we brought together a number of data science leaders who had previous experience with hackathons, and we added our data community manager. It also really helped to have someone from our People team involved in the early stages because they could help us #behuman with our ideas. Plus, we needed someone who had access to some company-wide resources. This was really important when almost all of the organisers had been at FARFETCH for less than one year!

For a summary of the important questions to ask yourselves as you get started see: Want to host a hackathon? Questions to answer before you start.

2/ Decide what your goal is

This is really important. When you say 'hackathon' some people have visions of a nerds-only 2 am sweatpants and cold pizza-fest. A lot of data scientists will probably picture an algorithm-specific type problem, where people try different approaches to the same problem on the same dataset (like Kaggle). For others, it means something more like a Design Sprint, with a broader, more design-focused approach to problem-solving.

You and your coalition of organisers need to decide what type of hackathon you want to build. Host a meeting and ask yourselves the following questions:

Who do we want to participate in the hackathon? Is it a relatively specific group of people? (Just data scientists or just people working on a particular product area?) Or is it broader than that? How broad? What if someone from a completely different department wanted to participate? Would that be ok?
What are the goals of the hackathon? What do you want to get out of it? Is having fun and learning something new more important than building something that works? Or do you really need to have something that can be put into production at the end of it?
What are your logistical constraints? What time zones will people be in? Are there any dates coming up that would be particularly good or bad to schedule a hackathon? How long will the hackathon be? One day? More? Are you expecting people to participate outside of normal working hours and would this be reasonable? Do you need to think about a minimum or a maximum number of participants?
What kind of support will you need? Think about how you'll incentivise people to participate. Will you need managers to give permission for people to have time away from normal duties or to decline meetings? Will you need prizes? What about additional support from other business functions like HR, Finance, Legal or Internal Communications? Do you need senior leaders to support, maybe by judging the submissions?

For our DataHack2020 we decided that we wanted the event to be open to as many people as possible, data people and non-data people. But, we wanted the focus to be strongly data-based. To do this, we aimed for high participation among data people, and thanks to official sponsorship from the Chief Data Officer, everyone in data already had permission to rearrange their work as much as possible to be able to participate. We also made sure that the ‘data’ component was central to our theme, even if the theme was broad enough to appeal to anyone: ‘Using data to put the customer at the heart of the FARFETCH experience to drive sustainable and profitable growth.’

We decided that meeting new people and having fun was more important than building something production-worthy, so we specified that submissions didn't necessarily have to be working code, but could be things like mock-ups, or exploratory data analysis.

We also wanted to use the hackathon as a chance to improve data literacy across the company, especially to get people who didn't think of themselves as 'data' people to start playing with data.

3/ Create your pitch document

Once you and your fellow organisers have agreed on your vision, draft a 1-2 page 'pitch' document. Here are some headings you might want to include:

Hackathon mission: the goal you agreed on with your team
Hackathon theme: write this as you would communicate it to your participants. Maybe at this stage, you have a couple of different ideas or only a vague sense of what would be appropriate. This is a great place where senior leaders can give their input.
Who are the participants?
What will the participants do?
How will the 'winners' be determined? Are there any judging criteria you know you'll need?
What data will people need access to? Are there specific datasets that are required? Is there any pre-processing needed?
Timeline of events - include the key events of the hackathon itself, plus pencil in some deadlines for things like sign-ups and pre-event communications.
Support requested: be specific and list out the things you're asking for and from whom

4/ Pitch your idea

Get official sign-off from senior leadership. Be ambitious and take it as high as it needs to. We were lucky that our idea had the support of our Chief Data Officer from the beginning, who was then able to showcase our idea to other FARFETCH Execs. It meant that we had a fantastic panel of Exec judges for our submissions, which helped enhance the profile of our event.

Part 2: Preparing for the Hackathon

So, you've been given a thumbs up to go ahead and been granted the resource that you asked for. What's next?

Here's a timeline: Timeline checklist for hosting a virtual hackathon.

1/ As soon as possible:

1.1/ Refine your problem theme

If you're going for a Kaggle-style data hack, your problem statement will probably be quite specific. If you’re going for a broader problem, that’s ok too.

For our Hack Day, our theme was quite broad: ‘Using data to put the customer at the heart of the FARFETCH experience to drive sustainable and profitable growth.’ The advantage of such a broad theme was that there were a number of interesting data-angles that anyone across the company could explore, from interesting sustainability challenges, to cost analysis, to amazing customer experiences.

The disadvantage of such a broad theme is that it might make people feel like they don’t know where to start. To address this problem, we had a company-wide panel discussion about the theme, with leaders across FARFETCH talking about what kind of ideas they would find exciting. There’s more detail about our panel talk below.

1.2/ Start writing FAQs

Circulate your pitch document to your key influencers. These can be both junior and senior people. Get their feedback. Start talking it up in 1-1s and team meetings to see what people think about your ideas. Keep track of the questions people ask and stick it in an FAQ document that all your fellow organisers have access to. You will inevitably come up with ambiguities and uncertainties that you as organisers need to resolve, and this document will help you keep track of it. It also means that when any prospective participants have questions, you can be confident that you’ll be giving them consistent answers.

1.3/ Get your internal marketing together

Make someone responsible for your internal marketing and communication for the event. In our case, it helped having someone with design skills who could make all of our documents and communications look slick, including our very own logo for the event! Since FARFETCH has an internal communication team, we were also able to liaise with them to advertise the event.

1.4/ Figure out what prizes you will offer

With a virtual hackathon across 9 countries, we had to pick 'virtual' prizes and give people an expense-able food budget for the day, because it wouldn't have been efficient to send people things individually. How do you want to allocate prizes? Do you want to go with the top judges pick? Do you want to have a popular vote? Are there any special categories that you want to recognise?

For our hack day, we chose the top judges pick as our first-prize winner. We also posted all of the video submissions internally and asked people to vote for the popular choice prize with a thumbs up. We also wanted to recognise a brilliant idea that merited further investigation, even if it didn't win the top prize because it wasn't feasible, which we called a 'moon shot' prize. We decided to let the judges nominate as many 'moon shot' prizes as they wanted.

1.5/ Create your sign up documents

You’ll need to draft a 'launch and sign-up' communication that you use to officially kick off the advertising for your event. This might be a re-working of your pitch deck, but from the participants' perspective. Make sure you also give people a place to go if they have additional questions. We launched our #datahack2020 open slack channel on the same day to create a central place where people could go.

At the same time, create your participant sign-up sheet. Think about what kind of information you’ll need to know about your participants. Are there any eligibility criteria? Will people be able to sign up as individuals? Or as teams? In our case, we gave them both options but we reserved the right to move people into teams. Most people signed up as individuals, and we built teams for them, making sure that people were from at least two different locations, at least two different departments, but while trying to keep time zones as close together as possible. Since our event was open to anyone at Farfetch, we also made sure that every team had a ‘data’ person, like a data scientist, data analyst or engineer to provide the technical expertise needed. If you have any similar parameters about team formation that you want to track, make sure you include these in your signup form. A Google Form made it easy to collect sign ups and any additional information we needed.

1.6/ Secure your judges' time

Judges are surprisingly important to make the event feel worthwhile for your participants. Go for the highest profile people you can, and reach out early because the more senior they are, the sooner their calendars will get booked up! Make sure your judges are aligned with the theme of your event, and are actually able to critique the submissions in the way that is needed. Create some publicity material to showcase your judges to potential participants (see Figure 2).

Figure 2. FARFETCH DataHack advertising material showing our judges and prize scheme.

2/ With 4 weeks to go:

Send your launch communications. Monitor your signups and communication channels (email, slack) for questions. Make use of your centrally-available FAQ (we used a Google doc) and keep updating when you get new questions. At this point, you don't necessarily have to open the signups. You can give people some time to think about your theme and start talking about what teams they want to make. Announcing the date with four weeks to go gives people a chance to block off their calendar with enough notice.

If you are sending any hackathon swag by post (rather than purely virtual like we did), check your shipping costs and dates now.

If you haven't already, start thinking about how the submission and judging process will go. What will people need to submit, and where? We decided to create a google drive folder for each team to submit whatever additional supporting documents they might have. We also asked each team to submit a Google Slides presentation.

3/ With 3 weeks to go:

Open your sign-up list and watch your participants come in!

Consider also holding other virtual publicity events to promote the hack day. We mentioned it on big department-wide meetings and promoted it on popular slack channels.

5/ With 2 weeks to go

At about this time, we realised that we were getting lots of people signing up, which was great, but it meant that it was going to be hard to judge all the submissions! 10 minutes per team with 28 teams was going to be almost 5 hours of judging!

To overcome this problem we split the judging into two stages: a group stage and a final stage. At the group stage, we split all the teams between 4 judges and assigned one judge to each group. The top 1 or 2 picks from each individual judge was then sent to the final round with all 4 judges. Each of the 4 judges was assigned someone from the organising team to act as a facilitator for the group-stage judging session.

To make things easier for our teams, we asked them to submit a video of their presentation, rather than present live. The video recording part was easy, since our teams were able to use the 'record meeting' function in our video conferencing software. However, you also have to make sure you can play the recordings back when everyone is together for the judging.

Before you do this, check that you can stream video on your video conferencing software. Due to the various video compression optimisations, you usually need to enable a separate mode to share a video in a meeting, rather than simply share your screen which is playing the video. Most remote meeting software systems have a way to make this work, so check that yours does! We ended up doing a couple of test calls to make sure that all the organisers knew how to do this.

6/ With One week to go

This is the final push for sign-ups! We organised additional promotional events in the days leading up to our hackathons, some 'how-to' or 'introduction' sessions provided by some of our 3rd party vendors to introduce people to some of the internal tools we use like Looker.

We also hosted a panel discussion with senior leaders (including some of our judges) talking about our hackathon theme and the sorts of problems we could tackle. This was a great way to help teams translate our broad theme into a specific project that would work for them.

We got 4 senior leaders from different areas of the company to participate, and we briefed them with a list of questions ahead of the event. We used questions like:

What big unsolved problems do you see in your areas where Data might be able to help reduce costs and drive efficiencies while still improving the customer experience?
Which trends do you foresee in the near future, either in your area in luxury fashion as a whole, that you think our Hack Day participants should be aware of?
What things should our Hack Day participants consider to build a solution that is authentic to FARFETCH’s brand and also technologically feasible?

We used a senior leader from within the Data department to moderate the event, and help keep the different perspectives balanced. There was very little preparation required, but it resulted in a high quality and informative session for everyone.

Part 3: The Big Day

Two days before

Close your sign-up list and finalise your teams. Inevitably at this point you will have people message you telling you that they'd like to sign up or drop out last minute. This was about 5% of people in our case. Luckily the number of drop-outs was approximately equal to the number of drop-ins, so we were usually able to find a slot for everyone. We also had whole teams drop out after signing up, which was easier to accommodate than individuals switching things up. We used a central spreadsheet to keep track of who was in which team.

Once we had the teams finalised, we divided each of them into their 'judging' groups, which meant that they each had one of the organisers as a 'facilitator'. In our case, each facilitator was looking after 7 teams. Because everyone was remote, we relied heavily on Slack for coordinating (see Figure 1). Each facilitator created a separate slack channel for each of their teams, adding all the team members. Since in some cases they didn't know each other already, we sent them an opening message encouraging them to get to know each other and maybe start thinking of an idea. It was helpful to have a facilitator involved in each team so that we could keep an eye on them. It helped us see if we had people dropping out who hadn't told us, and it meant that we could quickly answer questions on the day. We also had separate slack channels to coordinate amongst the organisers, and for the organisers to communicate to all the judges.

Figure 1. Diagram illustrating the slack communication strategy for FARFETCH’s virtual hackathon.

The day before

Host a brief session for everyone to officially 'launch' the Hackathon. Depending on your schedule, you may be able to do this immediately before you start. In our case, we did it the afternoon before. It was a chance for senior leaders to once again voice their support, for us to review the format for the day and for people to ask questions.

The day of the hack

We wanted our 'hack day' to be flexible, to accommodate people in different timezones. Although we set the submission deadline to 23:59 GMT, we told people that we only expected them to work a typical 8 hour day, and that they should coordinate with their team about what they wanted to do and when.

The teams were great at organising themselves - many of them just had a single video call going all day while they worked together. Others had a come-together, divide and come-together approach. All of our hack day organisers were based in India, UK and Portugal, and we were a little worried about signing off at the end of the day in case problems arose with our teams in the US. However, on the day, we saw that there were relatively few problems that we needed to step in to solve, so we were confident about switching off (and maybe keeping one eye on slack) and leaving the teams to it.

On the actual day, you’ll probably not be very busy, since your hard work is mostly finished, and now it’s over to the teams! This is a great chance for you to create a post-event survey. You’ve probably noticed things that might be worrying you: are people having fun? Were our instructions clear? Are they super burned out and stressed? A post-event survey is an important tool to find out whether the signals you’re seeing are real and widespread. It’s also a great chance to learn about things you might not have thought of to improve your event next time.

Our post event survey included some numeric scale questions asking participants how much from 1 (not a all) to 5 (very much) they agreed with different statements:

"I received sufficient information about the event in the run up to the Hack Day."
"I understood the purpose of the event when I signed up.”
"I am proud of the work my team did for the DataHack.”
"I was inspired by the theme of the DataHack”

We also asked participants how many hours they spent working on the hack over the course of the week, which parts they thought were well done, and which they thought needed improvement. By asking them a net-promoter style question (‘How likely are you to recommend the DataHack event to others if we ran it again?’) we would also be able to benchmark the quality of our event compared to future ones. Finally, we left some open text fields for people to give more detailed feedback.

The day after the hack

With face-to-face hackathons, you need to have the judging as part of the same event. With our virtual hackathon we were able to postpone the judging until the following day, to a time that worked for global time zones.

In the morning, one of our organisers based in India made sure all the submissions were correctly uploaded so that people could view them internally and vote for the popular vote winner. Each of the 4 facilitators checked that they could upload the submitted videos so that they could be played for the group judging stage.

It was a bit hectic trying to compress 28 teams worth of work into just a few hours of judging. We used 4 separate simultaneous meetings to judge the teams at the group stage. Each of these meetings was facilitated by one of the organisers.

We allowed the meeting to be open to anyone who might be interested in attending, although in practice we didn't have many people wander in. Then we held a quick 15 minute call with the judges so that they could talk through what they saw. We made a spreadsheet template for each of them to fill out so that all the facilitators knew what they were thinking in real time.

Once we narrowed the 28 teams down to 6 finalists, we had to quickly announce the finalists and let them know that they were up for the final presentation. The final presentation was done using our company's live streaming video conferencing software so that our participant size wasn't capped and we didn't have to worry about asking people to keep themselves on mute.

It was really helpful having all the video presentations pre-recorded since it made this last-minute announcement easier. Next time, we could have spread out the judging onto two different days to make it less frantic, or even scheduled a longer break between the two sessions.

It was really helpful to have an open slack channel dedicated to the hackathon so that we could post quick updates to let people know what was going on with the judging.

We announced the final winners over the livestreaming call, and our communications expert quickly made some beautiful-looking visuals of the team names to make it feel special. Once the winners were announced, we needed nothing more than a final word of congratulations from our Chief Data Officer before we all signed off. You can find out more about how successful the day was here.

Things we want to do better next time

We thought we were really clear about the expectations that people could ask for the full day's worth of time from their manager to participate, and that we didn't expect people to work beyond that. The feedback that we got from our participants was that in some cases it was frustrating to have team members keep dropping out to go to other meetings, and not be able to participate fully.

A lot of our survey respondents said that it was difficult to maintain a good work/life balance during the hack, and we saw lots of people continuing to be active online into the evening when they would have typically stopped work earlier.

We're not confident that spreading the hack day over two days would solve this problem since it might just make it worse if people try to do even more work! Next time, we'll make sure to reinforce our communications about work-life balance and our expectation that it should feel like a normal working day.

Although technical feasibility was a really important part of our judging criteria, in practice we found that our more technically-minded people wanted to be able to ask questions about each other's work. This was hard to do with the way we set up the timeline since there were only 5 minutes of video and 5 minutes of questions (from the Judge) per submission.

Since we also posted all of the videos internally, we could have used the video posts as a forum for a technical Q&A within the first 24 hours. This would give people a chance to publicly see this technical critique, and if we postponed the judging by one day, it would allow the judged to benefit from this critique.

We wanted to give people access to 'mentors' - perhaps people who were experts in specific subject matter areas that could be available to answer questions. In practice, this didn't really work out because if a team was inspired to solve a problem related to a specific area, it was likely because they already had a subject matter expert in the group. Plus, our theme was deliberately vague, meaning that we couldn't really pre-populate a group of volunteers to be on standby, since we didn't know what sorts of questions people would ask. On the other hand, in some cases it helped having an outside perspective for the teams to bounce ideas off of. Having this role of ‘mentor’ being separate from ‘organizer’ helped so that the organizers could focus on any last-minute logistical needs, which inevitably come up. Overall, we think mentors would be a good idea to try in the future, especially if we chose to go with a more specific problem theme.

Smart things we figured out that we should definitely remember next time:

Use the live streaming software for big events, rather than having to keep asking people to mute themselves!
Slack channels made it really easy to keep track of everyone, especially with a facilitator in each team's channel, plus an open channel for the whole event
The video recorded presentation was way easier than trying to get people to stick to time with a live presentation. The results were also really high quality.
The live panel about the hackathon theme was a great way to inspire people about the topic and didn't require much-advanced preparation

We hope this post inspires you to host your own virtual hackathon in your own company. From pitch, to prep, to the big day itself, we hope you can benefit from our experience, and ‘do what’s never been done’ in true FARFETCH style.