CRO projects are complicated. They require a team, management, accuracy, creativity, and organization. There are many pieces to make a project succeed. In this article, I’m going to break down the ins and outs of a CRO project, and how to help spark creative solutions to inspire greater persuasion (and conversions).
But before we start – what does the team look like?
A CRO team typically looks something like this:
- Conversion optimization specialist (2-3)
- UX designer
- Front-end developer
- Customer research specialist (part-time)
- Analytics specialist (part-time)
- Data analyst (part-time)
- Product manager/Program manager
Holy moly, that’s a lot of people.
According to Dennis van der Heijden:
“CRO is a commitment. Running a test on your homepage isn’t enough. Hiring a consultant for a week isn’t enough. You need time, dedication, coordination, and (though this is often underlooked) enough site traffic—to make conversion optimization work. The best way to think about a CRO strategy is, put simply, “long term.” Then get ready to invest the hours. The results will be well worth the effort.”
Of course, I have yet to see a non-agency team with this many individuals at the helm of CRO. That requires a big budget that most companies are unwilling to allocate for just CRO (although if they only knew what it could unlock…).
I went ahead and asked Khalid Saleh, CEO of Invesp, based on his many years of experience in CRO, what he thought are the components of a successful CRO project. Now, Khalid doesn’t run projects, but he’s pretty aware of the ins and outs of the projects. Based on his knowledge and what he’s seen, he mentioned a few things:
- setting expectations correctly
- using different methods to uncover meaningful insight
- the velocity of testing”
With Khalid, it is always about the client. And it’s true, whenever the client thinks the conversion rates are going to skyrocket after a couple of tests, you know the project is headed in the wrong direction. Also if a client doesn’t realize their involvement, again, that will just impact response time and data collection success.
Additionally, Khalid points to something interesting, his last two points: using different methods to uncover data and velocity of testing both are a cry for increased creativity and out of the box thinking on a test. Why? Well, when you look at the same sources of data, and you aren’t continuously seeking other sources that may relay a different story, you’ll be stuck in a rut. Additionally, if test slowly you’re limiting the scope of the project and ability to push the limits on several elements.
I asked one of our CRO success managers, Hatice, about what she thought the components of a successful project were:
- defining a sound hypothesis
- cross-device testing each launch
- before launching the test, test experience of original and variation designs with a few people who aren’t familiar with test designs
- limiting the number of the testing elements/idea that helps to calculate impacts correctly”
Hatice’s advice is more tactical because she spends the day to day working on projects, so she considers the small things when considering success. It’s interesting to see these very different perspectives from two people who work and handle projects very differently.
Since co-founding the Invesp, I’ve been managing CRO projects. My role has grown over the years, but for growth and learning purposes, I inject myself in some of the projects that we run.
What I’ve noticed, are that “well defined goals” as well as procedure and process following make or break a project. Once the project veers off the rails, whatever the reason may be (client initiated or internal management issues), the project wanes, and the client becomes frustrated.
So ultimately, there are many considerations to managing a project. In this post, I’ll dissect all the different points made by Khalid, Hatice, and myself in order to help you gauge what it takes to lead and manage a successful CRO project.
1. Setting expectations correctly
Whether you’re an in-house CRO expert, or a consultant, you need to set expectations correctly when it comes to the work you are doing. And this is essentially a key point in any project and isn’t unique to CRO. If expectations aren’t set correctly, you are going to end up with angry management.
How do you set expectations correctly. When we take on a full engagement, our client is essentially expecting that their conversion rates will increase. We have to do a little education on what that means: If a test increases CR for a specific page, it doesn’t translate into that increase for the overall site.
Additionally, some basic information on statistics and conversion rates in general is important to clarify how and what increases mean to the success of the project.
2. Using different methods to uncover meaningful insight
Managing a CRO project means the team needs to be able to conduct qualitative data, deduce meaningful data from it, conduct on analytics assessments, read heatmap data effectively, conduct usability tests, and more in order to get the full picture of what is happening on the website.
The reason why I mentioned at the beginning of the post that CRO is not a one man job because the data collection alone requires quite a few experts.
It’s rare to find someone who is good at analytics analysis as well as conducting sound qualitative data and knows UX and UX testing in and out. Krista Seiden from Google explains:
“My step by step process for A/B testing starts with analysis—in my opinion, this is the core of any good testing program. In the analysis stage, the goal is to analyze your analytics data, survey or UX data, or any other sources of customer insight you might have in order to understand where your opportunities for optimization are.”
The scrutinize phase of our conversion optimization SHIP method is 60-70% of the time a consultant spends on the project. Ultimately that research and discovery of the website, site visitors, and burning questions is what shapes the rest of the project.
3. Velocity of testing
We use velocity loosely because it is really highly dependent on the project and website we are working on. I can’t be conducting a lot of testing a site that has limited visitors and conversions. We use a different approach for those specific sites.
However, if a site has the traffic and conversion volume, we take our testing up a notch. What is critical however is to never do what we call: reckless testing. Tests that have no evidence or data or reasoning. Anyone can do that. As a CRO managing a CRO project, well, that is simply unacceptable. Your tests have to be rooted in meaningful data and a meaningful thought process behind it.
4. Defining a sound hypothesis
There’s confusion when it comes to creating a hypothesis. Many may think of as the problem that has been uncovered, while others jump to creating solutions. But that’s not what a hypothesis. We have found that most tests with a sound hypothesis tend to perform well. If we run tests that don’t have a strong hypothesis behind them, statistically, they have a greater likelihood of failure. According to Sharon Hurley Hall:
“When you start your testing without a hypothesis, you’re wasting time. A hypothesis is an idea about what you need to test and why, and what changes you’ll see after you make any changes. With this structure in place, you’ll know the scope of your test and when it succeeds or fails. Without it, your testing is just a guessing game.”
The hypothesis should be a group effort. A single problem can have multiple hypotheses, and it’s up to the team to consider the best hypothesis that will drive that greatest return. Putting the time and effort into defining the hypothesis is an important consideration for the success of a project.
5. Cross-device testing
Image Source: smashingmagazine
It’s sometimes the little things that matter most. Having a precise process on how to go about things is critical in order for a project to succeed and to minimize error. The activities may seem mundane, but less errors equals greater success. That’s why cross browser and device testing is a critical component to a test launch.
6. Pre-test prototype with an unfamiliar eye
You don’t have to do this every time, but very often, prior to developing a concept, prototyping it can help you have a more razor-sharp focus on the validity of a design – especially if it is a time consuming change that takes a lot of development time. Don’t just launch blindly, create a prototype and conduct usability testing to see the viability of the change prior to making it.
7. Limit the number of changes
Sometimes tests are bundled, meaning there are more than one change in a single test. That isn’t always the best approach if you want to identify what change had the greatest impact! That’s why we encourage the changes to be limited in order to pinpoint which change had the greatest impact.
8. Define the goals
Look at any marketing activity or project needs to have well defined goals. It’s really not rocket-science or anything new. The emphasis on KPIs and OKRs is something real that impacts conversion rate optimization first and foremost. The project overall should have an objective, and each change should have an accompanying goal. This will help define the direction of the project and even the way the changes will be made.
9. Process, process, process
I may sound like a broken record, but following a process is really important. We have our processes defined in playbooks for each portion of a CRO project. That doesn’t mean our process isn’t evolving. Online marketing is evolving so your process should NEVER be set in stone. However, if something is tried and succeeds, we like to define it so we can try it again and again, eventually building upon it. But you need to have a structure for a project, otherwise, things can get messy and confusing rather quickly.
The advice is simple, but important in order to have great success in whatever CRO project you are conducting. What do you think are the components of a successful CRO project?
