Note – I spend about 40% of my time talking to companies thinking of hiring a CRO firm to help them improve conversion rates. Some companies ask the right questions. Some don’t. So, I have compiled a list of the questions you should think about and ask as you looking into working with a conversion agency.
I update this article twice a year. We started with 12 questions, and we are up to 22 🙂
Hiring the right conversion optimization firm will have a significant impact on your bottom line.
A sharp CRO agency with a concrete methodology can generate significant increases in your conversion rates. Hiring the wrong team, on the other hand, will be throwing money down the drain. Even worse, it will be wasting meaningful time you could be using to generate more revenue.
There is no way around doing your due diligence when hiring a conversion rate optimization firm. You will have to look at the company track record, talk to the team working on the project, and set clear expectations.
This article will cover the types of company, the general cost of services, as well as the additional and specific questions you should ask any CRO firm you are thinking about hiring.
Different types of conversion optimization agencies
You can find several types of conversion optimization agencies:
Individual consultants: For the last three years, the number of freelance consultants who offer conversions increased considerably.
Search engine marketing companies: Several search engine marketing companies have added CRO to the list of services they provide to their clients.
Testing companies: Some multivariate and A/B testing software companies offer professional services to help their customers create test plans.
Specialized conversions optimization firms: These are companies that focus strictly on conversion optimization and do not handle any other types of marketing such as SEO, PPC, etc.
Which type of conversion optimization agency is right for your business depends on several factors.
Individual consultants come at a lower cost. However, they cannot bring the same breadth and depth of knowledge as a conversion firm.
Most search engine marketing companies offer CRO as an afterthought and not as a specialized practice in which they invest heavily. If you are already working with an SEM company for PPC and SEO, adding CRO could work to ensure synergy between the different teams.
Testing companies are great at creating software, and their professional services team will do their best to ensure your success. The challenge is that, by definition, most of these companies are focused on the testing aspect of a CRO program and do not delve deep into the other portions of a conversion project.
Specialized conversion optimization service firms are a mixed bag. Some have a well-established track record and a proven methodology. Working with these companies will help you increase conversion rates and impact your whole marketing approach. On the other hand, some firms want to conduct three to six projects, bill the client, and move on to the next one without delivering real results.
Cost of conversion optimization companies
I wrote a detailed post about the cost of hiring conversion optimization agencies. Keep in mind that a good consultant should increase your annual online revenue from 30% to 50%.
CRO consultants will charge anywhere from $40/hour up to $250/hour. A specialized conversion optimization company will most likely charge a monthly retainer of $2,000 up to $30,000 per month. Many of them will also require a multi-month commitment.
23 questions you should ask a conversion optimization firm
These are the specific questions you should ask before hiring a CRO firm:
1. How do you determine what pages you will optimize?
Choosing which page to optimize first and creating a conversion roadmap will impact your project. Remember that optimizing any page requires analysis, development of test plan, implementation of that test plan, and finally waiting for testing results.
This could translate into 2 to 4 weeks’ worth of work per page.
If the CRO firm chooses the wrong page, that time and money are wasted.
What you need to focus on is understanding the agency’s process to determine what to work on and how to prioritize different conversion-related work.
There should be no shortcuts here. Without following a process or giving a generic answer, such as using best practices, the firm is just randomly guessing.
If they are randomly guessing from the start, avoid them.
A top-tier conversion agency will use a mix of the following activities to come up with a conversion roadmap for the site:
- Expert reviews & heuristic analysis
- Jobs to be done interviews (JTBD)
- User research (qualitative analysis including on-site polls)
- Data analysis ((quantitative analysis)
- Usability testing
- Competitive analysis
2. How do you determine what changes you will make on a page?
This question helps you understand how the conversion company will determine what changes they should make on the page. Again, you are digging deep to understand the process the conversion firm uses.
You want to validate:
- The process they use to determine what elements will be changed and tested
- How that process will be applied to your particular website
Best practices are great in theory but difficult in practice.
How does that impact you?
Relying on good practices will not take you anywhere. We have seen this first hand with over 3,000 A/B tests. What works for one website may or may not work for your website. Different conversion optimization designs and processes work differently for websites in the same industry.
You can find several renowned methodologies in the CRO space. Invesp’s Conversion Framework, Widefunnel’s LIFT model, and MarketingExperiments MEC formula are well-known conversion models in the industry.
It is good when a company says they follow a specific process, but you should always ask to see a small demo of how that process would be applied on your website. The CRO company should choose a page of your site and use its conversion process to demonstrate weak areas on that page.
3. What type of user research (qualitative research) will you conduct on the project?
User research explains the “why” question on your website and informs your assumptions about your visitors. Multivariate and A/B tests validate these assumptions.
Good conversion optimization requires the firm to conduct some level of user research to understand your customers.
CRO qualitative research is a vast and complex field, including:
- Jobs to be done interviews
- Online surveys
- Focus groups
- One-on-One interviews
- Website exit surveys
You should avoid companies that ignore doing qualitative research, regardless of the excuse they might give.
Optimizing a website for conversion without conducting qualitative research is flying blind.
4. What type of data (quantitative) research will you conduct on the project?
Quantitative research answers the “what” question on your website:
- What are visitors doing on your website?
- What pages are performing well, and what pages are underperforming?
- What areas of your website are leaking visitors?
Analytics software provides a wealth of information for quantitative research.
You should assess the firm’s ability to provide deep analysis and insights beyond exit and bounce rates. Assess the CRO firm processes towards:
- Data segmentation
- Funnel and goal creation
- Heatmap analysis
5. What is the most challenging part of doing a conversion optimization project?
Increasing a website conversion rate is a complex problem. Otherwise, most websites will convert at 20 or 30%. Solving the conversion problems for a website requires a careful analysis that generates focused solutions with a higher probability of success.
You should ask this particular question to the team members who will work on your project. Listen carefully to their answer.
Do they find the analytics assessment difficult, or is it qualitative research? Perhaps, it is identifying problems on a page or creating a successful A/B test.
Your follow-up question is, “what are you doing to mitigate these challenges on my particular project?”
6. What was the worst conversion optimization project you worked on?
Most companies think of conversion optimization as a one-time effort, a campaign they conduct once, and they are done with it. Needless to say, that mentality is counterproductive.
As a result, a company that has been doing conversion optimization long enough must have run into a bad project now and then. If the company claims they never did, they are lying or have not been doing CRO long enough.
This is not unique to conversion optimization. It is the nature of doing business. Large and well-established consulting firms in all different industries have bad projects.
Assess the answer of the company about their worst project by analyzing:
- The reasons the firm gives for the bad project. Are they blaming the client? Even with lousy clients, and these do exist, a consulting firm should have a process to manage expectations and implementation.
- How the firm handles bad projects.
7. How many experiments do you launch per month?
It goes without saying, the more tests you can launch on a website, the higher the chances you have of increasing your website conversion rate. This, however, should NEVER come at the expense of following all the required and necessary steps to create a fruitful A/B test.
There is a fine balance between launching scant tests and an exaggerated number of tests.
You must understand how often the CRO company plans on launching tests on your website. You will also have to allocate your resources to conduct a test. If the company is running just a few tests, your development team would not be involved. These few tests are typically small and have a limited impact on your bottom line.
Different companies take different approaches to the number of tests they launch:
- Some CRO companies conduct one test every four to six weeks
- Some CRO companies conduct two tests per month
- Some CRO firms conduct three or more tests per month
Conducting one test every four to six weeks is too slow. Following this approach, you will conduct anywhere from 8 to 12 tests per year. In our experience, that will generate a lower ROI.
Conducting more than three tests per month means that tests are not carefully analyzed, nor are results studied to determine marketing insights. This approach of throwing things at the wall rarely works.
Conducting two tests per month is the ideal target where careful analysis is put into test creation and results. In this approach, you will launch 20-24 tests per year. An expert CRO team should easily generate a 35 to 65% uplift in conversions in this scenario.
8. Do you conduct concurrent split tests?
Our recommendation to conduct multiple tests per month comes with the caveat that:
- The tests are not run concurrently, or
- If tests are run concurrently, each test runs in a separate swim lane.
If tests run in separate swim lanes, traffic from one test intersects traffic from the other test. This rarely happens on large complex websites where visitors jump from one side to the next.
9. How do you handle mobile website optimization?
Mobile traffic accounts for 35 to 50% 70% of online visitors to most websites. Examining your website analytics will help you understand:
- What percentage of your website visitors are coming on mobile devices
- What is the conversion rate for mobile visitors compared to desktop visitors
Mobile website optimization requires a different set of processes compared to desktop optimization. Visitors on mobile devices typically come in two buying stages: the research phase or the action phase. Presenting mobile visitors with the right information will have a significant impact on your bottom line.
When evaluating different conversion rate optimization companies, ask them for examples of their mobile device optimization. Also, understand their mobile optimization process.
Stating that mobile device optimization is similar to desktop optimization is a red flag!
10. Who will handle implementing the test on our website?
Understanding who will handle implementing the A/B tests ensures you will allocate the required resources to guarantee the project’s success. It will also provide you with insights into the type of tests the firm plans on introducing during the project.
AB tests require development time. The amount of time depends on the complexity of the AB testing that you will run on the site. Most AB tests are implemented using front-end developers. These tests use javascript to manipulate how a page appears on the site. Some CRO agencies ask you to handle the development of the AB tests yourself. And some clients like that since it helps them control the cost of the project.
This, however, is sot a good idea.
I have seen it fail one too many times for two reasons:
- Your development team gets busy.
- AB test development is different from regular development. It requires a completely different skill-set.
Look for an agency that will handle the AB test development for you. That will save you a lot of time/effort/money.
11. What type of resource commitment do you require from my team?
Most conversion optimization services will require some level of commitment from you. You should meet with the company regularly to see their progress, provide feedback, and analyze results.
When you have a winner for an AB test, then your development team should implement that winner on your site. So, do account for some development time from your side as well.
12. What additional fees should I expect to pay in addition to the consulting fees?
Expectations must be set correctly. That will ensure the success of any conversion optimization project.
The conversion consulting agency will require you to install and use conversion optimization software to collect visitor data and launch the tests.
Some of the tracking and A/B testing software can get expensive.
This will largely depend on your website and the number of visitors it gets. If your website gets more than 100,000 visitors per month, the cost for the testing software can run you up to $500 per month. Even more, some software will require an annual contract commitment.
A recent company seeking conversion rate optimization services had 500,000 monthly visitors to their website. They were not allocating any money for the testing software. They had been negotiating with several optimization firms. No one bothered to explain to them that the cost of the testing software would run them close to $20,000 per year.
13. What type of education do you require from your conversion optimization staff?
Conversion optimization is an ever-changing field. If online marketing is on the cutting edge of marketing, conversion rate optimization is on the cutting edge of online marketing.
Good conversion optimization companies should require and invest in educating their staff. At a minimum, a conversion rate optimization consultant should be staying up-to-date with the latest in the conversion optimization field and analytics.
In addition, and since conversion optimization impacts different aspects of a website, an expert conversion optimization consultant should have a well-rounded knowledge of SEO, PPC, and social media.
The education a firm requires from its staff is a great insight into how much they want to see their clients succeed.
14. Can you guarantee an increase in conversion rate?
This an easy one. If anyone guarantees results for conversion optimization services, run away from them as fast as you can.
Many factors impact the result of conversion work. Among these factors, several are not controlled by the conversion rate optimization firm.
Take, for example, the quality of the traffic that is coming to your website. If the site gets a good number of visitors organically from search engines, a change in the quality of that traffic (such as an increase or a decrease in rankings) will directly impact your conversion rate.
The conversion company also does not control some market factors. A major ad campaign of one of your direct competitors will impact your conversion rate.
In our experience, companies that guarantee conversion optimization results fall in one of two.
- Companies that are trying to make quick money and will not be able to deliver on their promises
- Companies that guarantee an uplift define the uplift as for any type of success on any test they launch. This includes conducting a test that generates a 0.00001% increase in conversions!
Both practices are somewhat deceptive.
15. How do you determine the success of the project?
Website owners conduct conversion rate optimization because they want to see an increase in conversion rate revenue. But as we clarified in the previous question, no reputable CRO company can guarantee the results of their work.
If you set expectations correctly from the start of the project, you will save yourself many headaches and heartburn.
When we talk to clients, we define the success of a project according to three factors:
- The percentage of successful tests: in our experience, up to 70% of the tests we conduct generate an uplift in conversion.
- The detailed and well-thought process we demonstrate first-hand to the client in every interaction with our team.
- The learning process the client team receives from watching and interacting with our team on the project.
16. Why should we hire you vs. another?
Why is one firm different than another one?
Differences could include process, experience, deliverables, and cost.
All of these are valid differences that will have a direct impact on the success of your project. Ultimately, your goals are to:
- Assess how their firm distinguishes itself from competitors
- Validate the claim of the company on its differentiators
17. What is the makeup of the team working on my project?
CRO agencies have different approaches to how they structure their teams. At a minimum, your project team should include the following people:
- CRO specialist
- UX designer
- Front-end developer
- Quality assurance analyst
Smaller agencies might not have a project manager staffed. However, if you are going to run multiple experiments simultaneously, you will need someone to make sure work is running smoothly.
18. How many projects does the team working with me handle
I get asked about how many clients we handle at any point in time all the time. We limit the number of our projects to 15 at the same time. But that is our approach. We have a high-touch approach to running a CRO project. A better question is how many projects does a team handle at the same time.
Top-tier CRO agencies work in pods. A pod typically handles 3-4 projects at the most. That allows the CRO pod to focus on the clients and deliver the best results for them.
19. How often do we meet? Do you have weekly or monthly standing meetings?
There are two sides to this.
- For you as a client, meetings are important. You get to know what the team is working on. You provide feedback. You learn. I can go on about the benefits of meetings.
- For the agency, meetings eat time that could be used in doing work (for you or other clients).
While I appreciate that agencies try to watch the time they spend on a project, I think that successful CRO projects require regular meetings. My recommendation is to expect at least a meeting every two weeks, preferably once a week.
20. What methods of communication will we use on the project?
In addition to the standing meetings, how else are you going to communicate with the CRO agency? Avoid email at all costs. it is difficult to track. Instantaneous chat programs are okay, but it is NOT a good idea if you run a high-velocity testing program. A good project management tool (there are hundreds of them) provides a central repository of all the work on the project.
22. How fast do you launch the first AB test on my site?
While conversion research is important, some companies spend 6 to 8 weeks doing research. Their first AB test is not launched until the end of the second month. I do not think you need to wait that long to launch your first AB test. Our goal is to launch the first AB test within 2 weeks from the start of the project (3 weeks at the most).
23. How much time do you spend on my project?
Let’s face it. The agency goal is to spend as little time as possible on your project while generating the most wins for you. It is a win-win strategy.
At least, in theory.
CRO projects require a lot of time. There are too many resources involved.
Agencies that try to cut corners, end up launching simple tests (test the color of a button or the text on a CTA). While some of these experiments generate some results, it is NOT a winning long-term strategy.
Trying to measure the activity of a CRO agency by looking at the number of experiments they produce in a month is also not a good idea. You can create 10 experiments in month in less than 10 hours.
I like to understand how many hours the team I will be working with will spend on my project.
What if the agency doesn’t tell me how much time they spend on my project?
You can do simple math and figure this out.
Ask two questions:
How many people work in the agency.
How many clients they handle at the same time.
Let’s say that an agency has 25 resources. Most likely, 20 of them will be working on projects. Let’s also assume that the agency handles 50 clients at the same time.
So, the time of these 20 resources is spent working on 50 projects.
Let’s say a full-time resource spends 120 hours a month doing actual CRO work. That means that the company spends 20 resources * 120 hours = 2,400 hours working on client projects.
How much time do they spend working with each client?
2400 hours/ 50 clients =48 hours at best.
