Server-Side A/B Testing Vs. Client-Side A/B Testing

Wild guesses and opinions often only result in delusional outcomes. Though opinions are by far the most widely accepted truth, they rarely are factual. It is a better practice to lean on deductive reasoning and logic instead of guesswork. But how do you arrive at what is labeled as a fact or a valid statement? Experimentation is the key.

Businesses realize the importance of scientific experimentation and testing. Many product-based enterprises have embedded experimentation in their organizational culture. The reason – innovation cannot result without experimentation. Moreover, though there are millions of ideas that spur every minute, every hour, it is a critical business decision to calculate idea feasibility, and even more important to decide which ideas will work the best.

Experimentation and A/B testing

Anyone who has had experience in running experiments on their websites will concur with what’s written above. And they will know that an experimentation framework includes researching for data, data analysis, drawing hypothesis, testing hypothesis, reporting the findings, and giving feedback on hypothesis drawn. Testing is the next logical step.

Because testing is the secret to making sense out of data, businesses are starting to test everything – from their content to visuals to pricing to usability. A/B testing has emerged to be the de facto of all the website conversion testing techniques, being used by companies across the globe.

There are different types of a/b testing tools that serve different business needs. As a business, you can decide and deploy an a/b testing tool that suits your needs – website a/b testing, email marketing campaign a/b testing, social media ad campaign ab/ testing etc.

One of the bases on which you can differentiate and define your a/b testing requirement is client side vs. server side a/b testing. Most businesses often overlook this differentiating aspect, which can turn out to be a disappointment later on if they find a gap between what they expected out of an a/b testing tool and the actual result.

David Isquick explains;

“Client side: conversion rate optimization: traditionally used to optimize conversion rates in marketing or purchase funnels.”

“Server-side: product success: enables teams to optimize products and logged in experiences for engagement, retention, usage, and lifetime value.”

Differentiating between client side and server side A/B testing

Defining simply, you run a client-side test when you are creating page variations directly on the user’s browser as opposed to your server. Smart client-side tools use Javascript to create the test page variations. Because these alterations and changes are done in the visitors’ browser, it is termed as client-side a/b testing.

Gif Source: Javascript_muratoner

On the other hand, in server side testing, the test is rendered on the web server itself. The page variations are then delivered to the user’s browser where no subsequent modifications take place. The pages being tested are fetched randomly from the server and shown to the visitor.

Let’s understand the difference better using the following examples:

The marketing team at an e-commerce website conducts an analysis to determine why visitors are abandoning their cart. Based on the research, they decided to create a new cart page design. This new design removes the website navigation from the page to reduce distraction from CTAs.

Using a client-side AB testing tool, the new cart page design is created in the AB testing tool. The team uses the split testing tool to select the website navigation and remove it.

Now here is what happening when the test page is delivered with the purpose to perform client-side testing:

1. When a visitor navigates to the cart page, first the original cart page is loaded in the browser.
2. The testing software script is loaded in the page.
3. After the page gets loaded, the testing software script sets in motion the changes made to the page and displays the page to the visitor sans the top navigation.

So with client-side AB testing software, the visitor will always load the original page first, and then the changes are applied on the browser.

Here is what would happen when you are instead going to the server side testing tool:

1. When a visitor navigates to the cart page, the visitor’s browser requests the page the server.
2. The testing software randomly determines which of variation it will send to the visitor browser, so a specific set of visitors view the original page on their browser while the rest view the new cart page design.

This brings us to dig deeper into the differences between client side vs. server side a/b testing.

1.Best used when

Image Source: frinkiac_when

• Client-side a/b testing is best used when you are deploying an internal team of marketers to run experiments without the need for them to touch coding.
• Server-side a/b testing is best used when the scope of an experiment is bigger, for example, changes need to be made at the product level. This will require the coding know how.

2. Ease of use

• The client side a/b testing tools make it easier to implement changes. A WYSIWYG editor or any other visual editor can be used by almost anyone who wants to make quick changes to any element on a web page. For more advanced changes, like a different layout or page design, you might still require frontend developers, but the speed is still faster and cost and difficulty levels on the lower side compared to the server side testing.
• Server-side a/b testing tools are multilayered and complex to use. As you need the team of the front end as well as back-end developers.

3. Depth of experimentation

• Client-side experimentation is limited to the look and feel of the website. Experiments are done to see how changes in webpage elements affect conversions: you might change the look and feel of a single element, or change the layout of a page.
• While server side a/b testing tools can also be effectively used to alter the visual elements of the website, client-side testing is more suited to the purpose. The true value of server-side testing tools lies in the fact that they can be used to, test product features and proposed benefits. To explain, a functional change refers to any modification made to the technical functionality of an app or product. For example, integrating Google analytics experiments with your content management provider. In addition, server-side tools can be used on devices that do not support javascript (such as mobile apps, smart TVs, IoT devices).

4. Objectives

• The ultimate goal of client-side a/b testing is to optimize conversion rates on the webpage.
• While increasing conversions is the ultimate goal of server-side testing, there are also a number of mini goals along the way that one should tick off the list; such as enhancing the product performance, optimizing for engagement and overall product usage.

5. Deployment

• It is simple to deploy or implement client-side a/b testing. All you need is one line of code to run site-wide testing experiments.
• Ease or complexity of deployment of the test completely depends upon server-side technology stack.

6. Performance 

• The JavaScript used for client-side testing is susceptible to the characteristic drawbacks of both synchronous and asynchronous code. While asynchronous code can cause flicker effect, synchronous code often leads to increased page load time.
• With server-side testing, you are creating a self-sufficient new variation as opposed to relying on a piece of code to load itself on a third party interface and execute the changes. The direct result is a smooth website performance because any possible impact is indiscernible to the user.

7. Privacy

• The configuration of the experiment being implemented is visible both to the end user as well as to 3rd parties.Obfuscation of privacy is possible to some extent in this case.
• Security is strong in this case as details of the test experiment can be limited to the company’s server.

8. Caching

1. Integration with CDN is not required. When content is stored in CDN, experiments can be run on client-side once the cached content in CDN is delivered to the user. In this scenario, backend must serve the response to ensure there is no flicker effect. The group affiliation of the A/B test will be required by the backend for smooth website performance.
2. The changes made as part of server side testing have to be determined at CDN level. This makes the integration of CDN compulsory for running server-side test.

David Isquick explains;

“Client-side: compatible with any CDN provider: companies storing content in a CDN can run experiments client side after cached content is returned to the user.”

“Server-side: treatments must be cached at CDN layer: server-side implementations will only work if treatments are determined. A CDN integration or separate decision service is needed.”

9. Channels

• Client-side testing and experimentation are restricted to a single channel.
• Server-side testing and experimentation can be run on multiple channels at the same time. It is also suitable for running experiments in any channel at different points of time.

10. Process lifecycle

Image Source:hubprocess

• The process lifecycle in client-side testing is shorter.
  • You first create a hypothesis
  • Build the different variations using the AB testing tool
  • Analyze the results and determine the winner.
  • Only the winning variation is then built into the main application code and rolled out for all website users.
  • If the test does not generate any increase in conversions, then all variations are removed from the AB testing software and the original page design is served to visitors.

• Comparatively, the process lifecycle takes a longer route with server-side testing.
  • You first create a hypothesis
  • you implement each of your test variations on the server.
  • Analyze the results and determine the winner.
  • Since all variations are already built on the server, then when a winner is determined, it defaults to all users
  • If the test does not generate any increase in conversions, then all variations are removed from the server the original page design is served to visitors.

When to opt for client-side a/b testing

Image Source: serversidevsclientside

Even though from the differences that we have highlighted above, it might seem to some that server-side a/b testing is a superior technique over client side a/b testing, or the other way around, this isn’t exactly the way to go about it. Deciding between client-side a/b testing and server-side a/b testing is not always about which one is better. It is rather about your very specific experimentation needs and requirements.

Both these techniques call for a different sort of an experimentation roadmap, have unique objectives, and need very dissimilar organizational models.

For example, in client-side a/b testing, if the variation is a loser then there is no server code release required. On the other hand, in server side a/b testing, every experiment might require code release.

Let’s go back to our cart page example and see how the team handled implementing it using the two different A/B testing platforms:

1. Coding the cart page in a client-side platform is quick. It requires front-end developers to perform the implementation.
2. Coding the cart page in server-side platform requires both front and back-end developers to implement the test. The time is considerably longer compared to the option 1.

Let’s take two post-test scenarios and see how you will have to handle them based on your AB testing platform:

1. You run a test and one of your challengers is a winner

For client side AB testing platform, your dev team will have to now implement the winner in the platform. Depending on the complexity of the test, your dev team will be spending considerable time implementing the winner.

For server-side, the winning variation is already coded in the platform. Your dev team will have to remove all other variations and replace the original design with the winner. This is a quicker task for the dev team to handle.

2. You run a test and none of your challengers is a winner

For client side ab testing platform: you remove all the variations from the AB testing platform.  This is a quick task for the dev team to handle.

For server-side: the dev team will have to remove the all variation code from the server.  This is a longer task for the dev team. In this case, the cleanup required on the server end makes it a comparatively arduous task for companies and they need to make sure that there are no traces of the variation left.

Companies choose client-side a/b testing when there is a need for:

• Ease of experimentation: If the objective is to enable the marketing team to create and run experiments using a visual editor, which does not require code releases. This means client-side experiments don’t need your dev team to oversee execution. Also, it takes lesser time to run such experiments. Less work, more speed, and simple execution – if this is what your experimentation needs are then client-side testing is what you should opt for. And, as pointed out by Andrew Anderson in this post, speed is a critical testing requirement for businesses.
• Need for segmentation: More data is available in case of client-side a/b testing, for segmentation purposes. This is because client-side tests are run post a page has been loaded. Because these tests are done post the page loads.
• Minimum SEO impact: If you do not want the testing and experimentation on your website to negatively impact its ranking and searchability on Google, client-side is what you should be opting for. Typically, changes made through JavaScript are not accounted for or not indexed by Google, leaving absolutely no impact on your search engine optimization. As far as the question of human reviewers is concerned, as long as you are not trying to deceive, you won’t face the penalty. It’s only when it appears you are trying to trick Google that reviewers will flag you.

Juan Vega explains; 

“On the bright side, server-side rendering is great for SEO. Your content is present before you get it, so search engines are able to index it and crawl it just fine. Something that is not so with client-side rendering. At least not simply.”

Problem with client-side a/b testing

Now that you are aware when to pick client side a/b testing, you should also understand the inherent drawbacks of choosing this technique. Knowing what are the cons or problems with client-side a/b testing will help you make a sound decision.

• Flicker effect: In simple words, flicker effect is said to happen when for a very short duration of time your visitor sees the default version of the webpage before the test variation loads. Now why it matters or is counted as a drawback, is because it impacts tests results and can skew results. If people detect even a small hindrance to their onsite experience, their behavior changes. That is why the objective is to reduce flicker to an extent that it is not detectable by human eye.
• Impact on loading time: Loading time affects user behavior. Using synchronous code in client-side is known to cause slower load times.
• User restrictions: Even though client-side testing allows for a lot of ease and simplicity, most complex split tests would still require the involvement of front-end developer to implement.
• Browser related problems: There can be compatibility issues when a visitor’s Javascript has not been updated or updates very slow during a variation is being loaded. In the case where a variant being tested is not compatible with the browser, it results in visitor data becoming polluted.

When to opt for server-side a/b testing

Image Source: serverside_abtest

Next, we move to discussing when it is advantageous for companies to opt for server-side a/b testing. It needs to be clearly understood, and hence is being emphasized in this article again and again, that the limitations of one technique should not be the reason for opting the other.

• Mobile app testing: Server-side a/b testing is more suited for mobile apps because mobile sites serve dynamic content. It isn’t easy to run client-side experiments on dynamic content. Moreover, if phones have slow connectivity and weak browsers client-side Javascript based approach will only be a drawback.
• Where you need greater flexibility in experimentation: The depth and scope of server-side experimentation are higher than that of client-side experimentation. You are neither restricted to testing on a single channel nor to a few modifications that you are otherwise limited to while running client side a/b.

Robin Nichols explains;

“Server-side A/B testing is more oriented towards product teams; whose needs involve more business rules and which are tightly linked to product features. The keyword here is FLEXIBILITY.”

• Minimal website performance impact: Of course, everyone would want their website to run absolutely smoothly, without any flicker or hindrance. Since these experiments are not being rendered on the browser, your visitors hardly detect any changes.
• Enables experimentation on Omnichannel campaigns: The world is going Omnichannel and so should your digital marketing efforts. It does not any longer hold any sense to experiment only one element of a web page at a time. You need to run cross-channel and Omnichannel tests simultaneously to measure performance across all devices and on a unified client journey. This is possible only with server-side experimentation.
• Optimizes for long-term goals: Server-side a/b testing allows optimization for macro goals like customer lifetime value. It is possible to exclude certain conversions in this approach and have complete control on what you are measuring. For example, an e-commerce owner might want to exclude checkout values of above $200, which he considers as an unusually large purchase that does not reflect the true nature of purchases on the website. He does not want to take them into account because they can skew the conversion results. While an experienced developer can implement this in client-side ab testing tool, it is not as straightforward if implemented in a server-side tool.

Disadvantages of server-side a/b testing

Server-side a/b testing is not free of cons. There are some problems associated with this approach as well, which are discussed as follows:

• The server-side tool requires developers at the server end: To deploy codes and to establish any experiment, a development team is required. This means more investment in hiring professional developers.
• Longer development time (Slower execution): The client-side tool, due to its simplicity, has a quicker implementation. The server-side on the other hand requires more coding to run from the start of its operation, this makes it slower in terms of execution. With server side, the development team will actually have to code all the variations on the server side. After the test is conducted, and a winner is chosen all other variations that were coded on the server will be removed.

How to run an a/b test – client-side

The following are typical steps followed when setting up and running a client-side a/b test:

1. Insert code snippet on your website
2. Create test version using a visual editor
3. Create your conversion goals- increase in conversion goals, downloading files, clicking on a link etc.
4. Start the test on your choice of a/b testing tool
5. Keep a track of website version performance on the a/b testing tool dashboard

How to run an a/b test – server-side

To run a server side a/b test you need to follow the given steps:

1. Once a user lands on site, a unique visitor ID is generated using the API.
2. Call the API passing in parameters the unique visitor ID when the test has to be run. Alongside also pass in the test ID.
3. Responding to your request, the a/b testing tool sends the variation ID.
4. The server builds its response built on this variation ID.
5. Data servers are informed once a conversion takes place.

Conclusion

A detailed know-how about client side and server side a/b testing equip companies to make sound business decisions. If you too are deploying an a/b test tool for running tests and experiments, do read as much as you can on the uses, advantages, and downsides of both.

Khalid Saleh

Khalid Saleh is CEO and co-founder of Invesp. He is the co-author of Amazon.com bestselling book: "Conversion Optimization: The Art and Science of Converting Visitors into Customers." Khalid is an in-demand speaker who has presented at such industry events as SMX, SES, PubCon, Emetrics, ACCM and DMA, among others.

See Full Bio