This article gives an overview of how to use conjoint experiments in SightX surveys.
What's on this page:
What is a Conjoint Experiment?
A conjoint experiment is a statistical analysis technique designed to uncover the value people assign to different aspects of a product. It is especially useful when you’re in the process of developing new products or services, and you’re uncertain about which features matter most or least to consumers. The insights gained from a conjoint experiment provide valuable guidance during the creation of new products and services, ensuring they align with consumer preferences and expectations.
SightX offers a comprehensive simulator based on conjoint results that enables you to
- Divide your respondents into segments with similar preferences.
- Determine the market share of products of interest.
- Analyze how changes in pricing affect the demand for products in a competitive environment.
When to Use a Conjoint Experiment
Use a conjoint experiment any time you want to measure the value that consumers place on various features of a product or service. Conjoint experiments are great for optimizing product development and for product pricing exercises, because the analysis shows how important the attributes are relative to each other, and which levels within the attributes provide consumers with the most benefit or utility.
Setting Up a Conjoint Experiment in SightX
The initial step of conducting your conjoint experiment is to break down your product or service into distinctive features that may influence consumer preferences; these features are called 'attributes'. For each attribute, you then specify its respective variations, referred to as 'levels'.
This is better explained with an example: Imagine we're in the business of crafting sophisticated men's dress shoes, and we're on a mission to create new styles that truly resonate with our customers. To achieve this, we’re going to use a conjoint experiment to understand exactly what features our customers are looking for in a shoe. From experience we think that a shoe can be described by its style, material, color, and price. These will be our attributes. Based on what we can create at our factory, we define the possible levels of each attribute as follows:
| Style | Material | Color | Price |
| Oxford | Leather | Black | $160 |
| Monk strap | Faux leather | Brown | $170 |
| Derby | Canvas | Chestnut | $180 |
In SightX you can find Conjoint under the methodologies section when adding a new question to your project
Once you have added a conjoint experiment you will see the following:
In the first section of the question you need to define:
- Title: this will be used when referencing the experiment on the analysis dashboards. It will not be shown to respondents.
- Experiment description: this is a description that will be shown to respondents before the experiment commences. You can also add an image if necessary.
For example: "In this experiment, you will be shown different combinations of products. For each set, you will be asked to select the product that you would be most likely to purchase." - Choice sets question: this is the question that will prompt users to select their preferred options. This question will be shown for each each set of your experiment.
For example: "From the products displayed bellow, what is the product you would be most likely to purchase?"
Attributes and levels
In the Attributes section you can create your attributes (you can use up to 15) and define the levels (variations) of each attribute. To add and attribute click on the "+ Add attribute" button:
Once you added an attribute, you can define its name in the Attribute name input box. Next you can define its levels, your levels can be
- Categorical (e.g. colors, brands, shapes).
- Numeric (e.g. weight, volume, speed).
- Prices.
You specify their type by selecting any of the three buttons indicated bellow:
For categorical levels, you can add images by clicking on the add image button on the left side of the level label:
For numeric levels, you can specify their unit of measure in the Unit label input box (e.g. kg):
For price levels, you need to specify the currency of interest (e.g. USD):
In some cases we might know that some levels are preferred over others. For example, if we produce shoes we might know from experience that leather is usually preferred over faux leather, and that faux leather is preferred over canvas; we also know that lower prices are preferred over higher prices. For these cases, you can specify the order of the levels by activating the Ordered toggle:
You will see a guide on the right side indicating the order in which you should define your levels:
By ordering your levels our platform can create a better experiment, but this is true only if you are certain some levels are preferred over others.
Conditional attributes
In some cases you might have an attribute whose levels depend on another attribute, we call this a conditional attribute. For example, if you sell shoes you might want to price them according to the material they use; we can define low, medium, and high levels for each material:
Price |
Material |
||
|
Leather |
Faux Leather |
Canvas |
|
|
Low |
$170 |
$160 |
$150 |
|
Medium |
$180 |
$170 |
$160 |
|
High |
$190 |
$180 |
$170 |
With this configuration, respondents will see leather shoes with prices ranging from $170 to $190, faux leather from $160 to $180, and canvas shoes from $150 to $170. You can add a conditional attribute by clicking on the + Add conditional attribute button:
Once you add it you will see the following:
Here you need to define the type of levels you want to define, and under Independent attribute select the attribute that will determine the levels of your conditional attribute (e.g. material)
Optional advanced configurations
In some cases, you may have limitations on the products that are possible for you to create or sell. To take this into account, SightX allows you to:
- Add a None of these option
- Define combinations of levels that should not appear in the experiment
- Define products that should not be shown.
- Define products that must be shown.
Note that these constraints should be used sparingly, as they will affect the results of your experiment.
You can configure these constraints in the Optional advanced configurations section when building your experiment.
The “None of these” option
To give more flexibility to your respondents, you can allow them to not choose any of the products in a set. You can do this by adding an option to each set for I would not choose any of these products. Including this option could improve the results of your experiment by not forcing users to choose options that they would not choose in a real-life scenario. However, you may see that some respondents overuse the none option, so think carefully about whether this option would improve your study.
In SightX you can add the “none” option by clicking on the Add "none of these" alternative toggle:
Pairs to exclude
There could be cases where a level in one attribute can’t be paired with a particular level in another attribute due to certain circumstances. In our shoe example, suppose that we cannot produce leather shoes that are sold for less than $170; as our Price attribute has the $160 level, we can specify in the Pairs to exclude section that we don’t want to see Leather paired with $160; to do this, click on + Add exclusion, select the attributes under Combinations to exclude, and indicate the excluded pair in the table that appears:
This will prevent respondents seeing leather shoes that cost $160. We suggest using this feature sparingly and only when absolutely necessary as it will affect the results of your experiment.
Include and exclude products
There might be cases where you want to make sure some products are shown, and some are not. For example, we might want users to see our flagship shoe:
|
Derby |
Leather |
Chestnut |
$180 |
While avoiding them to see a shoe that we cannot produce:
|
Oxford |
Canvas |
Black |
$160 |
These products can be defined at the Products to exclude and Products to include sections:
We suggest using these features sparingly and only when absolutely necessary as it will affect the results of your experiment.
Create the products to show to respondents
When you’ve finished setting up the experiment, click the Generate products button, and SightX will design your balanced conjoint experiment.
After that you will see a window asking you to confirm as this process can take several minutes to complete, so you can navigate elsewhere and SightX will notify you when the experiment is ready.
On the backend, the SightX platform takes the attributes and levels you defined to create individual products, to test variations between those products against each other. Let’s return to our shoe example: We have four attributes (style, material, color, and price) and three levels for each, making 81 possible “products” - that is far too many options to present to our respondents, as they will get overwhelmed and fatigued when evaluating that many options.
Thus, SightX select a subset of these “products” which ensures that each level within an attribute and each combination of levels between two distinct levels are shown an equal number of times, while keeping the number of options shown to a minimum.
Once the products have been constructed, SightX needs to organize them into sets of several products that respondents will see at a single time. Out of each set they are presented, the respondent will compare the products within the set and choose their most preferred product. Constructing the balanced product sets is no easy task, but SightX has you covered; if you want to learn more about set construction, please refer to our technical document.
After we have constructed your products, you will asked to define how many products should be shown in each set:
Once your sets are ready you will see this notification:
And now you are ready to launch your survey and get responses!
What will respondents see?
Before your respondents see the product sets, they will see and introductory page explaining the experiment (as defined in the experiment description section) .
After they see the introduction page, respondents will be presented the product sets as a set of cards, each card representing a product. At the top of each set they will see a sentence prompting them to select their preferred product (as defined in the choice sets question section):
Sample Size
It's important to include a sufficient number of respondents in your conjoint experiment.
In general, the calculation for the appropriate number of respondents for a conjoint experiment is:
Where:
q is the number of sets shown to each respondent
a is the number of attributes per set
c is the maximum number of levels of any attribute
For example, if there are six sets, three attributes, and the greatest number of levels in any of the attributes is ten, then:
556 would be the minimum number of respondents for this study.
You can use our calculator bellow to get your minimum sample size:
What are the outputs of a conjoint experiment in SightX?
In the question analysis dashboard, you will get two graphs summarizing the results of your experiment. The first graph, Level performance, will show you how each level was valued on average by the respondents. The higher their value, the more the level is preferred.
The second graph, Attribute importance, shows how important the attributes are to respondents. The higher the percentage, the more important an attribute is to users when making a decision.
Before you see your results you will be asked to calculate the conjoint experiment part worths. For each respondent of your experiment, SightX assigns a numeric value to each level of your product; these values are assigned based on their responses and are used to measure how attractive a level is. These values are called part worths, calculating them is a complex process and can take several minutes; as such, we recommend starting this calculation once you have collected all your required answers. You can learn more about this process in the conjoint technical document.
In order to calculate your part worths, you will need to pause or close all your live campaigns as SightX uses all your responses to perform this calculation. Once this calculation is done SightX will notify you, and now you are ready to view your experiment results and use our simulator!
Conjoint simulator
The charts presented in the question analysis section of the SightX platform give a general perspective on how consumers value the different aspects of your product. To get a deeper understanding on their preferences, use our conjoint simulator to:
- Divide your respondents into segments with similar preferences.
- Determine the market share of products of interest.
- Analyze how changes in price affect the demand for products in a competitive environment.
To access the conjoint simulator click on the Go to simulator button on the question analysis charts
You can also go to the Simulators tab on the left side menu and click on Conjoint:
Segmentation tool
When running a conjoint experiment, your results will typically have clusters of respondents with similar preferences. For example, if we run a study about shoes, we could have a segment of people who like Oxford shoes, another segment where people prefer Derby shoes, and a segment where people are indifferent to shoe type and instead value price more. Let's say we are interested in launching a new Oxford shoe, so our target customer will be the people who like Oxford shoes, and potentially the people who are indifferent to shoe type. From there, we would investigate the characteristics of our target segments by looking at their responses to demographic and behavioral questions in the survey. To produce the segmentation, we use latent class multinomial logit. You can get a deeper understanding of this method by reading our technical documentation.
Our segmentation tool allows you to divide your surveyed population into groups based on their responses to the conjoint experiment. In the conjoint simulator page you will find a default segmentation SightX calculated using all of your your responses:
You can access this segmentation by clicking on it. Inside a segmentation you will see the following:
Under Number of segments you can select the number of groups in which you want to divide your population, by default SightX divides your population from 1 to 5 groups, and shows you the optimal way in which to divide your population:
For the selected number of segments you will see cards summarizing each group with the following information:
- The percentage of survey respondents in the segment.
- The number of survey respondents in the segment.
- The preferred levels in the segment.
By clicking on the magnifying glass button next to the segment name, you can see how people of each segment responded to other questions in your survey, allowing you to get a deeper understanding of who makes up each group. Outside of the conjoint simulator, you can use these segments to filter your analysis if you want to dig into a segment further.
Bellow these cards you will find charts indicating:
- The segment’s average valuation of each level.
- The importance of each attribute to members of the segment.
You can select the segment of interest on the dropdown menu located on the right side above these charts:
By selecting All segments, SightX will show you charts comparing your segments:
Creating a new segmentation
You can also calculate a segmentation within a subpopulation of interest. For example, you might be interest in finding groups of people within a specific city. To create a new segmentation, go to the conjoint simulator page and click on the Create new segmentation button:
You will be taken to the page shown bellow, here you need to define the name of your segmentation and define the filters that will specify your subpopulation of interest:
Once you have defined all these configurations click on the Calculate button. Calculating a new segmentation takes several minutes, and once it is ready SightX will notify you:
Market share tool
The average level performance and average attribute importance can give us important insights on how consumers value the different aspects of our product or service, but in doing so we lose information about individual preferences. Our market share tool uses individual preferences to determine what products consumers will choose when a set of products is presented to them.
To start a demand simulation go to the conjoint simulator page and click on the Start new simulation button:
You will be taken to the page shown bellow, here you need to define your products of interest. The simulator will calculate the demand of each product with respect to each other.
Continuing with our shoe example, let's say we would like to get an idea of how our shoe will perform against the shoes of our competitors. First, we need to use the levels in our experiment to define our specific proposed product, and the products that our competitors offer.
Once we have defined all of our products of interest, we click on Calculate demand. Now, we can see the market share distribution of the products we have defined:
Our simulator tells us that 19% of respondents would prefer our shoe (Derby shoe), while 81% would prefer a product from one of our competitors. From here, we may want to take a step back and analyze why our shoe is not very competitive in the market. To do that, we can change some levels of our product in the simulator and recalculate the market share until we find a shoe offering that performs better against our competitors.
Demand curves tool
The demand curve simulator allows you to study how the demand for a product, competing with other products, is affected by changes in price. To view the demand curves you experiment needs to include a Price attribute. Inside your simulation click on the Demand curves tab.
In our simulation, you can choose if you want to view the demand curves of just one product or all of the products. If you have more than one price attribute, you can select which attribute should be used. You will see a curve(s) indicating how the demand of the product(s) of interest changes as the price of the product increases:
The demand curve of a product is calculated by keeping the price of the other products constant, the price is set to the price defined in the product definition.
Demand curves allow you to estimate the price elasticity of demand (PED) of your products over a certain price range. The PED tells you how sensitive the demand for your product is with respect to changes in price over the specified range. If PED is less than 1 we say that the product is inelastic, i.e., changing the price of the product will have a small effect on the demand. On the other hand, if the PED is equal or greater than 1, we say that the product is elastic; that is changes in price will have a significant impact on the demand for the product.
You can also use the demand simulator to obtain the price that will maximize your revenue, and what that revenue will be:
First, specify the range of prices that it would be possible for you to offer to calculate elasticity. You can also specify the number of expected customers so that we can calculate the maximum estimated revenue of your products based on demand.