Methods for the Analysis of Choices and Evaluations

Components of appeal affecting advantage, disadvantage and share

  1. Sources of appeal impact on advantage and share

  2. Extraneous sources of variation in appeal

  3. The scope of current Options Analysis procedures

  4. Awareness as a factor affecting advantage and choice

  5. Group-wide systematic factors affecting advantage and choice

  6. Idiosyncratic factors affecting advantage and choice

  7. Individual appeal uncertainty factors affecting advantage and choice

  8. More information on appeal components and impacts on change

Sources of appeal impact on advantage and share

The first point to establish is whether any appeal exists. Can an option's appeal be measured? If a respondent does not know an option, there can be no rating. Alternative options are chosen and gain share from an option's absence.

Within a group, individual assessments of options' appeals compose a set of option appeal distributions. Through a series of controlled comparisons of these distributions, the nature of each option's impacts on advantage and share for the group may be analysed.

The three areas of appeal influence on option advantage and share are:

  1. Group-wide systematic factors reflecting:

    1. the option's general appeal level (mean),
    2. option appeal disparity (standard deviation),
    3. appeal leaning (skewness),
    4. appeal coherence (kurtosis), and
    5. systematic inter-option alignment of appeal (correlation).

  2. Idiosyncratic factors unique to the individual.

  3. Individual appeal uncertainty.

This classification reflects the levels, manner, extent and firmness with which individual appeal ratings are shared amongst individuals in ways that determine each option's advantage, disadvantage and share of choice.


Extraneous sources of variation in appeal

Before appeal is applied in the above analysis it usually important to adjust appeal ratings for sources of influence which do not impact on option advantage and share but which may otherwise distort the appeal statistics.

There are two sources of variation in option appeal which have no effect on advantage or choice intention but which do affect the appeal statistics. These are:

  1. Differences between individuals' attraction to a selection set i.e. to the options as a group of options.

  2. Differences between individual rating styles reflecting the "softness" or "hardness" of their ratings across options in a set. (Usually called "leniency error").

While both these sources of extraneous influence create differences between cases, the effects may be assumed to apply uniformly within each individual's ratings. The average effect can be represented by the mean for each individual's ratings. Consequently, differences in the influence of these factors between individuals can be measured in terms of differences between respondent means.

As advantage and choice intentions are only affected by appeal differences within each individual's set of ratings, this variation can be removed. This can be achieved by adjusting each individual's rating mean to equal the grand mean for the sample and adjusting the individual ratings by the same amount. The Options Analysis procedure for this is called "casewise adjustment".

While the presence of these extraneous factors do not affect the global results for choice, share, advantage, or disadvantage, they can misleadingly affect estimates of group-wide systematic factor contributions to share and advantage. For this reason, analysis of group-wide systematic factor contributions to share and advantage usually involves the removal of these extraneous factors.


The scope of current Options Analysis procedures

Current procedures include adjustment and analysis of all these sources of variation in appeal listed above and their impacts on advantage and choice within a group, except for individual uncertainty.


Awareness as a factor affecting advantage and choice

The single most important reason for not being able to obtain an appeal rating for an option is that the respondent does not know the option. This includes the situation where the respondent feels insufficiently familiar to be able to assign an appeal value. Awareness can be a very important factor in achieved option share. Options Analysis is able to provide an integrated analysis of the respective roles of awareness and appeal in respondent decision-making.


Group-wide systematic factors affecting advantage and choice

Much of Options Analysis is concerned with appreciating the nature and scale of the group-wise factor effects on individual advantage, disadvantage and choice. Included here is analysis of the roles of appeal levels, disparity, leaning, coherence and inter-alignment within and across the options.

Such analysis is of great importance as it reveals the character of the forces having the greatest impact on advantage, disadvantage and share within the group. The analysis enables the focusing of attention on those areas of greatest interest.

Analysis of the group-wise factor effects is achieved through systematic manipulation and analysis of the option appeal distributions in the selection set. With standardisation of all or selected parameters it is possible to evaluate their separate and combined contributions.

The analytical process is complicated by entanglement of the effects of appeal distribution parameters on advantage, disadvantage and share. This entanglement relates to both an option's own parameters and the parameters of the remaining options. An important consequence is that without the kind of analysis involved here, any change to an option may have substantially unpredictable effects. This can mean, for example, that a minor change in an option's appeal may result in next to no movement in share of choice, or a very dramatic change, or anything in between. This means that situations providing substantial changes in choice for small changes in appeal can be sought. Likewise, situations yielding small changes in share for much effort can be avoided.

Options Analysis provides a range of methods to understand the basic input data, the parameter composition of each option's current status, and simulation of the effects of changing one or more parameters.


Idiosyncratic factors affecting advantage and choice

These are factors which are unique to the individual. Statistically, idiosyncrasy is that component of appeal which is unaccounted for by the systematic group-wise factors. Its role may be substantial if its accounts for a large proportion of the variation in appeal. This is most likely to arise in heterogeneous selection sets where the options have little in common.

By default, idiosyncratic effects are currently included in the inter-alignment analysis of Options Analysis. However, their separate contribution to the overall advantage, disadvantage and share of choice effects can be estimated using Options Analysis procedures.


Individual appeal uncertainty factors affecting advantage and choice

Appeal uncertainty concerns the range and incidence of possible appeals that an option may deliver to an individual. The range of appeals are not random but are unpredictable at the time of evaluation. For example, weather may be a factor influencing the appeal of an option. So, the option may be said to have a range of appeals depending on the weather.

Instead of one data point for each individual for each option, measurement of uncertainty would involve from three to six plus data items on appeal of each option. Effectively, what is involved is defining the character of each individual's appeal uncertainty distribution for each option plus the correlations between them. As may be readily appreciated, fully elaborating these distributions:

However, there are some important advantages in gathering at least some uncertainty data. Two key areas are:

  1. Assessing the scale of appeal uncertainty associated with each option. For a product or service large uncertainty might indicate unreliability, for example. For a policy proposition, large uncertainty may indicate that the concept is too vague. (Note, however, that large uncertainty is not always seen as a bad thing. It may be an inherent part of the attraction as in recreational fishing and gambling.)

  2. Calculating individual choice probabilities for each option's selection. This can yield more accurate share of choice data as well as indicate the presence and levels of option substitution by individuals.

At the simplest and probably most useful level, sufficiently suitable data for these purposes may consist of each respondent's assessment of the range in which an option's appeal will most likely occur. This needs to be coupled with respondent information on what "most likely" means. Other characteristics of each individual's uncertainty distribution for each option would need to be assumed. This includes each option's skewness and kurtosis plus correlations between an individual's uncertainty distributions for the remaining options. However, the significance of these can be tested using sensitivity analysis.

Options Analysis does not currently offer group analysis of appeal uncertainty data. However, with adaptation of concepts, Options Analysis can be undertaken of a single individual's option evaluations and decision-making.


Notes index


Copyright 2012 Corporate R & D Ltd
All rights reserved. Email: