Selection bias in marketing

Almost every decision we make tends to be in­flu­enced by sub­con­scious biases. The technical term for this psy­cho­lo­gic­al phe­nomen­on is “selection bias,” also known as sampling bias. The cognitive bias describes how in­ac­cur­ate results are caused by selection errors (e.g. in the selection of sample units). Selection bias affects market research and should, therefore, be taken into account by brands and companies.

Selection bias sig­ni­fic­antly affects the selection of in­form­a­tion – and not just in market research or sci­entif­ic dis­cip­lines but also in everyday situ­ations. Only those who are aware of their own in­fal­lib­il­ity can benefit from the use of biases (also referred to as dis­tor­tions) and make use of their effects. Being aware of such errors in judgement can yield sig­ni­fic­ant benefits for companies. By con­sciously trying to coun­ter­act selection biases, busi­nesses can obtain more mean­ing­ful results from their consumer studies.

Selection bias is a stat­ist­ic­al bias in the selection of sample units. This bias needs to be avoided to obtain mean­ing­ful data and results. For example, in marketing, selection bias jeop­ard­ises the ob­jectiv­ity of customer surveys and other market research methods. There are numerous reasons why selection bias may occur, but they often involve either the par­ti­cipants or the in­di­vidu­als selecting the sample units. For example, one such bias involves people’s will­ing­ness to par­ti­cip­ate in research, provided that this is not randomly conducted. Some well-known selection biases include “par­ti­cip­a­tion bias,” “self-selection bias,”, and “sur­viv­or­ship bias.” An example of sur­viv­or­ship bias is when an analysis of successes or failures un­in­ten­tion­ally only includes the data from the successes (i.e. the “survivors”) in the results.

The pre­cau­tions and coun­ter­meas­ures required to best minimise or com­pletely avoid selection bias are re­l­at­ively complex. Stat­ist­ic­al tech­niques can be useful here, such as the Heckman cor­rec­tion, to obtain accurate results in empirical social research or market research.

Selection bias is not just important in research. Busi­nesses and people in their private lives also tend to select in­form­a­tion in a biased manner or are presented with biased data due to selection. Cognitive biases con­trib­ute to us making mistakes in the selection process which then in­ev­it­ably skews the results.

Such con­stantly recurring selection biases clearly show that we are not impartial and that im­par­ti­al­ity requires a lot of effort. The following examples of selection bias il­lus­trate the far-reaching con­sequences of sampling bias.

In the first example, a survey on general brand awareness for a health dietary sup­ple­ment is to be conducted. If the survey is conducted in a gym, health food store, or organic su­per­mar­ket, the products’ target audiences are being surveyed. This can be useful. However, the results of such market research must be in­ter­preted with care because selection bias has already occurred. People who frequent gyms, health food stores, and organic su­per­mar­kets are generally more open to the ef­fect­ive­ness and benefits of health products. Therefore, it can be assumed that brand awareness is higher among this group of people because they were not surveyed in a neutral en­vir­on­ment.

The second example of selection bias il­lus­trates the far-reaching con­sequences of not making a truly random selection. When studying the economy, economic re­search­ers should use a sample that is rep­res­ent­at­ive of all the companies in a country or region under study. If data selection is based on a register of limited and trading companies, for example, the final sample will exclude small busi­nesses and suc­cess­ful freel­an­cers (e.g. lawyers, doctors, ar­chi­tects), artists, and part-time workers.

It is an obvious example and ex­per­i­enced re­search­ers are unlikely to make such a mistake. However, numerous smaller sampling biases can add up and quickly skew a country’s economic forecast.

Selection bias is mainly a challenge in market research, and less so in active marketing campaigns. But brands should consider potential selection biases when assessing the success of an ad­vert­ising campaign if they want to be able to show accurate marketing results, since campaigns are also a form of market research.

In customer and user surveys, one type of selection bias called self-selection bias plays an important role. This bias occurs where par­ti­cipants are given the choice to par­ti­cip­ate in a survey. If the non-par­ti­cipants and the par­ti­cipants have a sig­ni­fic­antly different opinion (e.g. dis­sat­is­fied versus satisfied with the company, re­spect­ively), the selection bias can lead to customer sat­is­fac­tion being over­es­tim­ated. The problem of selection bias can and must be minimised here using stat­ist­ic­al weighting tech­niques.

Selection bias is tricky in marketing because the sampling bias often occurs in con­junc­tion with other kinds of biases. To be able to take the necessary stat­ist­ic­al coun­ter­meas­ures, it is necessary to un­der­stand the com­bin­a­tion of biases at work. These may include pub­lic­a­tion bias (only positive results are published) or the pre­vi­ously mentioned self-selection bias (only certain groups par­ti­cip­ate).

The smaller the amount of available data and/or the smaller the sample taken from it, the more likely it is that selection bias will make data-driven marketing im­possible. Any un­detec­ted errors can skew studies and surveys and lead to dis­astrous decision-making errors for marketers. A properly designed study should always ensure that sub­con­scious effects, un­in­ten­tion­al biases, and potential tampering are avoided.

Please note the legal dis­claim­er relating to this article.