A Comparison of Full and Partial Choice Set Designs in a Labelled Discrete Choice Experiment

Background: One approach to reducing complexity of large labelled choice tasks is to use a partial choice set design (PCSD) in which a subset of alternatives is shown in each choice task in contrast to a traditional full choice set design (FCSD) where all alternatives are shown. Using a within-respondent comparison, we aimed to (1) test the convergent validity of the PCSD and FCSD; (2) verify if the PCSD reduces cognitive burden; (3) explore respondents’ preferences between the FCSD and PCSD. Methods: Labelled utility functions were rewritten into a single generic utility function using a label dummy variable and indicator functions, which was used to create a PCSD with 3 alternatives in each task (out of 6). The convergent validity of two designs’ results from conditional logit and mixed logit models were tested using the Swait and Louviere test and the convolution test, respectively. The PSCD’s impact on choice variances was examined using a heteroscedastic conditional logit model. Results: Using data from 790 respondents, we found preference estimates from the FSCD and PSCD are statistically different up to scale. These results remain different even accounting for more flexible substitution patterns across alternatives using the MIXL model. We found that the PCSD appeared to induce smaller choice variance than the FCSD, which reflects positively on its purpose of reducing the cognitive burden. The PSCD was preferred by female and when phones were used to answer the survey. Conclusions: Our findings indicate that the PCSD can reduce cognitive burden and we suggest its use for surveys accessible by mobile phone. While both a PCSD and FCSD should capture the same behaviour, our study reveals statistically significant differences, perhaps because respondents in a FCSD were not trading off on all attributes and alternatives due to choice task complexity, but without more research on external validity it is not possible to conclude which design type better uncovers true preferences.