The Benefits of Controlled Experimentation at Scale

DSpace Repository

The Benefits of Controlled Experimentation at Scale

Details

Files for download
Icon
Overview of item record
Publication Paper in proceeding
Title The Benefits of Controlled Experimentation at Scale
Author Fabijan, Aleksander ; Dmitriev, Pavel ; Holmström Olsson, Helena ; Bosh, Jan
Date 2017
English abstract
Online controlled experiments (for example A/B tests) are increasingly being performed to guide product development and accelerate innovation in online software product companies. The benefits of controlled experiments have been shown in many cases with incremental product improvement as the objective. In this paper, we demonstrate that the value of controlled experimentation at scale extends beyond this recognized scenario. Based on an exhaustive and collaborative case study in a large software-intensive company with highly developed experimentation culture, we inductively derive the benefits of controlled experimentation. The contribution of our paper is twofold. First, we present a comprehensive list of benefits and illustrate our findings with five case examples of controlled experiments conducted at Microsoft. Second, we provide guidance on how to achieve each of the benefits. With our work, we aim to provide practitioners in the online domain with knowledge on how to use controlled experimentation to maximize the benefits on the portfolio, product and team level.
Conference
Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (2017 : Vienna, Austria)
DOI https://doi.org/10.1109/SEAA.2017.47 (link to publisher's fulltext.)
Publisher IEEE
Host/Issue 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA);
ISBN 978-1-5386-2141-7
Pages 18-26
Language eng (iso)
Subject controlled experimentation
leading metrics
lagging metrics
data-driven development
Technology
Research Subject Categories::TECHNOLOGY
Handle http://hdl.handle.net/2043/24150 Permalink to this page
Link http://dsd-seaa2017.ocg.at/seaa2017.html... (external link to related web page)
Facebook

This item appears in the following Collection(s)

Details

Search


Browse

My Account

Statistics