An Activity and Metric Model for Online Controlled Experiments

DSpace Repository

An Activity and Metric Model for Online Controlled Experiments

Details

Files for download
Icon
Accepted manuscript
Overview of item record
Publication Paper in proceeding
Title An Activity and Metric Model for Online Controlled Experiments
Author Mattos Issa, David ; Fabijan, Aleksander ; Dimitriev, Pavel ; Bosch, Jan ; Olsson Holmström, Helena
Editor Kuhrmann, Marco ; Schneider, Kurt ; Pfahl, Dietmar ; Amasaki, Sousuke ; Ciolkowski, Marcus ; Hebig, Regina ; Tell, Paolo ; Klünder, Jil ; Küpper, Steffen
Date 2018
English abstract
Accurate prioritization of efforts in product and services development is critical to the success of every company. Online controlled experiments, also known as A/B tests, enable software companies to establish causal relationships between changes in their systems and the movements in the metrics. By experimenting, product development can be directed towards identifying and delivering value. Previous research stresses the need for data-driven development and experimentation. However, the level of granularity in which existing models explain the experimentation process is neither sufficient, in terms of details, nor scalable, in terms of how to increase number and run different types of experiments, in an online setting. Based on a case study of multiple products running online controlled experiments at Microsoft, we provide an experimentation framework composed of two detailed experimentation models focused on two main aspects; the experimentation activities and the experimentation metrics. This work intends to provide guidelines to companies and practitioners on how to set and organize experimentation activities for running trustworthy online controlled experiments.
Conference
International Conference of Product-focused Software Process Improvement (November 28-30 : Wolfsburg, Germany)
DOI https://doi.org/10.1007/978-3-030-03673-7_14 (link to publisher's fulltext.)
Publisher Springer
Host/Issue PROFES 2018 : Product-Focused Software Process Improvement;
Series/Issue Lecture Notes in Computer Science;11271
Programming and Software Engineering;11271
ISSN 0302-9743
ISBN 978-3-030-03672-0
978-3-030-03673-7
Page 182-198
Language eng (iso)
Subject Data-driven development
A/B tests
Online controlled experiments
Technology
Research Subject Categories::TECHNOLOGY
Handle http://hdl.handle.net/2043/28009 Permalink to this page
Link https://profes2018.wordpress.com/... (external link to related web page)
Facebook

This item appears in the following Collection(s)

Details

Search


Browse

My Account

Statistics