Call for Empirical Track

In the REFSQ community it is commonly agreed now that empirical validation of RE methods and tools is a necessity to prove their effectiveness, robustness and appropriateness. However, the RE literature to date, including that of the REFSQ series, is not presenting much of such case studies and experiments.  This lack is assumed to be at least partly due to the difficulties of

• bringing academics and practitioners together to pursue empirical studies and
• finding and convincing the participation of a sufficient number of suitable subjects for experiments.

Therefore, REFSQ 2014 will offer practitioners, researchers and educators the opportunity to conduct a small number of empirical studies during REFSQ 2014 involving all the conference attendees. The goals of this opportunity, besides those of the studies themselves, are to raise awareness for the necessity and benefits of empirical studies in the REFSQ community, and to show that participating in them should not be considered harmful.

The call is also available as a pdf.

The Empirical Track calls for two types of empirical studies proposals

• Alive studies: a controlled experiment, requiring no more than 90 minutes in total. It will be run twice during the industry day, in parallel with industry sessions, so that a significant number of attendees are expected to participate one session or another. Only one experiment will be selected. A draft analysis of the experiment will be required to be presented the last day of the conference.
• Online questionnaires: an online questionnaire (survey), designed to require no more than 15 minutes, that is promoted at REFSQ and that can be filled out by all interested REFSQ participants, in their spare time at the conference. We expect to select 3-4 questionnaires. A written report with the data gathered from the questionnaire will be required to be sent for dissemination one month after the conference.

The protocol of accepted proposals (6 pages long in LNCS format) will be published in the REFSQ 2014 workshop CEUR proceedings.

Both types of studies should be designed to be performed during the conference itself. Please consider in your proposal that REFSQ participants will have highly heterogeneous backgrounds and profile. It is therefore highly important that your experiment or questionnaire allows REFSQ participants from industry and academia to be subjects.

Submission and evaluation of proposals for alive studies and online questionnaires

Each proposal is limited to 4 pages in LNCS format containing the following information for the proposed study:

• the title of the proposal,
• the type of study (experiment or online questionnaire),
• goals, including the hypothesis,
• a detailed description of the intended procedure: preparation, conduction & analysis, including timing details
• the benefits of results for the community,
• the benefits to the subjects of participating in the study, including incentives,
• a description or profile of the intended subjects: prerequisites for being a subject, such as experience, background, etc.,
• a short discussion of the threats to validity that might originate from the constraints of performing the study at REFSQ,
• the name, organization, and contact details of proposal submitters,
• a record of past empirical studies performed so far by the submitters,
• role of the proposed study in a generic research plan (e.g., it is expected to confirm some theory, to explore some emergent research topic, …)
• how you plan to make publicity of your study if it is selected, and how do you plan to attract respondents to your study during the conference
• equipment and infrastructure needed for performing the experiment: PCs, software, flipcharts, Internet, etc.

A link to a complete description of the protocol, although not mandatory, is highly recommended.

We encourage proposals from both academia and industry!

Submit to https://www.easychair.org/conferences/?conf=etrefsq2014.

Each proposal will be evaluated by the program committee based on the following criteria:

• the benefits to the subjects for participating in the study, including what you are doing to motivate participation,
• the likelihood of persuading participation of enough subjects given the study’s prerequisites, publicity plans and benefits to participants,
• the benefits of the expected results to the RE community,
• the quality of the proposal’s description of the study procedure,
• the experience of the submitters in conducting empirical studies
• the plans on the continuation of the study after REFSQ’14.

The results of the study will be made available by the end of May, 2014 on the REFSQ 2014 homepage. A description of the results might be published in the workshop proceedings of next REFSQ conference, REFSQ 2015.

Important Dates

Empirical Track Co-Chairs

Xavier Franch, Universitat Politècnica de Catalunya, Spain
Jolita Ralyté, Université de Genève, Switzerland

Programme Committee

Dan Berry, U. Waterloo, Canada
Travis Breaux, CMU, USA
David Callele, Experience First Design Inc., Canada
Maya Daneva, U. of Twente, The Netherlands
Oscar Dieste, U. Politécnica Madrid, Spain
Sergio España, U. Politécnica Valencia, Spain
Matthias Galster, U. Canterbury, New Zealand
Andrea Hermann, Independent Researcher, Germany
Marjo Kauppinen, U. Aalto, Finland
Barbara Paech, U. Heidelberg, Germany
Anna Perini, FBK, Italy
Carme Quer, U. Politècnica Catalunya, Spain
Bjorn Regnell, U. Lund, Sweden
Krzysztof Wnuk, U. Lund, Sweden