by Anaïs Fernández (E-Seniors)

As new cyber threats emerge and TRUST aWARE’s partners seek to find solutions, what role can pilot studies play in helping develop these solutions? What type of research, quantitive or qualitative, is best suited to assisting with these pilot studies? Can representative control groups conducting small-scale tests help lead the way in securing the privacy of modern internet users?

The definition of a pilot study refers to a “small-scale test of the methods and procedures to be used on a larger scale” in order to assess the feasibility or the acceptability of an approach, a particular research instrument or a specific study design [1]. The benefit of conducting these types of experiments is that it allows understanding where the project research could fail or whether the methods or procedures are inappropriate or too complicated in order to adjust them on a later stage [2]. Another essential point is that pilot studies can be conducted and analysed through quantitative and/or qualitative methods.   

Partners form the project TRUST aWARE worked on the development of a solution that will ensure privacy and security to the users while they use their digital devices and browse online. Thus, technical partners from Trilateral developed the dashboard TRUST aWARE during the last months. This platform provides a set of novel and integrated tools and services, co-created by citizens and stakeholders specialised in privacy and security, to identify, audit, analyse, prevent, and mitigate the impact of the various threat related to privacy and security that are associated with your digital activities.

The methodology used during the pilot sessions for testing the dashboard in the framework of the TRUST aWARE project was divided in three parts: preparation, execution and validation. During the first part, the pilot representatives elaborated the materials needed in order to conduct the sessions (dissemination materials to recruit the participants, consent forms, qualitative and quantitative questionnaires, tables to be filled during the sessions etc.). They also ensured a demographic sample of the participants, as balanced as possible between minors, general public, seniors and other relevant stakeholders and professionals. During the second phase dedicated to the execution of the pilot study, the pilot organisations followed the same frame in order to test the main components of the TRUST aWARE dashboard. Pilot organisations organised both event and workshop sessions in order to reach the target groups. Finally, the validation phase consisted of evaluating the dashboard.  On the one hand, a quantitative evaluation was executed, horizontally homogenised for the four pilot countries in Spain, Italy, Romania and France with a questionnaire proposed online or in paper version. On the other hand, for the qualitative assessment, data were collected directly from the observations carried out during the pilot sessions.  

From January 2023, all pilot partners started the tests of the TRUST aWARE dashboard in their respective countries in Spain, Italy, Romania and France. In Spain, the participants came from an association that deals with people that have cognitive disabilities. One representative from CiberVoluntarios (CIB) collected data while a moderator presented the dashboard. In France, E-Seniors (ESE) followed the same format with two staff members conducting the sessions with both seniors and general public. In Italy, Fondazione Mondo Digitale (FMD) has been piloting the solution engaging seniors and students previously involved in the TRUST aWARE training sessions on privacy and cookies policy. The representatives of local associations and elders’ centres supported FMD, as well as school teachers of the students volunteering. Participants were motivated to contribute and provided valuable feedback, understanding the importance to participate in the TRUST AWARE co-creation process. Participants’ opinions were gathered by the end-user’s organisations and the main conclusions will be communicated to the technical partners in order to improve the solution. Those pilot studies will be useful to gain insight in accessibility and they will show the progress and improvements needed for the development of the dashboard. Those experiments corresponded to a three-months cycle that will end by March 31^st 2023.

References

[1] Porta M., A dictionary of epidemiology (5^th edition), ed. Oxford University Press, 2008

[2] Van Teijlingen E., Hundley V., Social Research Update:The importance of pilot studies, ed. Department of Sociology, University of Surrey, 2001