User Involvement in Testing
The activities related to User Involvement are described in documents prepared mainly in SP4. SP4 intends to assure the proper validation or repudiation of the develop software by those more interested in their usage. It is essential to have end-users involved in the evaluation of the user pilots, especially to test the concepts to be developed. To achieve that a user involvement plan is necessary. This plan have to include user involvement methodology, with a wide range of appropriate techniques, and also information on how to interest stakeholders and prospective users in the project and the products been developed.
The information regarding user involvement is presented in D401.1 Cloud4All User Population and UCD Implementation Plan
Due to the fact that one of the main objectives of evaluation is to end up with stakeholder engagement, it is important to set up the main strategies for this. Prosperity4all evaluation goes from technical validation, accessibility, usability and finally user take-up. It is important to define the main liaisons with the different stakeholders, user-organisations, implementers, etc. The final success of the P4all infrastructure will depend on the level of engagement of actual take up by users. According to the initial hypothesis, there is a need for the proposed set of solutions; nonetheless the challenge is not only to build those but to accomplish user engagement.
Special look at tails
Current solutions tend to focus large stakes of populations. Looking at the dow there is an objective to address those unmet needs. Disabilities such as Dyspraxia, Dyslexia, Dyscalculia, Blind-Deafness, etc.
Variety of Countries
Due to the international scope of the project, it is easy to carry out evaluation in different countries in order to see differences and similarities. Tests will be done in Spain, Germany, Austria and Greece.
Variety of Cultures/subcultures
Since social class is many times a determinant of technology use, disability and life opportunities, an even distribution among the most common class segmentation will be done. The way in which social class will be segmented will follow the latest division done by a large BBC survey . This survey ended up with the conclusion that there were 7 social classes : Elite : Highest level of financial, cultural and social capital Established middle class: This class scores high in all three capitals. This is made up of reputed professionals having worked for many years and having realised satisfactory investments. Technical middle class: This is a small class which is properoous financialy but low in social and cultural capital. New affluent workers: The financial capital is not high whereas the cultural and social it is. Traditional working class : This class scores low on all forms of capital, nonetheless due to the lifeling savings (old group) they have valuable houses, some savings and job security. Emergent service workers: This is a new class found in large cities. Their level of education is high, social and cultural capital remains considerable, though they are very dependent financially Precariat, or precarious proletariat: This is the poorest, most deprived class. Their forms of insecurity makes them poor financially and also withouth tights or cultural interests. The main variables looked at this study are: Income, house value, savings, cultural and leisure activities and the occupations of friends 1.7 Accessiblity literacy This concept concerns only to developers & implementers. A distinction is intended in order to draw conclusions from both groups isolated. In order to make a difference among both groups a set of questions will be defined. 1.8 Technology literacy They will select participants with the help of these organisations, ensuring that they cover the age ranges and technical abilities that are representative of the wide targeted population. Older participants will be also included in the user sample through local organisations that will provide participants, given that these users often have additional requirements and preferences with regards to technology.
Tight to the dissemination material produced by the consortium an engagement strategy must be in place for the different stages of the evaluation. Relating organisations with P4all
3 Towards Natural environments In spite of the inevitability of using lab related environments to test some components, the objective when possible, is to walk toward evaluating our tools in environments that are already in place (Guadalinfo centers, workcenters, etc) Natura environments Vs laboratories Steps Recruitment toolkit Differnet formats Messages Contacting facilitators Contacting users
1.1.2. Recruitment The active recruitment of users for the activities that involve them directly (i.e. pilot tests, online user forum, demonstrators) has to be supported by other dissemination activities included in the project. Therefore, the attendance to workshops, conferences and other forums, as well as the social networks, can also be invited to participate in the pilot activities. In this way, new end-users, people with disabilities, experts and developers interested in the aim of the project can participate in the evaluation phases by entering in the pool of candidates to participate in the sites. Following, the guidelines related to the criteria to be followed in the participants’ recruitment for internal use of Consortium will be described: • All members of the Consortium sent to the partner in charge of the work packages 401 and 501 information about their relationship with partners and stakeholders that will help us all for the recruitment of participants in the iterative process, conforming so the pool of participants, both with disabled people and developers. • Regarding the relationship between the partners and the users’ organizations for the provision of participants, they can sign a Memorandum of Understanding (MoU) if the partners or the users’ organizations require it. The MoU will specify the collaboration between them as a contract. This point is optional, but available for who requires it. • Those partners involved in the recruitment will have to fill in a template with the basic information of their users involved in the evaluation. Of course all that information will be anonymous. That template will allow an exhaustive control of the fixed quotas and the justification of possible deviations. The report with the evaluation results of each pilot phase will include a chapter related to the recruitment, where the criteria and the fulfillment of them will be detailed, so that a representative sample is guaranteed and thus, representative results of the evaluation are got. • According to the ethical guidelines defined in WP 104, when recruiting, participants cannot be coerced into taking part in the study (financial incentives can be seen as coercion due to peer pressure, social perception, economic circumstances, etc., as explained in the Ethical Guidelines) and particular care needs to be taken when recruiting from vulnerable groups. Researchers must explicitly state that participants: • do not have to take part in the study • do not have to give a reason for declining • declining to participate will not affect or influence anything • can withdraw from the research even after giving their consent. • It should be special consideration in the recruitment for people with disabilities: • For people with intellectual disabilities it is often necessary to have prior contact with the tutor or a professional close to the person, to guide us on their possible participation. • For people with cognitive impairments, the messages and consent forms must be simple. • With people with hearing impairments, may be preferable to contact with them via email or SMS. • In general for users with disabilities, it is important to ask them for their specific support needs and to offer them a suitable transport if it is necessary when they have to travel to participate in an event. 1.1.3. Guidelines for managing end-users participation In each activity that requires the participation of users is necessary ensuring that it follow the procedure in the management of users’ involvement that it is explain below: 1. First contact with end-users. The contact with people who are potential participants in the project can be done directly or through organizations that they are members. In both cases, the person responsible for managing users must inform (either the organization responsible for or the user directly) of the activity objectives for which they will participate, as well as the meeting dates and the conditions of participation (length of sessions, venue, etc.). This information shall also be provided in writing via e-mail, if it is requested by the organization and / or user. 2. Confirmation of user participation. It will be in the second contact with organizations and / or users. They will confirm their participation in the activity. 3. Estimate of compensation for users. Once the schedule of meetings is closed, it is necessary to estimate a gratification for each user. These gratifications can be of different type and value, either one or the other will be fixed in advance, depending on the budget available for this activity and the characteristics of the tasks to be performed by users (duration, difficulty, etc.). 4. Reminder of the participants’ data required. Between one week before and the day before the expected date of session, those responsible for managing users will get in contact with the participants of that session to remind them of dates and other details of the session, concrete transportation needs if any, and send email consent documents available in Word format so they can read and, if possible, return signed prior to the completion of the session. For the signature of this documentation it shall be considered valid digital signature or the express permission of the person concerned by e-mail response to sending consent documents. Before participants give their consent or take part in the research they must be given information in terms that are meaningful to them. They should understand: • what the research is about • who is carrying out the research • who is funding it • any benefits to individuals or groups • any adverse effects • what the participants will have to do • length of the research • location of the research • what the research findings will be used for • what will happen to the results • whether they will receive a summary of the results • confidentiality agreements • entitlement to service statement 5. Reception of participants at sessions face-to-face. On the date and time indicated, the person responsible for management of users is responsible for receiving those invited to the session, and all persons whose involvement is necessary in relation to the participants (attendants, interpreters, etc.). This reception will include the introduction of participants to the person responsible of the activity and the collection of consent document signed, if it has not been sent by the participants in advance. 6. Developing sessions. The manager of the activity will be responsible of developing the session. If the presence of those responsible for management of users is necessary during the session, it shall be provided in advance. For people who are easily fatigued, researchers must be flexible and provide breaks or even cancel the session, if necessary (Harris & Roberts, 2003). It is possible that fatigue could lead in to biases in information collected. 7. Delivery of gratification. The delivery of gratification to participants is the responsibility of those responsible for user management. If the responsible of user management is not preset during the session, it will be needed to assess the appropriateness of letting activity managers to deal with rewards delivery. In any case, it is essential that participants sign a delivery receipt of the gratification.
7.1.1. Users involvement: keeping users interested One major issue in involving representative user groups in all research and development phases, and especially in that one of user evaluation, is the assurance of their availability in all project phases. It is considered wiser to keep involved the same pool of participants to the maximum possible extent, especially in the evaluation phases, in order to keep track of the degree up to which their initial expectations and needs have been fulfilled by the project iteratively upgraded solutions. On the other hand, frequent use of the same user groups risk to make these user groups by default become more technology-aware. In order to control this risk, it will be counterbalanced in Cloud4All via having also a percentage of users in each test to be newcomers. In order to make the contact and engagement process as systematic as possible, a forum of user representatives and stakeholders will be established in each pilot site in conjunction with A502.4: End users’ and beneficiaries’ forum. In addition, to use the forum to keep users informed of project activities in which they can participate, e-mail communications will be sent to those users who have participated previously in initial activities of WP101, such as A101.1, informing them of the results and project development, and inviting them to participate in new activities.
6.1 Inquiry letter This is a sample of the inquiry mail informing about our project and inviting users to be involved: Dear Sir/Madame The Prosperity4all team is currently researching about the disabled users` and developers´ views on feedback and feedforward mechanism regarding accessible software development. This is basically on how disable users express their needs and how this information is transmitted to developers. Moreover this project deals with the whole development cycle (Needs, requirements, ideas validation, Alpha & Beta testing) and how to improve the interactions user-developer in every stage. We are interested in an in-depth interview so that you can tell us what are the main flows, difficulties and ideas that can improve the aforementioned process. The interview will take about 20 minutes and we will try to cover 10 topics, all related with the user involvement in the whole development process. Therefore, if you can find a timeslot and are willing to take this interview, let us know. We are grateful in advance. None the less, If you want to know more about the Prosperity4all project visit: https://www.linkedin.com/groups/Prosperity4All-7467475 or contact firstname.lastname@example.org we will be delighted to tell you more about the project development. Kind Regards Victor Manuel Hernández Ingelmo Social Researcher at Fundosa Technosite 6.2 Informed Consent • Overall objective of Prosperity4all Prosperity4all, focuses on developing the infrastructure to allow a new ecosystem to grow; one that is based on self-rewarding collaboration, that can reduce redundant development, lower costs, increase market reach and penetration internationally, and create the robust cross-platform spectrum of mainstream and assistive technology based access solutions required. This will be done through a process based on true value propositions for all stakeholders and resulting in a system that can profitably serve markets as small as one, at a personally and societally affordable cost. This infrastructure will use cloud, crowd, game and smart technologies, to bring new players with both low and high technical skills into the development and delivery ecosystem, introduce accessibility as a ubiquitous service, and combine auto-configured access features built into mainstream products with assistive technologies and services to create the rich milieu of options needed to bring this diverse population of populations into our digital future. • Objective of this particular study In order to deepen on what user´s needs are related to their capabilities and preferences, Technosite´s team has decided to proceed to engage with users and developers in order to see the connection points that could be developed. Regarding users, researchers are going to engage in Guadalinfo centres in order to have a firsthand knowledge about what current status of user navigation and needs they see, moreover in-depth interviews will take place in order to know more about this connection points such as social networks, games, etc.It is very important that the user meets the specified requirements.He should be able to participate in the usability testing. On the developer side, a series of in depth interviews will take place in order to test some initial ideas such as connection of existing user for a and social networks with developers ‘communities All the information conveyed will be only used for research purposes and it will be treated according to the Spanish law of data protection (LOPD 15/99) harmonized with the European Directives on the topic. Research findings will be published, therefore data concerning your demographic information and first name will be public in such a way, as an example: “Juan, a retired 68 year old Guadalinfo user who is learning to use internet finds difficult to log in in a facebook app due to …” If you want your first name to be changed, please specify it You are aware that you can cancel the participation in this research any time you want