WP204/WP205 Meeting on 2012-12-07 12:00 UTC
- 1 Telephone Bridge
- 2 Important Information for the Meeting
- 3 Participants
- 4 Approval: Minutes of the last WP204 meeting
- 5 Update: Activities
- 6 WP204/5 deliverables for 2012
- 7 First Evaluation Phase
- 8 Next Meetings and Topics
1. To talk free using the Web click: https://www3.gotomeeting.com/join/705233502 Be sure to set AUDIO to Mic and Speaker (VoIP) - a headset is recommended.
2. Or, call in using your telephone.
Meeting ID/Access code: 705-233-502
- Australia: +61 (0) 7 3123 6029
- Austria: +43 (0) 7 2088 1400
- Belgium: +32 (0) 92 98 0592
- Canada: +1 (416) 900-1165
- Denmark: +45 (0) 69 91 88 62
- Finland: +358 (0) 942 41 5778
- France: +33 (0) 182 880 456
- Germany: +49 (0) 898 7806 6461
- Ireland: +353 (0) 14 845 976
- Italy: +39 0 247 92 12 39
- Netherlands: +31 (0) 208 080 379
- New Zealand: +64 (0) 4 974 7215
- Norway: +47 21 03 58 96
- Spain: +34 911 82 9782
- Sweden: +46 (0) 313 613 558
- Switzerland: +41 (0) 225 3314 51
- United Kingdom: +44 (0) 203 535 0621
- United States: +1 (786) 358-5410
Important Information for the Meeting
This is a meeting of WP204 and WP205 (Matchmaker) within the European Project Cloud4All. However, the minutes of the meeting will be made available to the public to allow people outside Cloud4All to follow the development of the matchmaker. If, in the meeting, you want to share confidential information that should not appear in the public minutes, you need to state so in the meeting. This is in line with the Cloud4All Consortium Agreement, section 10.
- Boyan (ASTEA)
- Christophe, Gottfried (HDM)
- Claudia (TUD)
- Kostas (CERTH/ITI)
- Kasper (RtF-I)
- Eleni (CERTH)
Approval: Minutes of the last WP204 meeting
(See also the preparations for the first evaluation phase, below.)
WP204/5 deliverables for 2012
This agenda item was skipped to focus on the first evaluation phase.
D204.1 Profile matching technical concept (due 2012-10-31)
ID205.1 Platform for experts to define a set of adaptation mechanisms for the virtual users in a specified use context (Report and prototype)
First Evaluation Phase
See also Cloud4All Testing.
Questions and actions from testing call on 2012-12-06 (WP402, SP2, SP3):
- What are the terms that we need. CS to check whether the terms from CL's user research can be mapped to ISO 24751:2008 terms, else if they map to other terms found by the N&P Ontology activity. When no matching terms are found, suggest new terms for the temporary Registry.
- CL to add columns to the table with solutions for the first evaluation phase, so the MM team has more information for scenarios etc.
- After the previous two steps, a new telecon will be organised, i.e. with the relevant SP3 developers, the Matchmaker team, WP402 and Architecture. Date: 13, 14 or 17 December, to be determined after a Doodle poll.
- Will the rule-based matchmaker and the statistics-based matchmaker be integrated into the Architecture or run locally?
- By the second phase, will the rule-based matchmaker and the statistics-based matchmaker be integrated into the Architecture or run locally?
Eleni has asked for having all applications tested by the rule-based matchmaker.
- But we have only one scenario for the matchmaker.
- Not a problem in general, but we will not have time for that.
- Maybe create a separate scenario for integration ("instantiate settings"), but that's not our job. Kasper: Working on this with the SP3.
Decisions for testing:
- For matchmaker tests, we will focus only on:
- Platform migration scenario
- Use case C2 (Windows) and C1 (Linux)
- Windows big screen, Linux small screen
- Preference terms from Claudia's set and FLUID panel settings (but aligned with ISO/IEC 24751 as much as possible)
- Blind users and user with or without vision impairment
- Blind users group: Windows Narrator and ORCA on Linux
- Sighted users group: FLUID settings on a Web page
- Tasks to perform on a web page:
- READING: Reading an email or a longer text (with FLUID / WCAG 2.0 AA compliant)
- BROWSING: Searching information within a web page (with FLUID / WCAG 2.0 AA compliant)
- FORM FILLING: Online banking simulation (with FLUID / WCAG 2.0 AA compliant)
- Strip off FLUID panel for blind users
- We need to test the pages on Web accessibility with actual users
- Independent variables:
- Demographic data...
- Familiar with computer, with IT products, with their needs and preferences, use of AT
- This will happen during the first setup time, setting the user's preferred settings on the first platform
- At the end of the first platform, the user has to check whether they are satisfied with the settings.
- Use a subset of the metrics at Matchmaker Benchmarking Metrics:
- Completion (binary)
- Post-task: User-perceived difficulty level per task (Likert 1-7)
- Post-task: Ask the user for good and bad experiences.
- Standard questionnaire per matchmaker (!AttrakDiff)
- Independent variables:
- Between-subject tests!
- We also need an expert-provided matchmaking result, for comparison with the two matchmakers. This would be a third group.
Comment: Eleni wants to include this information in D402.1:
"Concluding, the aim of Pilots in Cloud4all is not to test the solutions themselves, in terms of usability, effectiveness, efficiency, etc., since these are, in their vast majority, existing mainstream solutions and applications. On the other hand the major focus of the Cloud4all pilots will be the auto-configuration success/potential/effectiveness of the SP3 solutions/applications through the GPII/Cloud4all system, which is the one to be tested. Especially In the 1st iteration phase, a local version of the system will be employed in order to accommodate the autoconfiguration testing (Should we say here which components will be integrated to the CLOUD and which will run locally?). However in the upcoming evaluation phases, the autoconfiguration scenario will be employed through the GPII/Cloud4all system and also locally (will be run locally? do we care in having tested localy? or it will be only on Cloud??)
- Further question from Eleni: How long before the actual tests do you need the N&P set of the users been gathered, in order to insert them in the system??
Next Meetings and Topics
- Next Fri Dec. 14, 2pm CET
Previous meetings: Matchmaker Meetings