ANNEX A: KPIs across Prosperity4All project

From wiki.gpii
Jump to: navigation, search

Prosperity4All vision

Key Performance Indicators

Relevant SP/WP

Measuring techniques/tools

1) Reduce costs

Reduction of time required by implementers for developing/adapting/modifying an SP3 application, using the SP2 tools = at least 15% reduction reported for developers knowledgeable on how to make an application accessible, and at least 30% reduction reported by developers with little experience in accessibility issues (mean value over all project evaluations with developers in all iterations).

In the context of P4All WP402

 Qualitative assessment via interviews and questionnaire feedback forms

 

At least 25% of users report that they are able to use the user‐facing tools and/or SP3 applications that they were not able to do before, or are more efficient; without additional cost or ‐even‐ at reduced cost.

In the context of P4All WP403

Related to Efficiency and indirectly to Usability (as a positive effect to output quality; reduce time and costs)

At least 60% of involved stakeholders present a positive WTH attitude and 30% of them a positive WTP range.

In the context of P4All WP404

Online feedback forms (part of impact assessment)

2) Address the full range of users

Year 3: SP2 tools related to individuals with various access needs achieve a usability and user acceptance rating (mean) over 3 in a 0‐5 scale.

Year 4: SP2 tools related to individuals with various access needs achieve a usability and user acceptance rating (mean) over 3.5 in a 0‐5 scale.

The full range of beneficiaries is specifically testing the Prosperity4All developments within WP403.  Use of common performance indicators for all groups for comparison reasons.

Usability (e.g. SUS and task-based usability evaluations), accessibility, and User acceptance scale (e.g. TAM) standardized scales

 

 

T401.2: Recruitment – ensure diversity by exploitation of Prosperity4All Collaborative Network

3) Address the tails and the tails’ tails

At least 20% of the developers who use the DeveloperSpace or Tools report that they are more likely or will develop for smaller markets as a result of the DeveloperSpace, Tools, or the Unified Listing and Marketplace. This would be significant because AT vendors are struggling already and asking them to address the tails where the profit is very small, is very hard for them.

In the context of WP402 for implementers and in WP404 for a more extended view coming from all types of stakeholders

User acceptance questionnaires (e.g. TAM), open-questions, feedback mechanisms for platform (e.g. online feedback tools)

4) Address all technologies

70% of developers enhancing/developing SP3 applications make successful use of SP2 tools (in each application domain).

In the context of WP402 for implementers

Interviews &Utility and Usability metrics (aspects of utility are inherent in usability)

For any SP3 application enhanced with Prosperity4All SP2 tools, user acceptance is improved by at least 1 unit in a 0‐5 scale (compared to the initial rating of the non optimized applications)

In the context of WP403 for end users

User acceptance (e.g. TAM)

5) Provide a plan/mechanism for creating a vibrant, profitable, assistive technology market

The number of business models deemed appropriate for the functioning and viability of the Prosperity4All ecosystem provided by SP1. At least 65% of the Consortium stakeholders (notably companies), Project Collaborators and other external stakeholders report that the SP1 market mechanisms proposed address their needs and could be adopted by them.

In the context of SP1 and WP404. Also related to exploitation activities (SP5)

Quality of life, cost-effectiveness, and organizational quality are valid measuring indicators for impact assessment and will be assessed via online feedback forms

These quality attributes will be enriched as soon as business models within sP1 will be finalised

 

6) Decrease costs and expertise required of mainstream         companies

Same as for number 1‐A) above. Also the number of mainstream companies embracing the Prosperity4All ecosystem and using our tools.

At least 3 unfunded mainstream companies express strong interest in the project results during the project lifetime and/or become Collaborators.

In the context of WP402 for implementers, WP404 and exploitation related

Online feedback forms (rate of new comers)

7) Do a better job of moving research and development to market

The number of exploitable components from other research that are accessed from the Prosperity4All/GPII developerspace and incorporated into a commercial product or service. Number of times a Prosperity4All/GPII service infrastructure is used by SME to offer a service. (also during project execution possible)

Exploitation related

These measuring techniques/tools will be set by activities within SP5 and addressed in respective deliverables.

8) Involve consumer and consumer expertise in product development

At least 30 consumers per site involved in product development design during the course of the project using the P4A platform and tools.

In the context of WP404 for stakeholders; also exploitation and dissemination related

Usage/popularity, globalization/diversity can be measured in impact assessment of (as an outcome of exploitation, dissemination activities)

9) Be based on realities, business cases, and value propositions

Same as for objective number 5 above

 

 

In the context of SP1 and Exploitation related

Prosperity4All infrastructure will be based on sound business models that will be developed in SP1 and will be assessed in the context of WP404 impact assessment. Built-in feedback mechanisms will collect the stakeholder’ opinion on their value. Relevant evaluation objectives: Quality of life (of all different participating parties), Cost-effectiveness, Organizational quality.

10) Recruit and engage more players

Number of unsubsidised external developers (SME and Large) that engage in the project during its run >10 (of which at least 5 with little or no experience in making products accessible)

In the context of T401.2: “Participants recruitment”, reinforced through the dissemination and exploitation activities of SP5.  

This is basically a success criterion of the recruitment and dissemination processes

Recruitment utilizes the Prosperity4All Collaboration Network to ensure this KPI is met

For stakeholders: feedback monitoring tools (i.e. popularity/usage indicator)

Number of unsubsidised external stakeholders engaged in P4A during the project duration >15

(>10 involved in crowdsourcing paradigm, > 5 clinical)

In the context of WP404 for stakeholders Also:

In the context of T401.2: “Participants recruitment”, reinforced through the dissemination and exploitation activities of SP5.  

The engagement of stakeholders will be monitored/rated through the “popularity/usage” indices of impact assessment, which will be performed automatically i.e. (feedback tools)

As mentioned above, this is basically a success criterion of the recruitment and dissemination processes. 

 

11) Not forget documents, media,and services

Number and quality of results coming out of WP204 on Media and Material Automated/Crowd sourced Transformation Infrastructures.

 

Extent of usage of the enhanced features of these tools produced within the Prosperity4All project.

In the context of WP204 and WP404

Number and quality of WP204 results will be set within the work carried out by SP2.

 

Assessed in impact assessment with indicators such as: popularity/usage, conversion rates (visits with actions/visits no actions), number of downloads/access, average potential revenue per visit (for WP404)

12) Provide both technology and human accessibility service support

70% of users rating the Assistance on Demand over 3 in a 0‐5 scale (of those that tested it)

 

In the context of WP403

User acceptance scores (e.g. TAM) compared to reference case & user experience metrics (e.g. questionnaires, interviews, service diaries, etc.)

>75% of users at each site able to use Assistance on Demand service.

In the context of WP403

Accessibility, utility, and general user experience qualitative and quantitative data

13) Work across all domains of life

Implementations by external unfunded parties in all of the identified domains within the project cycle.

In the context of P4All WP402 in collaboration with activities within SP2 and SP3 and accommodated by dissemination and exploitation activities of SP5. This is a KPI addressed to most SPs in the project.

Records of number and types of external implementers interested in the project (i.e. interviews, communication reports, etc.)

For any SP3 applications enhanced with P4All tools, user acceptance improved by at least 1 unit in an 0‐5 scale.

In the context of WP403

User acceptance ratings (e.g. TAM) and user experience metrics (diaries, questionnaires, etc.)

14) Be applicable, and work internationally

External implementers participating in evaluations from at least 8 countries including 2 non-European.

In the context of WP402, motivated through SP3 and accommodated through Dissemination and Exploitation WPs.

Effective utilisation of networks already established(i.e. various entities have signed letter of collaboration) will be defined in recruitment protocols for evaluation phases (discussed in chapter )

This is a success criterion also for dissemination and collaboration activities within the project in order to attract external implementers to participate in evaluation among other project activities

Stakeholders participating in Evaluations from at least 8 countries including 2 non-European.

In the context of WP404, accommodated through T502.3 End users’ and Stakeholder Connections

Relevant success indicators could be diversity, popularity as an impact assessment indicator

 It is a prime dissemination target to attract stakeholders and engage them in any project activities including evaluation