User Preferences UX Meeting 2013-02-12

From wiki.gpii
Jump to: navigation, search


Rating System

One of the goals for Cloud4all is the development of a tool that allows users to provide feedback:

(...) develop and test the tools for users to manually correct the results of the applied matching algorithm, in order to make the change effective immediately and for similar cases in the future (learning algorithm). Also, the user can provide feedback by rating the usefulness of the current adaptation settings – which will again be fed back into the matching process.

Development activities include:

  • User interface for fitter to provide application-generic controls for user-controlled corrections of automatic adaptations.
  • User interface for rating system in which the user can assess the usefulness of the currently selected adaptation options in a particular context. Usefulness will be measured along multiple dimensions, including effectiveness, efficiency and user satisfaction.

Feedback questions may be very simple, like the questions that Skype asks about the quality of a call after the end of a call, but the number of questions should be low (possibly just two questions). However, asking the user to rate each change suggested by the matchmaker would be irritating. The user should be in control. (Providing feedback may be an opt-in feature.)

Some matchmaker changes may be surprising for a user; the user may not understand them. Other changes would be acceptable. For example, when a sensor in a phone detects a change in ambient light and adapts the brightness of the screen, this would be acceptable and less jarring than some other changes.

Another issue with asking user feedback or approval of changes has to do with usability. For example, on a cell phone, you can't review all the changes if it's a large number.

What would we do with data from ratings? What does a rating between 1 - 5 say? Perhaps the rating of matchmaker changes could be derived from something more useful: when the matchmaker suggests a change, and the user changes the setting to another value (or puts back the original value), we could calculate the distance between the matchmaker's suggestion and the user-selected value. Such distance metrics will be used in the first evaluation phase; see the table with preferences selected for matchmaker testing. A short distance would imply a good rating, a long distance would imply a bad rating. However, distance metrics would need to be defined for each setting or preference term, including terms that use a set of enumerated values as its value space. (The changes made by the user in response to a change suggested by the matchmaker would also be fed back to the matchmaker, so the rating would be derived from data that can be used for improving the matchmaker algorithms.)

When changes by the user (like the above) are recorded together with the context in which they occur (data from sensors, the application itself, the time of day), we get lots of data that can improve matchmaking. But we would also be harvesting a lot of data about users and their habits. This has implications for ethics and privacy. Users should be able to turn this off.

The matchmaker feedback mechanism should have a button to "put it back the way it was", and possibly something like "don't change this again".

PMT, PCP and Feedback to the Matchmaker

So far, the PCP and the Matchmaker Feedback Mechanism have been treated as a single tool. Perhaps they should be separate tools:

  • The Personal Control Panel (PCP) would only contain a minimal set of settings or options, only the settings that the user wants there. The PCP would be used when, for example, the user wants to temporarily increase the font size to read some small text and then decrease it again afterwards.
  • The Matchmaker Feedback Mechanism or MM Communication Panel would be just for feedback to the matchmaker. The PCP might have a button to launch this tool.
  • The Preferences Management Tool (PMT)

Merging the PCP and the Matchmaker Communication Panel may overwhelm users. Many users may never want to interact with matchmaker changes; they don't even know that the Control Panel on their OS exists; they don't want to deal with this level of complexity.

Note: Preferences Management Tool (PMT), Personal Control Panel (PCP) etc are names that we use internally in the project, but we want to find less nerdy names for them before we give them to users.

See also