Thursday, March 23, 2017

Evaluation of Privacy Permissions Usability in Windows Phone, Android and Apple iOS

By Saeed Alharthi

Every modern mobile system nowadays provides the user with a level of privacy permission control to limit what information about the user an application can have access to or gather. It is very important for users to know which information about them is shared with an application on their phone. Users should always be in control of what the application can know about them and what information does the application have access to.

For that, we have examined the privacy permission settings for applications in three mobile operating systems (Windows Phone, Android and iOS) to evaluate them from the user usability perspective. We focused on what options are not clear to the user and which options are not available. We highlighted the best practices we found from the three operating systems and suggested some solutions to issues found by eight test users.

Issues Found

  1. Privacy permissions section is not visible on installation of a new application
Figure 1: In Windows Phone, no privacy permissions section appears on the top of the page, it is found at the bottom of the page under "This app can"
Figure 2:  In Android, privacy permissions section doesn’t appear on the top of the page, it is found at the bottom of page under "Permission details"
Figure 3:  In Apple iOS, privacy permissions section doesn’t appear on the top of the page nor can it be found at the bottom of page
Two of the Windows Phone users didn’t think the information was available in the page. This was due to the location of the permission info at the bottom of the page and how the page was filled with many textual information that the user must read through to find the permission info (Figure 1). Android users didn’t have difficulties locating the option at the bottom of page, this is due to the large size font used and the clear icon (Figure 2). iOS however doesn’t have an option to show what permission an app may use so none of the test users could complete the task (Figure 3).

Solution suggested: A section for privacy permissions should be added near the top of the page. In Figure 4, 5 and 6, we show an example of how the option for the privacy permissions can be added in each system.

Figure 4: The "Permission info" can be moved under the application rating section in Windows Phone
Figure 5: The "permission details" link can be added beside the install button in Android
Figure 6: A "Privacy" tab can be added to the links on top of the page

  1. No option to access the application permission from the app menu (shortcuts when holding the app icon)
When asked to perform the second task, one tester reported that Windows Phone, Android and Apple iOS do not provide a shortcut option to access the app privacy permission from application icon. On the other hand, some android third party launchers do have this shortcut option and its more convenient method to find the privacy permissions for an application as the tester suggested.

Solution suggested: An entry can be added to the application shortcuts menu that link to the system privacy permission page (or perhaps even to the complete settings menu) for the application as suggested in the figures below.


  1. No settings menu for a specific application’s permissions in Windows Phone. Only by permissions category.
Users were asked to find which permissions an application is granted. For apple iOS users, this was relatively easy since each application’s privacy permission page was accessible from the main settings menu. Even though there is no heading to point out that the privacy permissions were accessible from the application list. Similarly, for Android users the app permission is accessible through the apps section in the settings menu. However, for Windows Phone, the option was not available even after digging through the different sections of the settings menu. The only method to locate the privacy permissions in the settings was in a categorized list under the privacy option in the settings menu.

Solution suggested: A clear section for the privacy permission should be added to the setting menu in Windows Phone. For iOS, a heading needs to be added on top of the application list.

Figure 7: In Windows Phone, there is no option for permission list by application
Figure 8: Application permission is found under the "Apps" section in Android
Figure 9: Application permissions is found at the bottom of the settings menu without heading
  1. No option in Android and iOS to disable a permission category for all apps at once.
In Windows Phone, a feature that was noted by the test users was the option to disable a permission category with one toggle switch instead of having to disable each app’s access separately (Figure 10). A user noted that this is convenient because it works like an undo button when they re-enable a category (location for example), only the applications that have been granted the access previously will be re-enabled rather than having to go through the list and re-select which apps to grant the access to and which not.
This option is not available in Android and iOS however and based on the user feedback it will be a good feature to add in the future.

Figure 10: a single switch toggle to enable or disable access for all the apps to select permission
  1. Applications not working or crashing without a notification message of unauthorized access attempt in Windows Phone and Android
In Android: many applications do not detect the denied permission sitting and don’t give informative message to the user. For example, the camera in android will give the message “Can’t connect to the camera” which may suggest to the user that the issue might be in the camera hardware (Figure 12).
Similarly, in Windows Phone, it’s up to the developer to implement the privacy setting detection and notify the user which some developer may not pay attention to. For example, the camera application will display a warning message and ask the user to change the privacy setting where WhatsApp will simply ignore the user request with no feedback to the user (Figure 13).  

Solution suggested: To avoid such confusion between the apps and the operations system, a notification message should appear to the users notifying them of a denied access request from an application.

Figure 12: in Android, the camera app will not accurately identify the denied permissions
Figure 13: In Windows Phone, While the camera app will notify the user of denied permission, WhatsApp will simple ignore the user actions without feedback


The results we have from this evaluation show that each operating system approach has its advantages and disadvantages. The main flow in all three system is that they make it harder to access the privacy permission settings when it should be very easy. We pointed out where the design of the UI can be improved based on our testers feedback as well as some major issue with how applications are handling the denied permission errors.

Wednesday, March 22, 2017

Evaluation of Facebook's Fan page Creator module (with independent users)

 By Tolu Olufowobi

           A typical Facebook user is connected to about 88 pages according to a 2016 statistical infographic found on Blog Herald. That is to say, fan-pages have become commonplace on Facebook. With Facebook expanding its repertoire to accommodate businesses, bloggers, and smaller website owners, the Fan-page can be regarded as a crucial avenue for brand and business promotion within Facebook. It provides opportunities for organizations to establish a direct connection between the products and services they offer and users on Facebook. Therefore, it can be regarding as a critical part of Facebook's working model even though many regard it as an auxiliary feature.
For this reason, I conducted an evaluation of Facebook's Page creator module- the set of pages from which users can create and manage pages. I involved 4 independent users who have no stake in Facebook. The essence of the evaluation was to identify any usability problems that may have been missed by testers and developers at Facebook. 
          Over the course of the sessions, the users were able to identify a total of nine problems. The problems were categorized according to how they violate the set of guidelines proposed by Jakob Nielsen in 1994. 
Here are the issues pointed out by users along with accompanying categorizations (for usability enthusiasts and experts):

What was reported

      Problem I.  Obscure link to the Fan-page Creator link

          The “Page” link at the bottom of the sidebar is obscure to the untrained eye. Facebook has provided 2 alternative ways to navigate to the fan-page creator start page- one, through the “Pages” link in the same sidebar, and the other in the drop-down arrow on the search and menu bar. However, only the “Pages” sidebar link seems like an apparent  way to navigate to the page creator start page. The other, because it has been embedded in the drop down list, isn’t easy to find either. In addition, the term “Page”, on a social networking website that made up of a plethora of pages, can be quite vague. 
A gif showing a user's almost futile effort to navigate to the Fan page creator page.

Violated guideline(s): 
  • Guideline 4 (Consistency and Standards): “Users should not have to wonder whether different words, situations, or actions mean the same thing.”
  • Guideline 6 (Recognition rather than recall): “Minimize the user’s memory load by making objects, actions and options visible.”

  1. Change the font size of the links at the bottom of the sidebar to make them more noticeable.
  2. Adopt a different icon for representing extended menu items, or drop-down lists e.g. “…” or the “hamburger” menu icon – the icon with 3 horizontal bars.
  3. Use a tooltip to inform users of the use of the drop-down arrow, if it must stay.

    Problem II. Unclear distinction between the User account and Fan-page album

          Although the links exist on two separate pages, users, especially, first-time users, may assume that the “Photos” link in the Fan-page dashboard links to photos in their personal profile. One potential mistake that can arise from this, is unintentional posts meant for private use ending up on the fan-page. It goes without saying that anyone who runs into a problem like this might refrain from using fan-pages on Facebook in the future.

The pictures above show that the same label is used for photos in the user's homepage and their fanpage... potentially leading to confusion. 

Violated guideline(s):
  • Guideline 5 (Error Prevention): User errors, like the example given in the description above, haven’t been prevented, given the current adopted U.I.

  1. The “Photos” link display text in the fan-page could be changed to “Fan-page photos” or just simply “Page photos”.
  2. Photo uploads to user accounts and the fan-page could be designed with the possibility of getting them done in the same palace. This way, the user is immediately aware that the posts can be made to either of the two, leading them to choose appropriately.

 Problem III. No User Account – Fan-page media transfer options available

          It is impossible for users to move or copy pictures from their profile to fan-pages they manage. The closest thing to this is sharing pictures on the page itself. However, sharing only makes pictures available on the fan-page’s timeline. 

The user looks for a way to move pictures from their account to the fanpage , to no avail.

Violated guideline(s):
  • Guideline 7 (Flexibility and efficiency of use): The module, and by extension, the website, has failed to provide a way for users, in particular, perpetual Fan page admin users, to move pictures freely between their account and pages.

  1.  Add a feature that allows fan-page owners or administrators to perform user account to fan-page media transfer.

Problem IV. Profanity label is ambiguous

          At a time when internet trolling and cyber bullying are rampant, this setting is ever so important. With two clicks, users can use Facebook’s vulgar language blacklist to block offensive comments from being posted on the page. However, from the evaluation sessions conducted, only one of the users was able to identify the setting when asked to block offensive posts on the fan-page they were asked to manage. The others users either required some form hint or failed to understand the setting altogether.
After initially missing the setting, the user employs the information icon to learn more about the setting.

Violated guideline(s): 
  • Guideline 2 (Match between the system and real world): The 2nd Nielsen heuristic guideline states that “the system should speak the users’ language, with the words phrases and concepts familiar to the user…”. The term “profanity”, while clearly describing the setting it represents, is not a familiar word to most users.

  1. Change the choice of word used to identify the setting.
  2. Provide adequate description for the setting.

  Problem V. Illegible verbose text

          All the participants from the evaluations sessions indicated that, when viewed at a glance, the descriptions text for the settings in the “Settings” page are difficult to read. The small, close to fine print text, discouraged them from trying to spend much time understanding what it did. They mostly relied on the title of settings, to identify which ones they wanted to use. In addition to these, users who went ahead to seek assistance from the help search box, weren’t satisfied with the suggestions provided as they found them misleading.          
The text in the circled area was illegible according to the evaluation participants.

Violated guideline(s): 
  • Guideline 8 (Aesthetic and minimalist design): “Dialogues should not contain information which is irrelevant or rarely needed”- the descriptions given for some of the settings haven’t been constructed in a concise manner

  1. Make the descriptions legible.
  2. Adopt concise descriptions all around to encourage their reading.

Problem VI. Scheduling event posts

          The inclusion of an extra modal dialogue form, which can only be accessed by selecting “Schedule from the drop-down list next to the “Post” button,  to set the scheduled date for posts seems unnecessary. More so, as was the case in 2 evaluation sessions, users may set an invalid scheduled moment to make the post. Now, an invalid moment can be regarded as one whose date or time comes after the actual date or time of the event. 
Users should be prevented from making wrong data entries like the one shown above

Violated guideline(s): 
  • Guideline 5 (Error Prevention): The event post scheduling dialogue form prevents users from setting an invalid scheduled post date.
  • Guideline 6 (Recognition rather than recall): the positioning of the “Schedule” link violates this guideline.

  1. Post scheduling can be done on the same page by employing some extra controls -scheduled time and date field, and a checkbox- and some Ajax code or any technology that modifies page content asynchronously. 
  2. Although Facebook uses form validation to ensure that no invalid date or time can be selected, it may be more convenient to the user, if such values are not selectable in the first place.

Problem VII. Clutter around the post creation area

          The timeline posting feature on Facebook has been around almost since the beginning of Facebook itself and has undergone a series of changes over the years. Most recently, it contains new features that allow users to share new types of content such as videos, pictures, events, to name a few, as timeline posts. However, the layout adopted to display the features, shadows the textbox originally used to make posts. It’s hard to establish if this is by design, but one thing is for certain, some users will not be able to make simple text posts because of this. To compound the issue, Facebook has modified the “post” button within the fan-page home page, making it a thumbnail of the users or fan-page’s display picture. 
User fails to sight the post creation textbox.

Violated guideline(s):
  • Guideline 8 (Aesthetic and minimalist design): The clutter of features around the post entry text field violates contradicts this guideline.

  1. Provide a clear demarcation between the post entry field and the additional features available for posting.
  2. Replace the picture with the post button. The author of the post can be setting using radio buttons or a toggle control.

Problem VIII. Misinterpreted page deletion prompt

          The prompt displayed when a user attempts to delete a page from its settings, can be mistook for a feedback message. 
The user leaves the page before responding to the prompt.

Violated guideline(s):
  • Guideline 4 (Consistency and Standards): Links aren’t standard way to prompt users for a response.
  • Guideline 5 (Error prevention): User errors are bound to occur if the prompt message looks like a feedback message.

  1. Replace it with two buttons and a prompt message so users can easily identify that they are being prompted for a response.

Problem IX. Buttons that don’t do anything

          The “edit” buttons next to owned pages in the “Manage Pages” page don’t do anything other than become invisible when they are clicked. Furthermore, the icons used don’t agree with convention as they are different from the edit icons used on Facebook. 

Notice the buttons become invisible with the user remaining on the same page when the buttons are clicked

Violated guideline(s):
  • Guideline 4 (Consistency and Standards): The use of different edit icons in the Page creator is in violation of guideline 4.
  • Guideline 5 (Error prevention): The existence of the buttons is in itself of a violation of Nielsen’s heuristics guideline 5.

  1. The page can be taken out since any editing that can be done on a Fan-pages can be done from within its dashboard.
  2. If the buttons aren’t responsive because a condition hasn’t been satisfied, then the link to be shouldn’t be available until after the condition has been satisfied. Also, stick to the icons already associated with editing on facebook.


          It appears that much of the problems identified by the users arose from Facebook’s adoption of a minimalistic and aesthetic design. Furthermore, the argument can be made that design or performance related justifications are behind some of the decisions Facebook has taken for it user interface design. Conclusively, none of the recommendations provided are absolute, neither have they been tested. They are simply suggestions that may provide a starting point for the development of a concrete user-centric U.I.s.


Tuesday, March 21, 2017

Usability issues in CuLearn

By Sania Hamid

CuLearn is Carleton’s collaboration and online learning system. It offers support to instructors, teaching assistants and students. It has features like assignment submission, discussion forums, grades upload, course announcements etc. [1]. It is based on Moodle which is an open source learning management system, distributed under the GNU license [2].
Despite the advantages of using this online system, it suffers from many usability issues. This blog lists few of the usability issues faced by a teaching assistant while using CuLearn and provides recommendations to tackle those problems. These issues, if fixed will provide a better user experience.

Usability Issues

1. Problems while adding recipients to an email
The interface for adding recipients to an email is shown in Figure 1. There is no search option to look for a particular user. Multiple recipients cannot be added at a time. The email address of the recipients is not provided.

Figure 1: Interface for adding recipients to an email
The interface should be made condensed with an option to search for a particular recipient. Checkboxes should be provided to add multiple recipients and. The email address of the recipients should be made available.

2. Problems while signature to an email
Figure 2 shows the interface for composing an email. To add a signature while composing an email the user must select the signature which was previously saved under a certain title. Even after selecting the signature, its contents are not visible. To create a new signature the user has to go to a different webpage as shown in figure 3 and save it under a title. This option is not available on the compose email webpage.

Figure 2: Interface for composing an email

Figure 3: Interface for creating a signature
There should be a “Create Signature” button available while composing an email. The signature should be displayed when it is selected to add to an email.

 3. Issues with accessing the contents of a previous email
Figure 4 shows the interface which lists the history of the previously sent emails. It provides a short summary with date, subject, attachment(s) and status. It also provides two actions which are open and delete. The problem faced is that, if the user is looking for the content in a particular email then he has to open each and every email to find it. There is no option for him to search through all the emails at once. The emails do not open when the subject is clicked. The magnifying glass icon under the Action column is misleading because on clicking it opens the email instead of providing an option to search.

 Figure 4: Interface of View History 
There should be a search option, to search through the contents of the previously sent emails. Subject names should be made as active links. The magnifying glass icon should be removed as it is misleading.

4. Feedbacks while marking assignments
Figure 5 shows the interface were teaching assistants can upload or change grades and add feedback comments while marking assignments. Two incorrect feedbacks are encountered while marking assignments. The first one is shown in figure 6. This modal dialogue shows up when users make changes either to the grades or the feedback comments and does not clearly specifies where exactly the changes are made. After saving all the changes made, when users try to exit the pages a pop-up message shows up as shown in figure 7. This message is completely wrong as the users saved all the changes.

Figure 5: Interface for marking assignments

Figure 6: Feedback after saving grades or feedback comments

Figure 7: Feedback while exiting the page 
The first feedback should only indicate the changes which were saved so that the users are not confused. Another change can be by simply highlighting the changed and saved fields with a different color like green, this will help in indicating that the changes were saved and the modal dialogue can be avoided.  The second feedback should be removed as it is completely wrong and unnecessary.

 5. Issues with marking attendance
The interface where users can start marking attendance is shown in Figure 8. The first option shown in the table needs to be selected to start marking. The green icon in the Actions column should be clicked to start marking. The icon is not representative of what it indicates. For the first-time users, it seems like an indication that the class is in session but that is not true. It is a button which takes the users to a new webpage where they can mark attendance.

Figure 8: Interface for marking attendance
Since the green icon confuses the first time users, it should be replaced by a button that says “Mark Attendance”.


[2] Moodle, URL:

Monday, March 20, 2017

Usability Issues with Battlefield 1 & EA Origin

By Troy D’Angelo

Online gaming communities are social networks built between players who share an interest in a game. The game Battlefield 1 allow players to organize themselves into squads, the purpose of a squad is to complete team-based objectives directed by their squad leader, while the compilation of multiple squads achieves the entire game objective. Players can invite their friends to ‘game parties’ where the party can play in the same squad - allowing for a personalized squad-team. Thus, the formation of squads amongst peers is critical to success within the game, however, several usability issues are present that hinder the ability for successful task completion and form the motivation for the user study.

This user evaluation features three participants who are all active gamers (1+ hr. per day) and focused on the task completion of forming a squad from a list of friends and entering a game together.  This blog post contains a summary of usability issues indicated within the study conducted. Two tasks are featured in the post: Formation of a Game Party & Joining a Game.

1.0 Formation of a Game Party

1.1 Problem: Controller Incompatibility

Battlefield 1 is launched with full controller support; however, participants were unable to access the game party menu (highlighted orange - figure 1) when using a controller. Most of the UI available in the main screen of the game (highlighted blue – figure 1) may be navigated using the analog stick or directional pad controls. The error was propagated from the participants abandoning use for the controller after numerous attempts and resorting to keyboard-mouse interactions until they were in-game. Additionally, participants reported difficulty locating the game party menu due to an inconsistency in design between other primary UI elements such as the navigation tabs (highlighted green – figure 1).

Figure 1: Battlefield 1 – Main Screen – shown is furthest access point by controller

Error Category
Error Type
Unable to select UI elements using controller
Human-Computer Interaction (Physical) Error
Interaction Element Problem
Every use. Demonstrated process is the only option

Full controller support needs to be implemented for the game. Whether this requires retooling how controller inputs are handled or updating the UI, the error simply should not exist. This would allow users to scroll from either the panel or tabs to reach the game party options. Another possible solution would be to integrate a Friends tab in the taskbar -the main elements of the game are accessed through this method. Adding a tab would allow more intuitive and familiar option.

1.2 Problem: Lack of Game Party Feedback

Participants faced task completion failures when attempting to successfully accept a game party invite. When invited, the participant is prompted to press SHIFT-F1 to pull up the Origin Overlay (Figure 2). They are then presented with a menu where they must navigate to the Friends List index (highlighted blue – figure 2). Upon selection, a dialog (highlighted orange – figure 2) is shown where an arrow icon (highlighted green – figure 2) is to be clicked.  There is no indication (feedback) to the participant that they have joined the party or even successfully clicked the button resulting in participants to believe the task was incorrectly completed and causing them to navigate through other options and menus.

Figure 2: Battlefield 1 – Origin Overlay – shown is party invite accept dialog, no feedback is given

Error Category
Error Type
No feedback given to user when confirming the joining of a game party.
Overly Complex UI and Lack of Feedback.
UI (Client) Issue
Interaction Element(s) Problem
Every use. Demonstrated process is the only option

Integrating a party system within the game itself would eliminate most the errors within the task or the UI needs to be redesigned to allow for a logical workflow. Navigation through multi-dialog pop-ups while effectively minimizing the game in the background results in user confusion, frustration and detracts from gameplay. Addition of a direct method to accept the party invite from the game such as pressing Shift-F3 (unused key binding) would provide a method that is familiar (Shift-F1 launches the Origin Overlay) and prevent errors from dialog navigation. Implementation of feedback is additionally required - including a small prompt indicating successful joining of the game party would result in the UI aligning with traditional Heuristic guidelines.

2.0 Task: Joining a Game

2.1 Problem: Queues

Upon selection of joining a friend in game there is a significant chance (20% Chance, 10 trials) that the joining player will be placed in Queue (Figure 3). This is a result of games that are available within the area having only one spot available. Available games are decided by location, skill and rank, having a party with two varying player’s increases difficulty of finding an appropriate server significantly. Often the waiting player will be placed in Queue, entering the game when another player leaves. Once exiting the queue the joining player may be placed on the opposing team rather than within a squad with their friends.

Figure 3: Battlefield 1 – Game Queue Screen

Error Category
Error Type
Game party members may be relegated to wait in queue while other members play
Bug or Functionality Defect with Quick Match
Conceptual Model Problem (Game Algorithm)
20% (N = 10)

The algorithm should be updated or redesigned to integrate squads into games. Several algorithmic option that could be implemented are:

  • Reserve the queue for squads - eliminating the issues where a player waited in Queue spot #3 (shown Figure 3) when part of a two-person group. Single players would not be allowed to queue ahead of a party when one member of that party was already in-game. 
  • Place all users into the queue for a new matchmaking game - often new games take 2-3 minutes to completely fill, and the game party can form their squad correctly from the beginning of the match (eliminating the opposite team scenario)
Proposals will not fix all issues within matchmaking decision making, however, considering the difficulties involved for a two-person squad were to such an extent - a change needs to be made to improve social playability.


Battlefield 1 features a content rich and immersive gaming experience – coupled with EA’s Origin platform gamers from across the world can partake in the epics of WW1. However, with respect to the social networks in the game – they are easy to construct but difficult to maintain. A talk-out-load heuristic evaluations was conducted with two participants. Tasks included adding a friend, forming a game party and joining a match. The results found participants experiencing difficulties with joining a game party, navigating vague UI’s and culminating with not even joining the same side on the battlefield after lengthy waits in Queue. The final recommendations of this report to DICE and EA are to implement an alternative to the Origin UI with respect to formation of game parties or greatly simplify the process of squad formation.