Research Project
Due Dates
- Proposals: Due Oct 22 (10%)
- Project Protocol and Ethics Report: Due Nov 5 (20%)
- Final Report: Due Dec 15 (40%)
- Project Poster Presentation: Due Dec 9 (30%)
All submissions should occur via blackboard as a single PDF for each of the due dates above, this includes submitting your poster (as a PDF).
Assignment Descriptions
You will complete a semester long project related to usability security and privacy. If you are not a PhD student, you may work in groups of two (or individually, in rare cases, with permission of the instructor). If you are a PhD student, you are required to work individually. It is expected that all projects have a user study that can either be qualitative or quantitative in nature, but you may also do a measurement or large/mid-scale comparison instead, if it makes sense for your project.
Historically, these small class projects can develop into real research projects and publications, so if you are interested in this research area, this is a great opportunity to explore an area that you are excited about. At the bottom are some sample research projects, but you are not required to use any of these. You can propose your own idea, if you want.
At the end of this project you will have completed a small research project, provided a written description of the research, produced a poster presentation with Q&A.
Mentorship
Every project will be assigned mentor. Mentors will either be graduate students or postdoctoral scholars. In addition to feedback from your instructor, you should get feedback from them. The mentors will also participate in grading process for project poster presentations.
Projects Proposal
Your proposal should include the following information, approximately one paragraph per-point.
- What research questions/hypothesis do you plan to address?
- What methods will you use in your investigate?
- What is your recruitment plan and target demographic?
- What is a timeline for your work?
- What are the ethical considerations? (see below)
- What is your analysis plan for any data you collect?
Following submission of your proposal, your group will schedule meetings with the instructor and mentors to get feedback and provide updates before final approval of your topic and research plan.
Project Protocol and Ethics Report
As this research project is for the purpose of training and educating students on how to conduct research and not to produce generalized knowledge research results, it is the general standard that this research does not require an ethics review. However, that does not mean you can conduct research unethically.
You are required to submit a preliminary draft of your study methods/protocol along with a justification that it meets ethical standards of treating participants fairly. Your ethics and protocol report should include the following section prompts with appropriate responses. The entire report should be 2-3 pages, not included attachments.
We will also use these ethic reports to provide feedback on your protocol. The goal is to help you scope your work properly to something that can be completed as a class project.
Ethics and Protocol Report
- Research Goals/Hypothesis and Questions
- 1-2 paragraphs of the research goals
- Detailed Description of Protocol
- Details of the entire protocol, what will be asked (~1-2 page)
- VERY IMPORTANT THAT THERE IS ENOUGH DETAIL IN YOUR PROTCOL TO PROVIDE MEANINGFUL FEEDBACK ON YOUR PROJECT
- Informed Consent
- How will participants be informed that they are participating in research (1 paragraph)
- Risks and Benefits
- What are the assocaited benefits (1 paragraph)
- What are the associated risks (1 paragraph) (EVERY STUDY HAS RISKS! You cannot say none.)
- How are those risks mitigated (1 paragraph)
- What is your plan to ensure participant confidentiality? (1 paragraph)
- Recruitment (1-2 paragraph)
- What is the recruitment plan?
- How many participants are needed?
- How are you going to ensure that all participants are treated fairly?
Attachment
- Survey/Interview Instrument
- A copy of the questions/instrument you plan to use in completing the study
- Recruitment
- A copy of any recruitment material
- Informed Consent
- A copy of your informed consent you will use
Project Status Meeting
While ungraded, we will conduct two meetings with mentors and instructor to check the status of your project. The first will occur shortly after your research protocol and ethics report, and the second will occur as you approach your poster presentation and final report.
Poster Presentation
Each team will generate and print a 48” x 24” (horizontally or vertically oriented) poster, which you will present as part of an end of term poster session. Your poster should be visual in nature (not too wordy!) but should also cover the topics below.
- Motivation
- Why this research?
- What has been done before? What is the gap?
- Methods
- What did you do?
- How did you analyze the data?
- Results
- What did you find? (visuals!)
- Implications/Discussion
- What do the results mean?
- Future Work
- What would you do next?
Do not forget to include good visuals of your result.
Printing your poster
You can print your poster for free through the GW library’s 3-D and Large Format Printing. There will be a special submission via a class project.
Note you may need to submit your poster well in advance of the poster session depending on the lead time of the library.
Project Final Report
Your final report should be between 6-8 pages, not including bibliography and appendix. It should be formatted as ACM, two column format. You can easily find this on overleaf. You should use the \documentclass[sigconf]{acmart}
header for the two-column format.
Your paper should have the following outline (and descriptions):
- Abstract
- One paragraph description of your research questions and motivations and a brief summary of the conclusion
- Introduction (containing the following details)
- Motivation (one/two-paragraph)
- Related work doesn’t cover this!
- Research Question (…)
- Method (…)
- Results (…)
- Conclusions/Contributions (…)
- Related Work
- Enumerate what’s come before but also(!) include how that related work matters to this research
- Methodology
- What was the method of investigation (survey/interview)
- Description of the survey
- Who do you recruit and from where
- Limitations
- Ethical Considerations
- Results
- How did you analyze it?
- What did you find?
- Address each hypothesis/research question, what is their answer?
- Discussion
- Interpretation and place in the context
- Now that you know the answer to a RQ, what doe that mean…
- What does this mean and how do we apply
- Future work/future directions
- Conclusion
- Rehashing of the motivations/research-question/methods/results/contributions
- Bibliography
- Your paper should cite at least 10 other papers!
- Appendix
- Entire survey/interview instrument
- Codebook (qualitative)
- Any additional figures and material is relevant for me to review
Bibliography
You should make sure you cite at least 10 papers in your final report bibliography. These papers should be related to your topic. They can also overlap with papers you covered in your deep dive or other papers from the class. Most importantly, these articles should come from reputable sources, such as:
- USENIX Security (Sec) (USENIX)
- Symposium on Usable Security and Privacy (SOUPS)
- IEEE Symposium on Security and Privacy (S&P) (Oakland)
- ACM Conference on Communication Systems (CCS)
- ACM SIGCHI Conference on Human Factors in Computing (CHI)
- ISOC Network and Distributed System Symposium (NDSS)
- Proceedings on Privacy Enhancing Technologies (PoPETS)
- Annual Computer Applications Conference (ACSAC)
- IEEE European Symposium on Security and Privacy (EuroSP)
- Symposium on Usable Security and Privacy (USEC)
- European Symposium on Usable Security and Privacy (EuroUSEC)
- ACM Asian Conference on Communication Systems (AsiaCCS)
- IEEE European Symposium on Security and Privacy (EuroSP)
- ACM SIGCHI Conference on Computer-Supported Cooperative Work & Social Computing (CSCW)
You may find papers outside of these venues, and you should check with the instructor if they are appropriate to use.
If your project overlaps with your deep dive, you are welcome to reuse those articles as part of your bibliography.
Resources
As this is a class project, you will likely rely on friends and family to be your research participants – or your class mates :) If you wnat, you can explore using paneling platforms like Prolific or Qualtrics, but this costs money to you. Alternatively, you can post your survey or requests to r/SampleSize subreddit which is a easy, but somewhat unreliable way to recruit participants for surveys. Crowdsourcing on twitter (X) or other social media is also a good choice.
If you are running a survey, you can use a number of free-ish survey platforms to host your survey. Many of these do have fees, but have a free version. This includes SurveyJS, LimeSurvey, Qualtrics, and, of course, Google Forms. Additionally, Microsoft also has forms through their Office 365 account that could be used to complete this project.
If you are conducting interviews, you may want to consider using audio transcription tools. However, I strongly caution against trusting these transcriptions completely and should check outputs for consistency. There are a number of products that link to Zoom and Google Meets to do transcriptions you can use, as well as a number of smartphone apps.
Grading
We will use the following grading scheme for all project grading schemes:
- Beyond Expectations: 100%
- Meets all requirements with high quality details and descriptions beyond expectations
- Meets Expectations: 95%
- Meets all requirements at a high level
- Satisfactory: 90%
- Meets all requirements with minor areas for improvement.
- Needs Improvement: 85%
- Significant portions should be improved
- Needs Significant Improvement: 75%
- Notable portions are incomplete and require significant improvement
- Unsatisfactory: 50%
- Does not meaningful satisfy the requirements of the assignment
For project proposal, ethics review, and annotated bibliography, for grades at or below 95%, you can resubmit.
Posters
For this part of the project, you will be graded by at least three reviewers (made up of the mentors and instructor).
- Content (30%)
- Are all aspects of the presentation covered, included motivation, methods, results, implications, and future work?
- Organization (20%)
- Are the details presented in a reasonable way with logical flow from one section to the next?
- Conclusions/Summaries (20%)
- Are the results properly summarized and implications given?
- Visual Aids (20%)
- Are there sufficient visual aids to enhance the understanding of the presentation without distracting?
- Q&A Session (10%)
- Are the presenters able to properly answer questions about their project?
Final Report
For this part of the project, we will use the grading scheme applied to each expected portion of the report.
- Motivation/Introduction/Abstrcated (20%)
- Is the motivation and goals of the project properly articulated?
- Are the results and conclusions properly summarized?
- Related Work (10%)
- Are related work presented clearly?
- Is the related work offering context to this result?
- Methodology (20%)
- Are the methods of the research clearly explained?
- Are recruitment and participant background discussed?
- Are ethical considerations discussed?
- Are limitations discussion?
- Results (20%)
- Are the results presented clear?
- Is there sufficient visual aids used and referenced?
- Discussion/Conclusion (20%)
- Are the results properly and meaningfully interpreted?
- Are conclusions offered?
- Bibliography/Appendices (5%)
- Are their sufficient resources cited
- Are additional material provided as needed, such as the survey instrument, qualitative codebook?
Project Topics
Below is a non-exhaustive list of topics and some general research questions that you can use to build a proposal. You may also propose your own topics. Note that you will need to develop your own more specific research question and methods of investigation for your research.
-
Authentication
- Biometric Authentication
- As more and more devices use biometrics, how does this new convenient method of authentication impact knowledge based methods of authentication choices and the perceptions therein?
- Do people choose weaker PINs/Passwords if they have a biometric? What are people’s opinions comparing biometrics with knowledge based authentication?
- Mobile Authentication
- How do users manage passwords on their mobile devices, how does that differ than how they manage them in traditional settings? Does it differ between applications and browsing?
- Some secure messaging services, like Signal, ask users to select a PIN, do users do this? Do they understand how the PINs are used? Also there was a lot o reminders about entering your PIN on signal, did they work or annoy?
- Password Managers
- Why do users or why do they not use a password manager in different settings? Mobile vs. Desktop?
- What are users preferences and features for password managers?
- How well do Password Managers actually work at auto filling? Can you empirically measure auto-filling success and failures of password managers in different settings?
- Many password managers have features for users to review and update passwords. How well do these work? Do users user them? If they do use them, do they understand them?
- What if we asked users who weren’t previously using a password manager to set one up? What do they do? How does it affect them?
- There’s a lot of discussion of passwords managers on online forums, like Reddit, what problems do people report on and what advice is given?
- Biometric Authentication
- Digital Sharing
- Users want/need to share secrets online, such as a password or PII. If so tasked, how would they do it? Would they use email or text message or something else? What are the threat models and security understanding of users when they share secrets online?
- We share a lot of documents in the cloud, how often have you gone back to actually review all of that sharing? If users were to reflect on documents they previously shared, would they change anything by adding more restrictions, or just leave it be? What are the threat models of sharing documents this way?
- Security of Signal/Whatsapp/SMS
- More and more users are using texting applications that provide end-to-end encryption. Do users user all these features? How do they understand the security provided?
- Can two users properly establish secure channels using these apps?
- (see the mobile authentication one for PINs in signal)
- Voice Assistants
- Creepy or necessary? How do people understand and use voice assistant technology?
- Voice assistants often record conversations even when users are not aware, but you can go a look at these recordings. How do people feel about these un-aware recordings?
- Two Factor Authentication
- It’s clearly better, but what might stop users from using it? How many accounts do ussrs actually have 2fa installed on?
- If there is two-factor, do people end up making worse choices elsewhere because they have a false sense of security?
- Video Conferencing Privacy and Security
- So many more people are using video conferencing, what are their concerns? How do they mitigate them?
- How do users manage private spaces in their home. Do they choose remote backgrounds, yes or no?
- Private Browsing Mode
- What do people actually use this for and what is their expectations of privacy when using private browsing mode?
- VPNs
- VPNs are very common for people to use but misunderstand. What are the common reasons for people using VPNs compared to the actually security and privacy they provide.
- There is a lot of VPN advertising on YouTube – how does the claims of the advertisers compare to the actually features that VPNs provide
- Developer Studies
- So much development happens in the open on github—that’s public information!—so take a look at all the open issues for security related issues. How quickly are they closed? Do developers care about them?
- LLMS and Generative AI
- If you were seeking security and privacy advice from an AI systems, is it good? What kind of variety of advice?
- How do users compare the advice from a LLM compared to that of an actual security and privacy expert
- Breaches and Identify Theft
- We have all probably been a part of a breach, but how do we understand what happens when we are a pat of breach? Do we do anything about it? You can study participants responses after looking up their information in a breach database.
- You may have signed up for identify theft, perhaps in response to a breach or other notification— what then? How often do you use? Is it helpful? How usable are these services in the first place?
- Phone Shops
- Small mobile phone shops and their workers offer technical support to customers about their devices. What questions are they asked about security and privacy and what answers do they provide? Are they good?