A case study on the development of a CISO dashboard to provide security leaders with real-time visibility and actionable insights.
Researcher J2BD: Why, What, How
1
Why
The key reasons for this research are the uncertainty of the end user, unspecified CISO Dashboard KPIs and requirements, and the uncertainty of the legacy components needed for the CISO Dashboard view.
2
What
The research aims to validate the end user focus persona (CISO/CIO), define the CISO Dashboard functional requirements, and reveal the legacy relevant components to migrate to the new product.
3
How
The research will align with the end user needs - the CISO - to understand the risk in the organization and fill the unknown for the CISO.
Research Questions
Who is the CISO persona?
The research aims to validate the end user focus persona as the CISO/CIO, understanding their responsibilities and day-to-day activities.
What are the CISO responsibilities?
The research will explore the CISO's key responsibilities, such as managing cybersecurity risks, ensuring compliance, and making data-driven decisions.
What is the CISO day in a life?
The research will aim to understand the typical day-to-day activities and workflow of a CISO, to ensure the dashboard aligns with their needs.
Will the CISO be the main end user persona?
The research will validate the assumption that the CISO will be the primary end user of the dashboard, and explore any other potential user groups.
What are the CISO must have KPIs?
The research will identify the key performance indicators (KPIs) that are most critical for the CISO, to ensure the dashboard provides the necessary insights.
What is the estimation of usage?
The research will explore the expected frequency and duration of usage for the CISO dashboard, to inform the design and development process.
Research Tool Kit
Planning & Tracking
Jira - planning and progress tracking
Collaboration
Miro - collaboration
Documentation
Word Office - Research questions and transcripts
Confluence - documenting data
User Research
UserTesting - recruitment, interviews & feedback
Microsoft forms - questionnaire
Generative Research Overview
SME Interviews
Conducted qualitative interviews with internal subject matter experts to gather in-depth insights and perspectives.
Qualitative External CISO Interviews
Interviewed CISOs from other organizations to understand their responsibilities, pain points, and requirements.
Online Desk Secondary Research
Performed online open-source intelligence (OSINT) Desk research to gather additional insights and validate findings.
Thematic Analysis: KPI Definition
Conducted a thematic analysis to define the key performance indicators (KPIs) that are most critical for the CISO.
Prioritization Matrix Survey
Gathered feedback from CISOs through a prioritization matrix survey to understand their top priorities and requirements.
Gap Analysis: New KPI vs. Legacy Ability
Performed a gap analysis to identify the differences between the new CISO dashboard KPIs and the former capabilities.
"Quick Wins" Identification
Identified "quick win" opportunities that could be implemented quickly to provide immediate value to the CISO.
Generative Research: Interviews
1
Qualitative External CISO Interviews
Interviewed CISOs from other organizations to understand their responsibilities, pain points, and requirements.
2
SME Interviews
Conducted qualitative interviews with internal subject matter experts to gather in-depth insights and perspectives.
Generative Research: Online Research
Online OSINT Secondary Research
Performed online open-source intelligence (OSINT) research to gather additional insights and validate findings.
Comprehensive Approach
Conducted thorough online research to complement the qualitative interviews and uncover a broader range of data points and perspectives.
Generative Research: Thematic Analysis
Thematic Analysis: KPI Definition
Conducted a thematic analysis to define the key performance indicators (KPIs) that are most critical for the CISO.
Comprehensive Approach
The thematic analysis provided a structured and rigorous framework to deeply examine the research data and identify the most important KPIs for the CISO dashboard.
Generative Research: Prioritisation Matrix
Prioritisation Matrix Survey
Gathered feedback from CISOs through a prioritisation matrix survey to understand their top priorities and requirements.
Collaborative Approach
The prioritisation matrix survey allowed us to directly engage with CISOs and gather their input on the most critical KPIs and features for the dashboard.
Data-Driven Insights
By analysing the survey results, we were able to identify the highest priority KPIs and requirements from the CISO perspective, informing the development of the dashboard.
Validation of Assumptions
The survey helped validate our assumptions about the CISO's needs and priorities, ensuring we were aligned with their real-world requirements.
Generative Research: Gap Analysis
Gap Analysis: New KPI vs. Legacy Ability
Performed a gap analysis to identify the differences between the new CISO dashboard KPIs and the capabilities of the legacy security systems.
Identifying Misalignments
The gap analysis allowed us to pinpoint areas where the new KPIs and requirements were not fully supported by the existing security tools and components. This helped us understand where integration challenges or additional development might be needed.
"Quick Wins" Identification
Identifying Quick Wins
The research team highlighted "quick win" opportunities that could be implemented quickly to provide immediate value to the CISO.
Pragmatic Approach
By identifying these "quick win" solutions, the team was able to take a pragmatic and action-oriented approach to delivering tangible improvements to the CISO dashboard.
Research Outcomes
The Product roadmap got clearer and tighten - mini workshop of Now - Soon - Later tasks.
UX design team started designing the CISO dashboard according to the "quick wins" KPI.
Multiple product teams had started to gather the data for the CISO Dashboard
New Research J2bd - Discovery of the other persona that will use this tool
Product Pivot
Stakeholders pivoted into approaching lower level persona
Stakeholders Rethink
Stakeholders rethought the persona and considered the time and effort required for the changes.
Adjusted Solution
The team adjusted the solution to better address the persona's problem space, taking into account the new insights gained from the research.
Research Outcome Impact
Product Time & Effort understanding
The research provided a better understanding of the time and effort required to implement the product changes.
Company money saved
The research insights helped the company save money by identifying more efficient solutions.
New personas definition
The research discovered the personas that will use the tool, providing valuable insights to better address their needs.
Increased sales orientation to product decisions
The research helped orient product decisions towards a stronger sales focus, improving the product's market fit.
Stronger strategically collaboration with our stakeholders
The research fostered stronger strategic collaboration with the stakeholders, aligning the product roadmap and development.
Lessons Learned
Ongoing Process
The researcher's job is never over. This is an ongoing process.
Assumptions vs. Reality
Assumptions and reality are two different things.
Questioning the Research
Always ask questions, even question the research itself.
Raising Flags
Raising flags is important. When necessary, do that.
Continuous Feedback
Continually sharing back is super critical. Can save money.
Pivoting
Pivot can be a very good outcome as well!
Market Fit
Products are only great if there is a precise market for them.
Quick & Dirty Approach
Streamlined Research
Interview less external CISO's, conduct 2 researches at once (both personas), and delegate to other team members to simultaneously research in addition to the lead researcher.
Leveraging AI
If allowed, use AI to cross-check the primary research insights with the Osint secondary insights, to accelerate the process.