How to Build a List of Hypotheses for Mobile App (Guide for Hypothesis-Driven Development)

Henn Akimov

Marketer, Startup Advisor

Ihor Polych

CEO at Devlight

“Most businesses die because they offer a product that consumers don’t need” — this is a famous saying of Eric Ries, the author of the Lean Startup methodology. So how can a hypothesis-driven development help your project avoid this trap?

The answer is simple — it aims to research the demand for your future product before making a mobile app. So it is worth starting the research by compiling a set of hypotheses about the needs of consumers. That answers the question about what problems and difficulties your future product will help solve.

Forming hypotheses is a creative process, and it is difficult to follow a certain procedure, but some rules still apply. In this article, we will describe an algorithm for creating a set of product hypotheses and their further verification upon user surveys.

What Is Hypothesis-Driven Development?

A hypothesis-based approach allows product developers to design, test, and refactor a product until it is acceptable to consumers. This methodology involves testing and refining the product based on consumer feedback to verify the assumptions made during the ideation process. The utilization of this approach helps to eliminate any uncertainties in the design phase and leads to a final product that is well-received by users.

Here are some examples of hypotheses for mobile app development from various segments:

• The behavioral hypothesis demonstrates user behavior under various conditions and what drives people to act in a certain manner;
• The difficulties users encounter and the justifications for why they regard such challenges as obstacles to their objectives are covered by the problem hypothesis;
• The motivation hypothesis focuses on the wants of users and the reasons why they are currently ineffective in accomplishing their objectives;
• The blocker hypothesis reveals the cause of the present ineffective conduct or difficulty.

Why Do We Use a Hypothesis-Driven Development?

When developing a product, you define your hypotheses, find the fastest ways to test them, and use the results to change your strategy.

You have a lot of assumptions, to begin with. You predict what users want, what they are looking for, what the design should be, what marketing strategy to use, what architecture will be most effective, and how to monetize the product.

Some hypotheses need to be corrected. You don’t know which ones. CB Insights determined that a lack of market demand was one of the main causes of startup failure. Almost half of these projects spent months, or even years, building a product.

The only way to test a list of hypotheses for the mobile app is to give the product to a potential customer as soon as possible. If you follow this methodology consistently, you will realize that most hypotheses fail. You assume, fail, and have to go back to the beginning each time to test new hypotheses.

This approach is not an innovation in product development. When you write a book or essay, you spend a lot of time editing and revising. When you write code, you also redo it. Every creative endeavor requires a huge amount of trial and error.

In this world, the one who detects their own mistakes and corrects them faster becomes the winner. The most important thing is to determine which of your hypotheses is wrong with the help of feedback from real users. Thus, when you’re building a product, writing code, or developing a marketing plan, always ask yourself a few questions:

• Which hypothesis in the project is the most doubtful?
• What is the fastest way to check it?

How Does A Hypothesis-Driven Development Look Like in Real Life?

Let’s look at a simple example. Let’s say we choose a project approach (one that sets a task, not puts forward hypotheses) to a service for selling goods. We decide to add a delivery option to it. We decide to hire delivery people, buy them branded clothes, bags, and possibly transport. The development team creates a page where you can enter the delivery address and the desired date. Then we write a service that transfers the order from the store to the delivery person and an application for those delivery people. What happens in the negative scenario? That’s right. We’re losing hundreds of thousands of dollars.

What if we had a hypothesis-driven approach? First of all, we would write hypotheses and confirm that customers, in the first place, need the delivery option. Then we would understand the optimal cost and delivery time to calculate the unit economy. Next, surveys or interviews of users would be conducted to give us an understanding of user needs. Then we would make a fake “delivery” button on the website or in the app to see how many clients would try to use it. 

Of course, this action cannot be used to calculate the exact demand because there are still dozens of ways to kill the conversion after the user’s clicking the button: complicated fill-out forms, poorly available delivery periods, high costs, etc. But, at least, we would understand how many people out of 10 thousand, who saw the button, tried to use it — three or eight thousand. So then, to test the hypothesis in real-life conditions, we would use a ready-made B2B solution rather than develop our delivery feature. 

Moreover, to save the integration time, we would collect orders, put them in a database, and then pass them over to our manager, who would manually issue each delivery through the third-party service web form. What would happen in the worst-case scenario? Nothing too serious. We wouldn’t have wasted hundreds of thousands of dollars and many weeks on development.

To sum up, hypothesis-driven development aims to understand what product feature will bring the greatest value at the moment and test this feature in the simplest possible way. To put it bluntly, try to refute each of your hypotheses as soon as possible. Proving to yourself that an idea is worthless without spending time on its development is morally difficult but very effective from the company’s activities point of view.

A hypothesis-driven approach provides a structured way of consolidating ideas and building hypotheses based on objective criteria. This methodology provides a deep understanding of prioritizing features related to business goals and desired user outcomes.

How to Test the Hypothesis of Product Demand and Value Without Development

Starting development without testing the key hypotheses behind the new product is a widely spread mistake. In this case, you are completely sure of your idea and see no point in testing it but begin the development process immediately.

The second most common hypothesis-driven development mistake is to look for confirmation of a hypothesis instead of testing it. Often, demand or value testing becomes a formal step. The decision is not based on received data but on initial settings and startup owners’ prejudices. This cognitive distortion happens for several reasons:

• Commitment to an idea blocks critical thinking (typical of startups);
• The bureaucratic apparatus perceives testing hypotheses for mobile app development as that part of the project development process, which is inevitably followed by implementation, regardless of the results of the test (typical of corporations). Even if all the early tests show that the product in its current form does not stand a chance, it still goes into development;
• The third mistake is testing unimportant things. Instead of testing key risks (demand and value), secondary elements related to subjective perception (appearance, non-core functions, etc.) are tested. As a result, time is wasted, and the hypothesis-testing process itself is devalued.

Testing the Demand Hypothesis for a New Product

The demand hypothesis is one of the riskiest assumptions behind a new product. This hypothesis assumes that the potential audience is interested in solving a certain problem. The demand hypothesis is also called the need hypothesis or the problem hypothesis. 

It is necessary to study the target audience and its tasks in order to check the demand, sometimes to sell a product that has not been created:

• The most common way to test the demand is to create a landing page with a detailed description and illustrations of the product and show it to potential buyers;
• In some cases, you don’t need to create your site— just place an ad on a platform attracting the audience of potential customers for the product;
• The demand for some products is difficult to check with a landing page or an announcement on social networks. Especially if the sales process includes a long conversation, a call, and sometimes a meeting with the buyer. You can use targeted advertising and personal communication in such situations. Again, without yet creating an actual product;
• If deciding to buy your product requires minimal experience interacting with it, you can offer customers a shell without filling;
• One of the easiest and most effective ways to test demand without development is to show users videos simulating how the product works. This way, you can demonstrate its capabilities, interface, design, and situations where the product will be useful.

Testing the New Product Value Hypothesis

Once the demand list of hypotheses for the mobile app is validated and you know that the product solves the desired problem among potential buyers, the next key risk is the value. The value hypothesis assumes that the product’s intended implementation will bring customers real value. It usually means that the product will solve users’ problems more effectively than alternatives available on the market. Otherwise, users will have no motivation to switch from one solution to another:

• Allowing users to try something as close to the future product is the most proper way to test a value hypothesis. This can be done with the help of third-party services that reproduce complex functions and automate their work without writing your code;
• Alternatively, to check value hypotheses for mobile app development without development, you can reproduce the process of the system in manual mode;
• The third method lies in value validation through the prototype. Usability testing of prototypes allows you to see the process of using the product, and subsequent interviews give a fairly accurate understanding of the presence or absence of the value of the solution being studied.

How to Build and Test List of Hypotheses for Mobile App

The HADI (Hypothesis – Action – Data – Insights) methodology is the simplest algorithm for cyclical testing of ideas – from hypothesis through action to data and conclusions.

The hypothesis-driven development management cycle begins with formulating a hypothesis according to the “if” and “then” principles. In the second stage, it is necessary to carry out several works to launch the experiment (Action), then collect data for a given period (Data), and at the end, make an unambiguous conclusion about whether the hypothesis was successful, what and how it can be improved by launching the next cycle of hypothesis testing (Insights).

Step 1. Forming a Hypothesis

Here you formulate what you want to know. What problem are you trying to solve? Determine which product level you should test:

• Value level —  test the problem your product is supposed to solve. Understand whether it is worth solving;
• Feature level is a functionality through which the user quickly realizes the value of our product;
• Design level means the design and visualization. How does your functionality work in terms of user experience? Simply put, will people intuitively figure out how to manage, where to click, and what to do with your product?
• The feasibility level of hypothesis-driven development is about the technical implementation of everything you have created.

The hypothesis is based on the principle “If…, then…”. (if…then)

You can also prioritize the hypotheses to be tested. Think about what might have the biggest impact on your users’ needs and prioritize accordingly. You can also use the ICE Score framework, which includes these  three elements:

1. Impact;
2. Confidence;
3. Ease.

ICE is calculated as follows:

Of course, this is only one option — several formulas you can choose from existing. However, remember that the formula for all hypotheses you compare should stay the same, and your ICE should have the same rating range — either 1 to 10, 1 to 100, or another scale (determine it in the beginning).

Impact

Impact estimates how much an idea will positively affect the metric you’re trying to improve. To determine the impact, we ask the following questions: How effective will it be? How much will this affect the metrics (conversion, retention, LTV, MAU, DAU)?

Confidence

Confidence shows how much you trust the impact estimates and ease of implementation. To determine this hypothesis-driven development metric, you need to answer the question: How confident are you that this feature will lead to the improvement described in Impact and be as easy to implement as described in Ease?

Easy

Ease of implementation is an estimate of how much effort and resources are required to implement this hypothesis. The easier the task, the higher the number. To determine the ease of implementation, you need to answer the question: How long will testing these hypotheses for mobile app development or developing this feature take? How many people will be involved? Consider the work of the development, design, and marketing departments.

Step 2. Performing the Action

At the beginning of each cycle, we take several hypotheses and start testing them using the next methods:

• A/B Testing or Split Testing

In such testing, the main thing is clearly defining the sample or its size. This is important so that the results are as realistic and statistically significant as possible. We recommend conducting split testing with at least ten thousand active monthly users. If your audience still needs to be bigger, it is better to use other tools.

• Quantitative User Survey

Use special services facilitating survey creation and implementation. For example, Survey Monkey. Such services allow you to select the desired audience and ask them questions. With the free plan, you can create questionnaires with up to 10 questions. The link to the questionnaire can be placed on the website or social networks.

• Qualitative Research or Customer Development

This research type is a direct conversation with consumers or a certain group of potential product consumers. Such hypothesis-driven development interviews can be divided into two groups:

1. Usability— will help to understand whether users can use your product at all to solve their tasks with its help and achieve their desired goals;
2. Discovery – delves into the state, problems, and perceptions of users of a certain group in detail. In such interviews, we usually ask questions like “Who? How? Why? Where?”

How many such interviews do you need to test the hypothesis? We usually start with five. Then we continue until people stop giving new answers. You can stop as soon as the information starts to repeat itself. For hypotheses testing small product changes, 5-7 interviews may be enough. For the launch of a completely new product — 50-70 interviews.

Step 3. Data Analytics

At this stage, we collect data from our research. You should have a backlog of your hypotheses for mobile app development prioritized according to certain criteria to help you at various stages of development. Approach all feature development from the perspective of hypotheses. A good indicator is when you have two states: one in which an experiment is being planned to test the hypothesis and another in which functional validation is ongoing and data is being measured. 

Then, when your experiment is over, you can identify the hypothesis as being supported, refuted, or even abandoned if you decide to call it quits due to the results. You may ensure that you reach any necessary pivots as early as possible and prevent investing in needless work by always tackling the highest-risk hypotheses first.

Step 4. Insights

This stage can also be called interpretation. First, you should analyze whether your list of hypotheses for the mobile app was confirmed (worked). Whether a theory is confirmed or refuted, the process itself offers a chance to learn. Even if you cannot support the hypothesis, the result may offer insightful information that you might use for a different hypothesis.

Now that some of your hypotheses have been supported, you can proceed to development. But even once the product is released, testing must continue. In addition, you should be alert since certain aspects can need to change due to client needs, market trends, regional economics, and other factors.

List of Hypotheses for Mobile App: Example

ABC (name changed) is the largest provider of microcredits in Ukraine. They have no physical branches — their services are fully digital and offered online. However, ABC has thousands of contented customers and devoted staff members. The figures speak for themselves:

• During the first half of 2021, ABC’s net income was estimated at 44 million dollars (the biggest number among competitors);
• The company has a net profit of $1.4 million;
• ABC’s team consists of more than 700 workers;
• They are frequently used by 1.8 m Ukrainians;
• 6,000,000 loans were made in the service;
• Their total issued money circulation is $1 billion.

It is a sizable, contemporary, and well-run business that turned to us to help them with diversification and new ways of development. The customer’s goals were growing the company, diversifying the line of goods, and breaking into a new market. Devlight used this data when forming the list of hypotheses for the mobile app.

Internal Discussions and Hypotheses Forming

First, we gained a profound knowledge of the ABC team’s technology, product, capabilities, vision, and passion through our meetings. We saw that we could accomplish our ultimate objective thanks to our significant experience working with neo-banks and our business knowledge. 

The Ukrainian market was solely focused on loans as of 2021. Users voluntarily took out loans for various purposes, including small household and personal expenses, buying vehicles, and starting businesses. Loans were a common practice. It was typical, a common occurrence that clients were fully aware of.

However, the market’s offerings fell short of users’ needs. They were one-dimensional and impersonal. We concluded that this vulnerability is exactly where we can compete. Depending on their credit score, we can provide different consumers with flexible credit limits or high limits with a longer grace period. For instance, the market had nothing like a big credit limit of UAH 100,000 for 100 days. As a result of ABC’s significant experience in the credit industry, we can accomplish this relatively effortlessly.

One of the advantages of ABC’s business model was its capacity to deal with credit scores and potential hazards properly. These advantages enabled us to formulate the premise of a flexible product credit engine, which we could then use to develop the product’s key competitive advantage. The primary target audience had to be used to test this idea. It would serve as our foundation for hypotheses for mobile app development. 

Do you keep failing to form hypotheses for mobile app development? Devlight will be happy to point you in the right direction. Be sure to contact us!

Hypothesis-Driven Development: Summary

Do not worry that your hypotheses will be incorrect. Your objective is not to convince everyone that you are correct. Your objective is to establish a prosperous business. The hypotheses are merely a tool to get you there, so the more of them you debunk — the better. Finally, keep in mind that a hypothesis-driven development:

• is about a sequence of tests to support or refute a theory. Determine value!
• provides a quantifiable result and promotes ongoing learning;
• enables the user — a critical stakeholder — to provide continuous feedback to comprehend the unknowns better;
• enables us to comprehend the changing environment and gradually expose the value.

Apps that were developed based on tested hypotheses have a big and advantageous impact on a company’s business objectives. Utilizing data that is closely related to the company’s goal guarantees that customers’ needs are prioritized.