TOPIC
Metrics and measurement
These are the same questions found in the library, grouped here by the capability each one mainly examines.
200 questions
Questions 51–100 of 200 on this page.
A · Professional problem solving and implementation1
No questions in this section match your target. ·
B · Business understanding and domain modeling3
- The team plans to enter an adjacent market where users, payers, and fulfillment parties differ. How would you map the value chain and determine whose problem the product should solve first?
- Business reports mix orders, order lines, and payment attempts, so metrics disagree across pages. How would you establish fact-table grain and validate aggregation semantics?
- Two models call a feature with the same name but receive different values due to defaults and missing-data handling. How would you establish a contract and compatibility checks? Follow-up: If a critical feature refresh misses the real-time decision window, how would you choose among stale values, fallbacks, or refusing the decision?
No questions in this section match your target. ·
C · Value judgment and direction setting44
- A feature is used often but has not improved retention or payment. How would you determine whether it creates core value, reflects habit, or rewards the wrong behavior?
- Overall conversion improves, but new users and low-connectivity regions worsen. How would you examine segmentation, selection bias, and problems hidden by the aggregate?
- After several teams optimize the same north-star metric, duplicate content and low-quality interactions appear. How would you diagnose metric distortion and adjust guardrails?
- A new platform could reduce future development cost, but the legacy system needs two more years of support and this quarter funds only one major investment. How would you compare build, migration, and status-quo options?
- As the industry enters a stock era, the company demands a complete shift from thinking in transaction volume to thinking in net profit. How do you raise net profit by adjusting the take rate and optimizing supply-chain operating costs without triggering large-scale exits or protests from merchants or creators?
- How would you calculate a product direction’s return on investment? How would you account for labor, cost, and revenue, use annual recurring revenue (ARR), and identify where the calculation might overstate value?
- A free feature attracts many active users, but inference and support costs rise too. How would you evaluate the business role of the free tier?
- Marketplace subsidies increase transactions, but supply and demand fall back when subsidies stop. How would you judge whether the subsidies created sustainable value?
- During a brand campaign, organic search and direct visits rise, while search-ad spend also increases. How would you estimate brand contribution without counting conversions intercepted by performance ads as brand outcomes?
- A campaign randomly holds out 10% of users, but they may still receive sales outreach and organic recommendations. Conversion is higher in treatment, and the channel team wants more budget. How would you assess contamination and results?
- Orders rise 30% during a discount campaign, but many come from existing customers who would have bought anyway, and product mix changes. How would you estimate incremental gross margin?
- The algorithm team expects an upgraded recommendation model to lift conversion by 3%, but operations would lose the ability to adjust key placements manually. How would you design an experiment and decision rules while accounting for value the experiment cannot measure?
- After the bot upgrade, sessions with no handoff to a human are counted as resolved, and the rate went from 72% to 91%. In the same week the handoff button moved into a submenu and the denominator changed. How do you respond today to the proposal to cancel weekend human coverage?
- What business result did your project bring? How do you confirm it came from this project and not from something else?
- Retention rose during a marketing campaign, and the team credits the channel, but a new product flow also launched and participants may already have been more active. How would you establish a credible comparison, estimate the campaign’s net lift, and state the limits of the conclusion?
- Who you compare against changes the conclusion directly. Why did you choose these comparisons, and how would your conclusion change with a different one?
- When you benchmark against a competitor's data, how do you decide what is comparable and what is not? What decision did you finally use it for?
- The completion rate of a key task fell this week. Before explaining why, in what order do you confirm that the change is real, where it is concentrated, and what cost users bear?
- How is your core metric defined? What are the numerator, denominator, time window, de-duplication, and exclusion rules, and why is it the core metric?
- How was your funnel built? Why are the steps divided the way they are, where does most of the drop-off happen, and how do you confirm the steps neither overlap nor leave gaps?
- How do changes at each funnel stage ultimately act on the business metric? Improving which stage by a little moves the business metric the most, and what is that judgment based on?
- Once your metric is converted to the industry's common definition, where does it stand? What assumptions did the conversion make, and what information did it lose?
- A dashboard holds dozens of metrics and the team's attention keeps getting pulled to the most visible numbers. Pick one user task and explain how you would tier the metrics and decide which enter the core list.
- An improvement you made has no reliable numbers. What other evidence would you offer that it really improved something?
- A target was set for a metric you own. How do you break it down into specific levers and actions, and judge along the way whether it can be met?
- The metric was met, but you felt the business did not really get better. Has that happened to you? How did you notice, and what did you do?
- After a feature launches, how do you judge whether it met its original goal? When do you review it, and what would you decide differently afterwards?
- The boss raises a business pain, such as high acquisition cost and poor retention. How do you translate it into goals the technical team can tackle, and show how technical means act on those business metrics?
- For the last new feature you launched, what were the specific before-and-after changes in the core metric, such as conversion, click-through, or retention? If the numbers fall short of expectation, along which dimensions do you usually investigate the cause?
- A feature encourages invitations, so inviters and friends may land in different experiment groups. Results are significant, but spillover cannot be fully isolated. How would you assess the conclusion?
- User mix or external conditions change during a project, making the original baseline incomparable. How would you continue judging the outcome?
- The target metric improved, but refunds, complaints, or manual handling also worsened. How would you decide whether the outcome is acceptable?
- The initial effect of a feature is strong but fades over several months. How would you distinguish novelty from lasting behavior change?
- The final business outcome will take months to appear, but the team must decide whether to continue investing now. How would you use leading indicators?
- Several teams changed the product, pricing, and operations at once, and the overall outcome improved. How would you assess each contribution?
- An experiment shows no expected lift, but also does not prove the idea ineffective. How would you decide whether to continue, change, or stop?
- Two projects each meet some goals, but one delivers more user benefit per unit of investment. How would you reallocate resources?
- A pilot meets its target, but results vary when expanded to more regions or segments. How would you judge whether the result is replicable?
- A redesign improves paid conversion but leaves more users unable to complete a core task. How would you judge whether the growth is worth it?
- A new product exploration has consumed two quarters. Pilot completion stays below target, while the team argues one more feature will fix it. How would you judge evidence for further investment and stopping?
- A high-traffic flow was changed into sequential steps, raising overall completion. Support reports, however, show users struggle to find exit, undo, or change options. How would you determine whether this is a real experience improvement, a segment difference, or a metric artifact?
- A legacy feature has few users but supports month-end reconciliation, and maintenance cost keeps rising. How would you compare migration, paid maintenance, and retirement?
- A new process doubles weekly feature releases, but task completion is unchanged and support tickets rise slightly. How would you decide whether to continue, adjust, or stop?
- Describe a technology-platform project you worked on that few teams used and that produced little visible business value after launch. What evidence or assumptions justified the investment at the time? Which assumptions did later evidence disprove, and what validation did you add?
No questions in this section match your target. ·
D · Delivery and ongoing operations2
- A new product misses its first-week sales target, yet trial users show strong usage depth and referral intent. How would you determine whether the issue is reach, a longer purchase cycle, price threshold, or positioning, and decide what to invest in next?
- A channel has a low acquisition cost in a small test, but lead quality and conversion fall as spend increases. How would you validate the cost definition and true incrementality, distinguish channel saturation from audience or sales-capacity changes, and set a limit for additional spend?
No questions in this section match your target. ·