TOPIC
Growth experiments
These are the same questions found in the library, grouped here by the capability each one mainly examines.
43 questions
A · Professional problem solving and implementation1
No questions in this section match your target. ·
B · Business understanding and domain modeling1
C · Value judgment and direction setting20
- An acquisition campaign drives many registrations, but some regions cannot yet be served, reducing the share of new users who can actually receive the service. How would you define and report the campaign result?
- Brand advertising, search, and affiliate channels reach overlapping prospects, while the attribution platform assigns most conversions to the last click. How would you estimate each channel’s incremental contribution and state the limits of the conclusion?
- A checkout experiment improves payment completion, but complaints and refunds also increase. How would you judge whether the experiment succeeded, and how long would you wait before deciding?
- A membership program increases repeat-purchase frequency, but average order value and margin both decline. How would you distinguish a healthy habit change from purchases pulled forward or low-quality, promotion-driven growth?
- Overall conversion improves, but new users and low-connectivity regions worsen. How would you examine segmentation, selection bias, and problems hidden by the aggregate?
- After several teams optimize the same north-star metric, duplicate content and low-quality interactions appear. How would you diagnose metric distortion and adjust guardrails?
- During a brand campaign, organic search and direct visits rise, while search-ad spend also increases. How would you estimate brand contribution without counting conversions intercepted by performance ads as brand outcomes?
- A campaign randomly holds out 10% of users, but they may still receive sales outreach and organic recommendations. Conversion is higher in treatment, and the channel team wants more budget. How would you assess contamination and results?
- Orders rise 30% during a discount campaign, but many come from existing customers who would have bought anyway, and product mix changes. How would you estimate incremental gross margin?
- The algorithm team expects an upgraded recommendation model to lift conversion by 3%, but operations would lose the ability to adjust key placements manually. How would you design an experiment and decision rules while accounting for value the experiment cannot measure?
- Retention rose during a marketing campaign, and the team credits the channel, but a new product flow also launched and participants may already have been more active. How would you establish a credible comparison, estimate the campaign’s net lift, and state the limits of the conclusion?
- How was your funnel built? Why are the steps divided the way they are, where does most of the drop-off happen, and how do you confirm the steps neither overlap nor leave gaps?
- A feature encourages invitations, so inviters and friends may land in different experiment groups. Results are significant, but spillover cannot be fully isolated. How would you assess the conclusion?
- Checkout, recommendations, and promotion teams run experiments at the same time and share traffic and revenue measures; results conflict and each team claims success. How would you identify interactions, interference, and resource conflicts?
- Overall retention rises after a redesign, but fewer new users and more established users are in the cohort; retention within each group is nearly unchanged. How would you assess the product's contribution and restructure the analysis?
- New pricing raises average revenue, but churn among small customers increases and high-value customers use discounts to avoid the increase. How would you analyze willingness to pay, segment costs, and customer alternatives?
- During a randomized experiment, the platform changes pricing policy and affects treatment and control differently. How would you decide which conclusions the experiment still supports?
- Brand advertising expands reach, but short-term direct conversions do not change and the sales cycle may last months. How would you decide whether to wait, change creative, or stop?
- A recommendation algorithm changes content ranking and creator supply, so treatment behavior changes what the control group can see. How would you assess the experiment design?
- A brand partnership produces broad reach, but sales cannot trace lead sources and short-term conversion is limited. How would you combine leading indicators, validation, and stop conditions?
No questions in this section match your target. ·
D · Delivery and ongoing operations21
- An onboarding experiment does not change overall signup-to-activation conversion, but activated users complete their first key task sooner. How would you decide whether to roll it out and revise the definition of success?
- During a major promotion, landing-page traffic meets expectations but add-to-cart and order rates are far below target. How would you quickly distinguish traffic quality, page information, pricing rules, inventory, and fulfillment issues?
- A limited-time promotion launches, but popular products run short. Some channels have already promised availability while others continue sending traffic. How would you adjust promotion, page promises, replenishment, and customer communication?
- A brand campaign has delivered its planned total impressions, but actual reach among the core target audience is below expectation, and one-third of the budget remains. How would you decide whether to continue, change the media mix, or revise the objective?
- A community event has strong registration and interaction numbers, but participants rarely continue using the product or contributing content afterward. How would you locate where the connection to product value breaks and design the next operating cycle?
- To meet a quarterly goal, a partner asks to expand promotion delivery at short notice. An initial check suggests the new audience may exceed agreed authorization and brand boundaries. How would you move the decision forward within the deadline?
- A campaign creative gets high engagement on social channels but almost no conversion in search or stores. Another version has ordinary engagement but stronger sales. How would you decide on creative iteration and channel allocation?
- A broad discount meets the overall sales target, but valuable existing customers view it as unfair, and some users delay purchases they would have made anyway. How would you adjust future audiences and offers?
- A discount briefly increases volume, competitors follow, and category prices and margins decline. How would you decide whether to keep matching, change the value proposition, or exit the promotion?
- A new product misses its first-week sales target, yet trial users show strong usage depth and referral intent. How would you determine whether the issue is reach, a longer purchase cycle, price threshold, or positioning, and decide what to invest in next?
- A new-customer offer generates many registrations, but most users do not complete the first key action and the discount cost has already been incurred. How would you plan follow-up and decide which users merit further investment?
- A channel has a low acquisition cost in a small test, but lead quality and conversion fall as spend increases. How would you validate the cost definition and true incrementality, distinguish channel saturation from audience or sales-capacity changes, and set a limit for additional spend?
- A campaign brings in many new users, but support, moderation, and plans for following up with users and helping them complete a key action are not ready. How would you design ownership and capacity before and after the campaign?
- Operations staff directly change customer states, and later no one can explain who changed what or why. How would you design for efficiency, permission, and auditability?
- Describe a re-engagement or referral campaign you contributed to as an engineer. Which systems, data, or delivery work did you own, how did you clarify budget and key user behavior with operations, and how did you verify incremental impact and your actual contribution?
- The promotion strategy is set and you are responsible for execution. What goals, rules, resources, risks, and acceptance criteria would you confirm before launch?
- You are about to run a growth experiment in production. What checkpoints and stop conditions do you prepare first?
- How would you move from one-off campaigns and acquisition toward automated, data-informed lifecycle operations? Describe a case where data tracking and algorithms turned manual work into repeatable rules.
- A two-sided platform has a local supply-demand imbalance: peak-time orders exceed available providers, or a new city has providers but too few users. How would you identify the main constraint and adjust pricing, incentives, or supply while controlling cost and long-term effects?
- An acquisition campaign over-delivers on some channels while assets fail to launch on others; Operations wants immediate normalization, but Growth needs a valid control. How would you coordinate containment, record deviations, and continue evaluation?
- Five channels have different levels of maturity, so their short-term returns are not directly comparable. How would you allocate baseline funding, exploration budget, and review dates?
No questions in this section match your target. ·