AI engineering

• An AI assistant can create tickets, issue refunds, or change accounts on a user's behalf. A refund request times out, and the system cannot tell whether it completed; retrying may issue a second refund. How would you check the outcome, control retries, and ensure actions stay within the user's authorization?AI engineering
• Two reviewers often disagree about the same set of AI outputs. How would you tell whether the issue is annotator disagreement, an unclear evaluation standard, or model instability?AI engineering
• You built a customer-support Q&A prototype. How would you design an evaluation set to decide whether it may only assist retrieval, or can reply to customers directly?AI engineering
• What does an algorithm model and the features it depends on truly cost? How do you count training, inference, feature computation and storage, labeling, and maintenance, and how do you judge whether a feature is worth keeping?AI engineering
• A document assistant can submit most cases automatically, but mistakes have very different consequences. Human review has reduced errors, but the queue keeps growing. How would you decide which cases to automate or review, and keep the queue within the team’s capacity?AI engineering
• A generative feature produces good answers, but peak latency and per-request cost exceed the product budget. How would you determine whether the cause is the model, context, retrieval, or call chain?AI engineering
• After a foundation-model change, the average score improves, but outcomes worsen for a few important customers. How would you decide whether to ship, stage, or roll back?AI engineering
• Your company wants an internal knowledge assistant, and document permissions change as people move around. How would you make it answer with evidence while keeping it from exposing what a user should not see?AI engineering
• A resume line saying "used AI to improve engineering efficiency" carries little information. How would you rewrite it into an experience others can probe and verify?AI engineering
• User satisfaction with a knowledge assistant has fallen, but offline retrieval metrics are unchanged. Which parts of the system would you investigate?AI engineering
• You ask an AI to review a configuration or generate a piece of code. How do you decide whether it can go live?AI engineering
• Enterprise knowledge documents may change, be withdrawn, or become restricted, yet users still receive answers citing old content. How would you propagate changes from source documents through chunks, indexes, caches, and existing citations, and prove that an update or deletion took effect?AI engineering
• Production answers contain incorrect citations or degrade in quality, but prompts and retrieval context may include sensitive data and cannot be logged by default. What telemetry would help distinguish retrieval, model, and tool failures while controlling data exposure?AI engineering
• A new model raises overall accuracy from 91% to 94%, but errors remain concentrated in cases that affect customer rights. The team wants to ship based on the aggregate score. How would you decide?AI engineering
• A RAG assistant cites an internal document, but its owner has updated the process and the new version is not indexed yet. How would you define freshness goals, propagation, and answer behavior during the gap?AI engineering
• A multi-step agent reads project permissions at startup. Before step three, the user is removed from the project, but the task still holds a tool token. How would you design authorization checks and cancellation semantics?AI engineering
• In the technical area you work in, how is the industry developing? Is your solution adapting to that trend or going against it?Engineering systems
• How would you design a product where a person and an AI write together? How are the AI's edits shown, accepted, or rejected, and how are conflicts handled when both edit at once?System design
• How would you design a large-model chat system? How do you handle streamed output, multi-turn context, history, interruption, and retries, and how do you control cost and latency?System design
• How do the costs of a recommender system add up, including retrieval, ranking, feature serving, storage, and experimentation? How would you trade effect against cost?System design
• How would you design a real-time risk control system? How do rules, features, models, and manual review work together, and how do you trade false blocks against misses?System design
• How would you design a search system? How do indexing, retrieval, and ranking divide the work, how are Chinese segmentation and features built, and how do you judge how good the search is?System design
• Public benchmarks keep improving while internal task performance stalls. How would you investigate evaluation distortion?AI evaluation and governance
• The product wants confident AI conclusions, but evidence supports only probabilistic judgments. How would you design communication and traceability?AI evaluation and governance
• After AI output causes actual user harm, product, model, operations, and review teams all take part in the response. How would you divide incident handling and follow-up improvement?AI evaluation and governance
• After replacing a foundation model, overall metrics remain similar but some users report worse results. How would you identify what changed in model behavior and decide whether to roll back?AI evaluation and governance
• You need to sample production AI conversations for emerging risks, but they contain personal or commercial data. How would you design the review?AI evaluation and governance
• A third-party model provider changes its training, retention, or cross-border terms. When should calls pause or receive renewed approval?AI evaluation and governance
• Average AI quality meets the target, but a few high-risk errors can cause harm. How would you set the release gate?AI evaluation and governance
• An AI assistant can call internal tools, but tasks require different permissions. How would you design authorization and audit?AI evaluation and governance
• When the model cannot establish an answer, should the product refuse, return a low-confidence result, or escalate to a person? What distinguishes the choices?AI evaluation and governance
• Overall AI accuracy passes, but one language or user segment has a higher error rate. How would you assess and respond?AI evaluation and governance
• After changing an embedding model, old and new vectors are not directly comparable, while a full rebuild takes days. How would you design migration and query cutover?AI engineering
• A team discovers that a model may have seen some internal evaluation prompts in public corpora or fine-tuning data. How would you determine what offline scores still mean and rebuild a trustworthy evaluation?AI engineering
• A team fine-tunes on user feedback, human labels, and synthetic samples, then receives a request to delete some source data. How would you trace impact and decide the scope of retraining?AI engineering
• A new model improves offline evaluation, but answer quality may fall for a small set of high-value users. How would you stage a canary, observe it, and decide when to expand traffic?AI engineering
• A product depends on an external model API, and the provider may restrict request frequency or change available models. How would you design switch rules and temporary shutdowns for noncritical features so quality changes after switching are visible?AI engineering
• Users say answers sound plausible but omit a critical constraint present in the documents. How would you locate whether parsing, chunking, retrieval, ranking, or generation caused the failure?AI engineering
• Offline evaluations keep improving while a specific user segment regresses online. Which data leakage, slice coverage, and deployment differences would you investigate?AI engineering
• An AI assistant repeatedly calls search, databases, and external APIs for a small share of requests, making latency and cost far higher than average. How would you design per-task budgets, stop conditions, idempotency, and auditable execution records?AI engineering
• A new model scores much better on an offline test set, but a small production rollout shows lower task completion for one user group and more complaints. How would you determine where the evaluation set, metrics, or release decision failed?AI evaluation and governance
• An AI assistant retrieves documents from several business systems, where users have different permissions in each. How would you ensure retrieval, caching, summaries, and citations follow source permissions, including after access is revoked?AI engineering
• A model service used by an AI feature has highly variable latency, causing user requests to exceed the end-to-end deadline. How would you allocate budgets across retrieval, model calls, and post-processing, and design fallbacks after a timeout?AI engineering
• A user submits a long AI analysis, closes the page, and later checks from another device; the task may call several tools and produce partial results. How would you design task state, cancellation, retry, and result visibility?AI engineering
• An analytics assistant converts natural language into SQL against business data; a request may induce it to read sensitive columns or run an expensive query. How would you design read-only permissions, query validation, resource limits, and result redaction?AI engineering

No questions in this section match your target. Change target · View all questions

• An online feature must refresh every second, but it affects few requests and its maintenance cost keeps rising. How would you decide whether to keep it real time?Feature model
• A business rule is corrected retroactively. Reports need corrected results, while decision-makers also need the results available at the time. How would you manage both historical views?Model evolution
• A risk value is usually present in training data, but live serving often substitutes a default and model performance fluctuates. How would you find whether the cause is data collection, timing, missing-value handling, or implementation differences?Feature model

No questions in this section match your target. Change target · View all questions

• The algorithm team expects an upgraded recommendation model to lift conversion by 3%, but operations would lose the ability to adjust key placements manually. How would you design an experiment and decision rules while accounting for value the experiment cannot measure?Metrics and causal reasoning
• Drawing on available material and direct observation, which kinds of companies do you think this technology will affect most over the next two years? Explain your own judgment and evidence rather than only repeating industry reports. What does it mean for your company, and what have you done to test or respond to it?Product judgment

No questions in this section match your target. Change target · View all questions