Machine Learning Engineer interview questions
A machine learning engineer loop tests two jobs at once: can you write production software, and do you understand models well enough to keep them honest. Most candidates prepare hard for one half and get caught on the other. Here's what each round is checking and how to answer the questions that come up again and again.
What happens in each round
- 1
Recruiter screen
What happens
Whether your background leans toward modeling or infrastructure, and whether that matches the team. Be ready to say in plain words which models you've put into production and what they did.
- 2
Coding round
What happens
Standard data structures and algorithms in Python, sometimes with a data-flavored twist like parsing logs or computing a running metric. They want clean, tested code, not tricks.
- 3
Machine learning fundamentals
What happens
Bias and variance, overfitting, evaluation metrics, class imbalance, and how common models behave. Expect follow-ups that push past textbook definitions into what you'd actually do.
- 4
ML system design
What happens
Whether you can design a whole system, such as a feed ranker or a fraud score, from data collection through training, serving, monitoring and retraining. This round usually decides the level you're hired at.
- 5
Behavioral and project walkthrough
What happens
A past project told in detail: your part, the tradeoffs, what broke and what you'd change. Interviewers listen for ownership and for honesty about models that didn't work.
Questions you're likely to get
1.Design a system that recommends items on an app's home screen.
Why they ask
It's the classic ML system design prompt. It shows whether you think in whole systems or jump straight to a model.
How to answer
- Start by pinning down the goal and the metric: clicks, watch time or purchases, and what the business actually cares about
- Split it into candidate retrieval and ranking, and explain why a single model can't score every item on every request
- Name the features, where they come from, and which ones must be fresh at request time
- Cover serving latency, caching, and a fallback when the model is down
- Close with monitoring, retraining cadence and how you'd A/B test a new version
2.Your model scored well offline but made things worse in an A/B test. What do you check?
Why they ask
This happens constantly in the real job. They want a debugging order, not a guess.
How to answer
- Confirm the test itself is sound: randomization, sample ratio, and that the right model version was actually served
- Compare feature values in training against what the live service computed for the same users
- Check whether the offline metric matches the online goal, since better ranking accuracy can still hurt engagement
- Look for leakage in the training data that made offline results look too good
- Slice the results by user segment to see where the loss comes from
3.How would you detect that a model in production is getting worse?
Why they ask
Monitoring is a big part of what separates this role from data science. Labels often arrive late, which makes it hard.
How to answer
- Separate system health, like latency and error rates, from model health
- Track input drift on key features and shifts in the prediction distribution, since those show up before labels do
- Measure real performance once delayed labels arrive, and say how long that lag is for this problem
- Set alert thresholds that page a human only when action is needed
4.What's training-serving skew, and how do you prevent it?
Why they ask
It's one of the most common reasons models fail after launch, and people who've shipped models have usually been burned by it.
How to answer
- Define it: the features the model sees live differ from the ones it trained on
- Give causes such as duplicated feature logic in SQL and in service code, or time-travel leakage in training joins
- Prevent it with shared feature code or a feature store, and point-in-time correct joins
- Log the features actually served and compare them to training data
5.You have a fraud dataset where almost every example is legitimate. How do you train and evaluate?
Why they ask
Imbalanced data shows up in fraud, ads and medical use cases. It tests whether you pick metrics on purpose.
How to answer
- Explain why accuracy is useless here and pick precision, recall and the precision-recall curve instead
- Discuss class weights, resampling, or simply choosing the decision threshold with the business
- Tie the threshold to the cost of a missed fraud versus a blocked good customer
- Split the data by time so the test set looks like the future
6.How would you cut inference latency in half without losing much quality?
Why they ask
Serving cost and speed are the engineer's problem, not the scientist's. They want practical options with tradeoffs.
How to answer
- Profile first to find whether time goes to feature lookups, the network or the model itself
- Mention model-side options like quantization, distillation into a smaller model, or exporting to ONNX
- Mention system options like batching requests, caching predictions and precomputing features
- Say how you'd verify quality held up before rolling out
7.Walk me through how you'd set up retraining for a model that goes stale.
Why they ask
It checks whether you've built a real training pipeline or only trained once in a notebook.
How to answer
- Choose scheduled or triggered retraining and justify it from how fast the data changes
- Describe the pipeline in an orchestrator like Airflow or Kubeflow, with data validation up front
- Gate promotion on an evaluation set and compare against the current model before swapping
- Keep versions in a model registry so rollback is one step
8.Explain the bias-variance tradeoff with an example from your own work.
Why they ask
Every candidate can recite the definition. Asking for your own example shows whether you've actually felt it.
How to answer
- Give a short, correct definition in your own words
- Describe a model you built that overfit or underfit, and what the learning curves showed
- Say what you changed, such as regularization, more data or a simpler model, and what happened
9.Write a function that computes a moving average over a stream of numbers.
Why they ask
A typical coding-round task with a data angle. It tests clean code and whether you think about memory and edge cases.
How to answer
- Ask whether the window is fixed and what happens before it fills
- Use a queue and a running sum so each update is constant time
- Handle empty input and write a quick test out loud
- Mention how it would change for a time-based window
10.Tell me about a model you shipped that didn't work the way you expected.
Why they ask
Things break often in this job. They want to hear how you find out, what you own and what you learn.
How to answer
- Set up the project in a sentence or two, then get to the surprise
- Explain how you found the problem, ideally through monitoring you'd built
- Be specific about your own part in the fix
- Say what you changed in the process afterward so it wouldn't repeat
11.How do you work with data scientists who hand you models to deploy?
Why they ask
The handoff is where many teams struggle. They're checking whether you'll be a partner or a gatekeeper.
How to answer
- Describe getting involved early, before the notebook is finished, to flag features that can't be served
- Mention shared templates, packaging standards or a common training library that makes handoff easy
- Show respect for their modeling calls while being clear about production limits
Mistakes that sink good candidates
Talking only about model accuracy and never about latency, cost or monitoring
Listing every framework you've touched instead of explaining one system you built in depth
Claiming a model improved results without saying how you measured it
Freezing on the coding round because you prepared only for machine learning theory
Need more Machine Learning Engineer interviews to prep for?
HeroApply applies to Machine Learning Engineer jobs that match you, every day. 1,003 jobs are open today.