The role
From Google's own posting.
In Google Search, we're reimagining what it means to search for information – any way and anywhere. To do that, we need to solve complex engineering challenges and expand our infrastructure, while maintaining a universally accessible and useful experience that people around the world rely on.
In this role, you will work cross-functionally with leads from Engineering, Product, UX, and Research to shape how AI-powered experiences in Search are measured and improved. As AI systems across the industry become more capable at reasoning through complex, multi-step goals, the question of what "success" means for a search journey is evolving, and you will help define that answer: by developing new metrics and evaluation approaches, designing experiments for complex user flows, and translating findings into product direction. Your work will directly inform the future of Google Search and AI Mode, applying techniques like LLM-based evaluation and causal inference to some of the most consequential product questions at Google.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.
US: $138000 - $197000 (USD) + 15% bonus target + equity + benefits
Learn more about benefits at Google .
Frame and sol ve ambiguous product problems and communicate the results clearly to both technical and non-technical audiences.
Work cross-functionally with Product, Engineering, UX, and Research partners. Act as a thought partner and influence prioritization with data.
Define success metrics and quality frameworks for multi-step and conversational AI experiences in Search, spanning task completion, efficiency, trust, and user satisfaction.
Analyze human and au to-rater evaluations of AI experiences alongside behavioral signals to build a holistic view of product quality.
Design and assess live experiments to understand the impact of new AI features on user journeys, and make clear launch recommendations.
Minimum qualifications:
Bachelor's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
5 years of experience with analysis applications (e.g., extracting insights, performing statistical analysis, or solving business problems), and coding (e.g., Python, R, SQL) or 2 years of experience with a Master's degree.
Preferred qualifications:
Master's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
5 years of experience with analysis applications (e.g., extracting insights, performing statistical analysis, or solving business problems), and coding (e.g., Python, R, SQL).
Bachelor's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
5 years of experience with analysis applications (e.g., extracting insights, performing statistical analysis, or solving business problems), and coding (e.g., Python, R, SQL) or 2 years of experience with a Master's degree.