Senior Data Analytics Engineer

Overview MCAPS-Core accelerates customer outcomes and business growth. By uniting product, engineering, marketing, sales, customer success, and partners around a common customer mission, we help customers realize value faster and turn innovation into measurable business impact. Through deep technical expertise and differentiated go-to-market execution, we win competitively, capture market share, and scale repeatable growth motions. We differentiate through value creation - delivering world-class, AI-powered customer experiences that accelerate adoption, strengthen loyalty, and create sustainable competitive advantage, driving growth that outpaces the market. Within MCAPS - Core, the Customer Service & Support (CSS) organization builds trust and confidence for every person and organization through delivering a seamless support experience. In CSS, we are powered by Microsoft’s AI technology to help consumers, businesses, partners, and more, resolve their issues quickly and securely, helping prevent future problems from occurring and achieving more from their Microsoft investment. Strategy & Operations (S&O) serves as a cross-business unit within CSS, enabling strategic planning, governance, business performance, operational execution, and transformation across the organization. By aligning priorities, investments, AI and technology initiatives, and execution, we help CSS achieve its business objectives and deliver greater customer impact. Within S&O, CSS Intelligence & Applied AI builds the measurement science, applied AI, and decision systems that CSS uses to run the business. We are building an intelligence platform with three layers: certified measures and the pipelines that produce them, a detection engine that continuously scans production for quality and operational risk, and an executive knowledge layer that explains what the signals mean and answers questions in natural language. Every one of those layers rests on data engineering that has to be correct, fast, governed, and always on. We are seeking a Senior Data Analytics Engineer to build and own that foundation. You will design the real-time and batch pipelines, the semantic models, and the serving infrastructure that our detection and intelligence products run on. This is a platform role, not a reporting role. The measures you certify become the single answer that an alert, an executive brief, a monthly review deck, and an AI agent all resolve to, so the standard is high: one metric, one definition, one query path, with the freshness and lineage to prove it. If you have wanted to build the data foundation for AI-native decision systems rather than maintain another dashboard estate, this is that role. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities You will design and operate the data pipelines that feed CSS intelligence, spanning both batch processing at organizational volume and near real-time streams where detection latency matters. That includes ingesting case and agent telemetry, integrating with the unified data platform and Finance systems, and building the scoring pipelines that run our data science models continuously in production rather than in a notebook. You will be accountable for the reliability of those pipelines: monitoring, alerting on your own infrastructure, handling schema drift and late-arriving data, and making sure a failure surfaces to you before it surfaces to a leader reading a brief. You will build and own the semantic layer that sits between raw data and every consuming experience. This means dimensional models designed for the questions leaders actually ask, certified measures with documented definitions and clear ownership, consistent hierarchies across organization, product, offering, severity, and geography, and the security context that governs who sees what. Critically, you will design this layer to serve more than reports: our detection engine queries it for baselines and thresholds, and our executive experience queries it through a semantic API so that natural-language answers resolve to governed measures with a traceable query path rather than to improvised calculations. You will make trust in the platform mechanical rather than aspirational. You will build data quality and observability into the pipelines themselves, including freshness stamps, reconciliation tests against sources of record, contract checks between producers and consumers, and lineage that lets anyone trace a number on a slide back to the rows behind it. You will bring engineering discipline to analytics through source control, deployment pipelines, environment separation, and infrastructure defined as code, and you will manage platform cost and performance deliberately through partitioning, incremental refresh, and query optimization. You will work in close partnership with data scientists and applied AI engineers, turning experimental models into production services and preparing data for AI consumption, including the grounding, retrieval, and feature-serving patterns our agents depend on. You will also design for handoff: our intent is that patterns proven inside CSS can be adopted and operated more broadly, so schemas, APIs, deployment approaches, and documentation need to be legible to teams who did not build them. Core Skills & Experiences the ideal candidate will possess: • Deep SQL expertise and strong programming skills in Python, including experience with distributed processing frameworks such as Spark or PySpark. • Experience building production data pipelines in both batch and streaming or near real-time modes, using platforms such as Microsoft Fabric, Azure Data Factory, Synapse, Databricks, or equivalent. • Strong dimensional and semantic modeling skills, including star schema design, tabular or semantic models, calculation languages such as DAX, and the discipline of certified, documented, single-definition measures. • Experience designing layered data architectures that separate raw, conformed, and serving tiers, and knowing which transformations belong in which tier. • Experience exposing data and measures programmatically through APIs or query services for consumption by applications, services, or AI agents, not only by reporting tools. • Experience implementing data quality and observability practices, including reconciliation testing, data contracts, freshness and completeness monitoring, anomaly detection on pipelines, and end-to-end lineage. • Experience applying software engineering discipline to data work, including source control, CI/CD and deployment pipelines, environment separation, code review, and infrastructure as code. • Experience optimizing platform cost and performance through partitioning, incremental refresh, aggregation strategy, capacity management, and query tuning. • Familiarity with preparing data for AI and machine learning consumption, such as feature pipelines for model scoring, grounding and retrieval patterns for language models, embedding or vector stores, and ingestion of agent telemetry including identifiers, token usage, and evaluation output. • Experience partnering with data scientists to productionize models, and with business stakeholders to translate an operational question into a durable data design. • A platform mindset: a bias toward building reusable, documented capability rather than one-off extracts, and the judgment to know when a governed measure already answers the question. • Comfort operating with autonomy on a small team, owning a component end to end, and writing the documentation that lets someone else run it. Examples of work in this role may include: • Building the certifi