Amazon's Selection Monitoring team is responsible for building and operating large-scale data processing systems that power catalog intelligence across Amazon. Our systems process billions of products from multiple data sources—both real-time crawler feeds and pre-computed datasets—to create comprehensive, high-quality product information that drives business decisions across Amazon. We apply parallel processing, machine learning, and deep learning algorithms to evaluate and enrich product data at global scale. The datasets and insights produced by our team enable teams across Amazon to make data-backed decisions that improve customer experience.

Our work involves building state-of-the-art data processing pipelines and Information Retrieval (IR) systems that handle diverse data streams and automatically create structured entities from unstructured/semi-structured data. We constantly stretch the boundaries of large-scale distributed systems, streaming and batch processing technologies, Elastic Computing, Big Data, Artificial Intelligence, and service-oriented architectures to tackle challenges at Amazon's global scale.

Come join us in our journey to build the data infrastructure that powers Amazon's catalog intelligence!
We are looking for SDEs with strong technical knowledge, established background in engineering large-scale software systems, and passion for solving challenging problems. The role demands a high-performing and flexible candidate who can take responsibility for system success and drive solutions from design to coding, testing, and deployment, to achieve results in a fast-paced environment.

The SDE on this team will play a key role in solving complex problems and building innovative solutions that process and integrate data from multiple sources at scale. Candidates need to have a high sense of ownership, ability to develop end-to-end solutions, and drive measurable impact. This role will expose you to solving problems of scale in the areas of data engineering, real-time and batch processing, observability, artificial intelligence, big data, and distributed systems of cloud-scale.

Key job responsibilities
As a member of the team, in this role, you will:

* Design and build scalable data processing systems that handle both real-time crawler data and precomputed datasets from multiple sources, ensuring seamless integration and orchestration across diverse data streams.
* Develop robust pipelines and frameworks for ingesting, processing, transforming, and enriching product data at scale using AWS technologies including streaming and batch processing platforms.
* Work with Senior Engineers to drive the technical and architectural vision for systems responsible for generating structured domain entities from structured/semi-structured data across multiple crawler sources.
* Build comprehensive observability and metrics systems to proactively identify bottlenecks, monitor system health, and provide accurate, actionable reporting that enables data-backed decision making.
* Solve complex problems in data integration, entity resolution, deduplication, and classification across heterogeneous data sources with varying quality and formats.
* Design and implement efficient storage solutions for managing large-scale datasets, ensuring optimal performance for both write-heavy ingestion and read-heavy analytical workloads.
* Lead and mentor engineers, and drive best practices around design, coding, testability, operational excellence, and security.
* Own end-to-end delivery of critical systems from requirements gathering through design, implementation, testing, deployment, and ongoing operational support.

Software Engineer II, Selection Monitoring