Meta Andromeda is a proprietary machine learning (ML) system designed by Meta to enhance ad retrieval in their recommendation system. This system utilizes cutting-edge AI and hardware innovations, including the NVIDIA Grace Hopper Superchip and Meta Training and Inference Accelerator (MTIA), to improve performance and return on investment for advertisers.
Key Features and Innovations
- Deep Neural Networks: Custom-designed for the NVIDIA Grace Hopper Superchip, these networks deliver superior performance by learning higher-order interactions from user and ad data. This has resulted in a +6% recall improvement and +8% ad quality improvement in Meta's retrieval system.
- Hierarchical Indexing: Supports the exponential growth of ad creatives, particularly from Advantage+ creative, by organizing ads into a hierarchical index. This reduces inference steps and improves precision and recall.
- AI Development Efficiency: Andromeda minimizes system complexity, allowing for end-to-end performance optimization and faster adoption of future AI innovations.
Challenges Addressed
- Volume of Ad Candidates: Andromeda processes a vast number of ads, leveraging predictive targeting and generative AI tools to handle the increased volume.
- Tight Latency Constraints: The system ensures rapid ad selection to maintain a smooth user experience, even with frequent updates to delivery and user interests.
Technological Advancements
- Model Elasticity: Enhances system ROI by allowing agile resource allocation and adjusting model complexity in real time.
- Optimized Retrieval Model: Utilizes low-latency, high-throughput GPU operators and advanced software pipelining techniques to improve feature extraction latency and throughput by over 100x compared to previous CPU-based components.
Andromeda significantly enhances Meta's ad system by optimizing personalization capabilities at the retrieval stage, improving return on ad spend, and leveraging the fast industry adoption of Advantage+ automation and generative AI. The system is expected to transition to support an autoregressive loss function, further improving inference efficiency and ad diversity. Integrating Andromeda with MTIA and future GPU generations aims to achieve another 1,000x increase in model complexity, pushing the boundaries of scaling retrieval and improving advertiser performance.
