Your GPUs aren't the problem. Your storage might be.
Plus: reliable AI agents, MCP, database agents, and the latest infrastructure breakthroughs.
MongoDB Atlas gives BI teams a flexible, scalable cloud database with built-in search, vector search, and real-time analytics — no schema migrations, no data silos.
Stop forcing your data into rigid tables. MongoDB Atlas is the developer data platform built for modern analytics — flexible documents, multi-cloud on AWS, and Atlas Search out of the box. Free tier available on AWS Marketplace.
👋 Hi there, welcome to DataPro #181 – Building the infrastructure behind production AI
AI is entering a new engineering phase. The focus is no longer just on building smarter models, but on creating the infrastructure, storage, and operational systems that make AI reliable at scale. This edition explores the technologies powering production AI, from next-generation storage architectures and autonomous database operations to coding agents, multi-agent systems, and cloud-native AI platforms.
🌟 Community Spotlight
AI Data Storage: Challenges, Capabilities, and Comparative Analysis
As AI workloads continue to scale, storage is emerging as one of the biggest infrastructure bottlenecks. In this deep dive, JuiceFS explores why traditional storage architectures struggle with GPU-intensive training, billions of small files, and high-throughput data access, before comparing the capabilities modern AI storage platforms need to support today’s data pipelines.
🎙️ Expert Perspective
The Reliability Gap Holding AI Agents Back
What separates an impressive AI demo from a production-ready AI system? AI researcher and Developer Advocate Sonam Gupta shares the engineering practices behind building reliable AI agents, explaining why evaluation, observability, guardrails, and system design matter far more than model benchmarks when deploying AI at scale.
⚡ This Week’s Highlights
Behind the scenes of Google Agent Skills
Prime Agent introduces a self-improving coding harness
How LendingTree built a production multi-agent mortgage assistant
Google’s latest AI infrastructure updates
Meta Muse Code enters beta
Mobileye automates enterprise support with AgentCore
Google’s Borderless Lakehouse expands multi-cloud AI
AWS explains its MCP bridge architecture
NVIDIA launches Alpamayo 2 Super for autonomous driving
Google introduces Database Operations Agents
Happy reading!
Cheers,
Merlyn Shelley,
Growth Lead, Packt.
AI Data Storage: Challenges, Capabilities, and Comparative Analysis
Over the past five years, AI has advanced rapidly and has found applications in a wide range of industries. As a storage company, we’ve had a front-row seat to this expansion, watching more and more AI startups and established players emerge across fields like autonomous driving, computational biology, and quantitative investment. AI workloads have introduced new challenges to the field of data storage. Existing storage solutions are often inadequate to fully meet these demands.
In this article, we’ll deep dive into the storage challenges posed by AI workloads, the critical capabilities required to address them, and a comparative analysis of the leading storage solutions spanning cloud and on-premises environments.
Storage challenges for AI
AI workloads have brought new data patterns:
High-throughput data access challenges: In AI training pipelines, the growing use of GPUs by enterprises has outpaced the I/O capabilities of underlying storage systems. Enterprises require storage solutions that can provide high-throughput data access to fully leverage the computing power of GPUs. For instance, in smart manufacturing, where high-precision cameras capture images for defect detection models, the training dataset may consist of only 10,000 to 20,000 high-resolution images. However, each image is several gigabytes in size, resulting in a total dataset size in the scale of tens or hundreds of terabytes. If the storage system lacks the required throughput, it becomes a bottleneck during GPU-intensive AI training.
Managing storage for billions of files: AI use cases need storage solutions that can handle and provide quick access to datasets containing billions of files. For example, in autonomous driving, the training dataset consists of small images, each about several hundred kilobytes in size. A single training set comprises tens of millions of such images, each sized several hundred kilobytes. Each image is treated as an individual file. The total training data amounts to billions or even 10 billion files. This creates a major challenge in effectively managing large numbers of small files.
Scalable performance for hot data: In areas like quantitative investing, financial market data is much smaller in size compared to computer vision datasets. However, this data must be shared among many research teams, leading to hotspots where disk throughput is fully saturated but still cannot satisfy the application’s needs. This indicates that we need storage solutions that can steadily serve a lot of hot data with high throughput and low latency.
The infrastructure landscape for AI has also changed dramatically.
These days, with cloud computing and Kubernetes getting so popular, more and more AI companies are setting up their data pipelines on Kubernetes-based platforms. Algorithm engineers request resources on the platform, write code in Jupyter Notebook to debug algorithms, use workflow engines like Argo and Airflow to plan data processing workflows, use Fluid to manage datasets, and use BentoML to deploy models as applications. Cloud-native technologies have become a standard consideration when building storage platforms. As cloud computing matures, AI applications are increasingly relying on large-scale distributed clusters. With a significant increase in the number of nodes in these clusters, storage systems face new challenges related to handling concurrent access from tens of thousands of pods within Kubernetes clusters.
The evolving application workloads and computing environments significantly change the landscape for professionals managing the underlying infrastructure and platforms. Existing hardware-software-coupled storage solutions often suffer from several pain points, such as no elasticity, no distributed high availability, and constraints on cluster scalability. Distributed file systems like GlusterFS, CephFS, and those designed for high-performance computing (HPC) such as Lustre, BeeGFS, and GPFS are typically designed for physical machines and bare-metal disks. While they can be deployed as large capacity clusters, they cannot provide elastic capacity and flexible throughput, especially when dealing with storage demands in the order of tens or hundreds of billions of files.
Key capabilities for AI data storage
Considering these challenges, we’ll outline essential storage capabilities that are critical for AI scenarios, helping enterprises make informed decisions when selecting storage products.
POSIX compatibility and data consistency
In the AI/ML domain, POSIX is the most common API for data access. Previous-generation distributed file systems, except HDFS, are also POSIX-compatible, but products on the cloud have not been consistent in terms of their POSIX support:
Compatibility: Users should not solely rely on the description “POSIX-compatible product” to assess compatibility. Instead, use
pjdfstestand the Linux Test Project (LTP) framework for testing. We’ve conducted a POSIX compatibility test of cloud file systems for your reference.Strong data consistency guarantee: This is fundamental to ensuring computational correctness. Storage systems have various consistency implementations, with object storage systems often adopting eventual consistency, while file systems typically adhere to strong consistency. Careful evaluation is needed when selecting a storage system.
User mode or kernel mode: Early developers favored kernel mode due to its potential for optimized I/O operations. However, in recent years, we’ve witnessed a growing number of developers “escaping” from kernel mode for several reasons
The Reliability Gap Holding AI Agents Back
AI researcher and Developer Advocate Sonam Gupta explores the engineering practices that transform AI agents from impressive demos into dependable production systems.
Every AI demo looks impressive.
Until it meets a real user.
For the last two years, the AI industry has celebrated increasingly capable foundation models, autonomous agents, and sophisticated reasoning systems. Every product launch promises more intelligence. Every benchmark claims higher accuracy. Every new release inches us closer to what once felt impossible.
Yet behind the excitement lies a quieter reality that every engineering team eventually encounters.
The hardest part isn’t getting an AI system to work once.
It’s getting it to work consistently, reliably, and at production scale.
That challenge is quickly becoming one of the defining engineering problems of the Generative AI era. As AI applications evolve from experimental prototypes into products that millions depend on, success is no longer measured by how intelligent a model appears in a carefully orchestrated demo. It’s measured by how predictably an entire system performs when confronted with the messy, unpredictable behaviour of real users.
At Packt, we believe these are the conversations that deserve more attention.
That is why Sanjana Gupta, Relationship Lead at Packt, sat down with Sonam Gupta, a data scientist, Developer Advocate at Telnyx, and host of the AI Chronicles podcast, for an in-depth discussion on what it really takes to build reliable AI systems. Bringing together her experience of working closely with AI authors, practitioners, and technical experts across Packt’s learning ecosystem, Sanjana steered the conversation beyond the usual questions about models and benchmarks, focusing instead on the engineering realities that practitioners face every day.
As I listened to their discussion, one thing became immediately clear.
This wasn’t another conversation about the latest LLM.
It was about something far more important.
Join Stefan Jansen, best-selling author of Machine Learning for Trading, for a hands-on workshop where you’ll build a complete ML trading strategy using real market data and AI agents. Learn the end-to-end workflow used by professional quantitative teams—from feature engineering to backtesting.
Why do so many AI applications fail after the demo, and what separates production-ready AI systems from prototypes that never earn users’ trust?
Over the next thirty minutes, Sonam unpacked lessons from building AI agents, evaluating LLM workflows, deploying voice applications, and interviewing leaders across companies such as Google DeepMind, Microsoft, AWS, and emerging AI startups. Rather than offering theoretical advice, she shared the engineering mindset that modern AI teams need if they want their systems to perform reliably in the real world.What follows is not simply an interview.
It’s a masterclass on why the future of AI belongs not only to those who build intelligent systems, but to those who build systems people can trust.
Join Ben Auffarth to build a production-ready RAG application using open-source models. Learn how to improve retrieval, benchmark performance with RAGAS, add guardrails, and deploy reliable AI systems without expensive APIs.
The industry is asking the wrong question.
If you’ve attended an AI conference recently, you’ve probably noticed a familiar pattern.
Every discussion eventually circles back to the same debate.
Which model is better?
GPT or Gemini?
Open-source or closed-source?
Should developers switch frameworks?
Will AI agents replace traditional software?
These questions dominate keynote stages, LinkedIn discussions, and engineering forums. They’re important, certainly, but they also risk distracting us from a much larger engineering challenge.
As Sanjana opened the conversation, she didn’t ask Sonam which model she preferred.
Instead, she began somewhere much more fundamental.
Before discussing AI agents, evaluation frameworks, or production systems, she wanted readers to understand the perspective from which Sonam approaches the AI ecosystem itself.
Data Science & ML Research Roundup
⭕ Behind the scenes: How we build, test, and scale Google Agent Skills: Google has revealed how it built and governs Google Agent Skills, the open-source framework that gives AI coding agents structured domain knowledge to improve accuracy, reduce hallucinations, and follow best practices. The post explains how the project scaled from a Google Cloud Next 2026 initiative into a 15,000+ star GitHub repository, backed by standardized skill design, automated quality checks, continuous evaluations, clear ownership, and internal DevRel Skills that help teams automate and standardize workflows.
⭕ Prime Intellect Releases Prime Agent: An Open-Source RLM Harness Where Sub-Agents Are Function Calls Inside Persistent IPython Kernel: Prime Intellect has open-sourced Prime Agent, a self-improving AI coding harness that replaces fixed tool schemas with a persistent IPython kernel, allowing sub-agents to operate as function calls and continuously refine their own prompts, skills, and memory. The release details its deployable architecture, broad model compatibility, benchmark results, including 95.5% on ARC-AGI-3, and real-world use cases spanning software engineering, GPU optimization, AI research, and long-running autonomous coding tasks.
⭕ How LendingTree built a multi-agent mortgage assistant on Amazon Bedrock: LendingTree has shared how it built a production-ready multi-agent mortgage assistant on Amazon Bedrock to help borrowers navigate complex home financing decisions with personalized, compliant AI guidance. The post details its three-agent architecture, built with LangGraph, MCP, Amazon Bedrock Guardrails, and Knowledge Bases, along with deployment lessons, production metrics, and best practices for scaling secure, multi-agent systems in regulated industries.
⭕ What’s new in AI infrastructure this month: Google has published its latest AI infrastructure and orchestration roundup, highlighting new products, performance optimizations, and deployment guides for building and scaling AI and agentic workloads on Google Cloud. The update covers infrastructure launches such as Managed Lustre, C4N VMs, GKE enhancements, TPU tooling, AI security projects, customer deployments, and research showing how Google is improving AI performance, efficiency, and production readiness across its cloud platform.
⭕ Meta AI Releases Muse Code (Beta): A Terminal Coding Agent Powered by the New Muse Spark 1.2 Model: Meta AI has released Muse Code (beta), a terminal-based coding agent powered by the new Muse Spark 1.2 model, designed to tackle complex, long-running software engineering tasks across large codebases. The announcement outlines its persistent async agent architecture, replay-safe runtime, bundled planning skills, co-training approach, benchmark methodology, and real-world GPU kernel optimization case study, positioning it as a production-ready coding assistant for enterprise development workflows.
⭕ How Mobileye transformed support operations using Amazon Bedrock AgentCore: Mobileye has shared how it transformed its internal support operations with an AI Support Agent built on Amazon Bedrock AgentCore, reducing ticket response times by 90% while exceeding 95% accuracy. The post explains its hybrid production architecture, MCP-powered real-time data access, enterprise governance, and how the successful deployment evolved into a self-service platform that enables teams across the company to build and deploy production-grade AI agents.
⭕ Introducing the borderless Lakehouse: Google has unveiled major enhancements to its borderless Lakehouse, enabling AI agents to securely query, reason over, and act on data across on-premises systems, multiple clouds, and SaaS platforms without moving it. The announcement introduces Iceberg REST catalog federation, zero-copy cross-cloud analytics, Knowledge Catalog for trusted agent context, Gemini Enterprise integration, and cost optimizations that simplify building governed, multi-cloud AI agents while reducing data transfer and token costs.
⭕ How we built an MCP bridge to give our AgentCore-hosted AI agent access to local MCP tools: AWS has shared how it built an MCP bridge that lets cloud-hosted AI agents securely use tools and files running on a user’s local machine. The post explains the architecture behind the bridge, including AgentCore Runtime, browser extensions, WebSockets, native messaging, FastMCP, and stdio, showing how remote agents can work with local spreadsheets and systems without exposing credentials or requiring complex infrastructure.
⭕ NVIDIA Releases Alpamayo 2 Super: A 34B Open Vision-Language-Action Model for Robotaxis and Autonomous Driving Under OpenMDW-1.1: NVIDIA has released Alpamayo 2 Super, a 34B open vision-language-action model designed to help autonomous vehicles handle rare, complex driving situations. The release combines full-surround video understanding, trajectory planning, causal explanations, and meta-actions in one system, while introducing commercially usable weights, strong benchmark results, safety-focused reasoning traces, and tools for accelerating fleet-data annotation and autonomous-driving development.
⭕ Deep dive on new AI-powered database agents: Google has introduced Database Operations Agents, a pair of AI-powered assistants that automate database onboarding, monitoring, troubleshooting, and optimization across Google Cloud. The announcement details how the Database Onboarding Agent simplifies database selection and deployment, while the Database Observability Agent uses Gemini, telemetry, and MCP tools to diagnose issues, recommend remediations, and streamline database management across services such as Cloud SQL, Spanner, AlloyDB, and Bigtable.
See you next time!