The Missing Layer in Enterprise AI
Lessons from H-E-B's Data AI Product Manager on AI Product Analytics, SQL, Python, user behavior, and enterprise decision-making.
This NINJIO Insights Report dives into the key emotional susceptibilities that make social engineering work and offers concrete steps that your security team can take to equip your workforce to resist cyberattacks.
👋 Hi there, welcome to DataPro #177 – Special Weekend Edition.
The AI conversation has become increasingly dominated by models, benchmarks, and agents. Every week brings another breakthrough, another framework, or another promise of autonomous intelligence. Yet inside enterprise teams, a far more practical question continues to surface: How do you know whether your AI product is actually delivering value?
Building an AI application is only the beginning. Understanding how users interact with it, where they lose trust, what drives adoption, and whether it improves real business decisions is quickly becoming one of the most valuable skills for data professionals. It’s a discipline where data engineering, analytics, product thinking, SQL, Python, experimentation, and AI all converge.
That’s exactly what we’re exploring in this special weekend edition. In the latest Packt Talks session, hosted by Abhishek Kaushik, Prithvi Shivshankar, Data AI Product Manager at H-E-B, takes us inside the emerging world of AI Product Analytics—showing how modern teams instrument AI products, analyze user journeys, measure product adoption, and use behavioral data to continuously improve AI-powered experiences. Drawing on real enterprise examples, he walks through why successful AI products aren’t built around models alone, but around better decisions.
If you’re building AI applications, working with data products, or simply curious about what happens after an AI product reaches production, this is one read that’s well worth your weekend.
Happy reading!
Cheers,
Merlyn Shelley,
Growth Lead, Packt.
Why Every AI Product Needs Better Analytics Before Better Models
How SQL, Python, product analytics, and user behavior are becoming the foundation of successful AI products
Every AI product generates outputs.
Far fewer generate outcomes.
As enterprises race to embed generative AI into applications, copilots, and decision-support systems, the conversation has largely centered around models, prompts, reasoning capabilities, and agents. Yet once these products reach production, many organizations encounter a very different challenge.
Are people actually using them?
Do users trust the recommendations?
Where do they abandon the workflow?
More importantly, did the AI improve a business decision—or simply become another dashboard?
That’s exactly what the latest episode of the Packt Talks YouTube series, hosted by Abhishek Kaushik, set out to answer. Prithvi Shivshankar, Data AI Product Manager at H-E-B, walked viewers through the emerging discipline of AI Product Analytics, demonstrating how modern product teams combine SQL, Python, behavioral analytics, experimentation, and business metrics to understand user journeys, evaluate AI performance, and build AI products that people actually trust and use.
Drawing on his experience building enterprise-scale data products at one of the largest independently owned grocery retailers in the United States, Prithvi offered a practical perspective that many AI discussions overlook. The session wasn’t about choosing the best foundation model or writing better prompts. It was about what happens before an AI product is built—and, more importantly, what happens after it ships.
AI Product Analytics Starts with the Business, Not the Model
One of the first ideas Prithvi challenged was the way many AI projects begin.
Too often, teams start with the technology.
A powerful LLM.
An intelligent assistant.
A sophisticated recommendation engine.
Only afterwards do they begin searching for somewhere to use it.
According to Prithvi, successful AI products reverse that process entirely.
Before introducing AI, teams must first understand how decisions are made today. Who makes those decisions? What information do they rely on? Where do they spend most of their time? Which parts of the workflow create friction? Where does decision fatigue begin?
Only after mapping the existing business process should AI enter the conversation. Otherwise, even the most capable model risks becoming another impressive demonstration that never translates into measurable business value.
This simple shift—from starting with technology to starting with workflow—became the foundation for everything that followed during the session.
A Real-World AI Product: Solving Decision Latency in Retail
Rather than relying on theoretical examples, Prithvi grounded the discussion in a production AI product built for supply chain operations.
Managing inventory across a large grocery network is a constant balancing act.
Products move between suppliers, warehouses, transportation fleets, distribution centres, and retail stores before finally reaching customers. Every stage introduces new variables—supplier delays, transportation constraints, seasonal demand, promotions, warehouse capacity, and product shelf life.
Take everyday essentials like milk or bread.
Running out of stock means disappointed customers and lost revenue.
Holding too much inventory means spoilage, shrinkage, and unnecessary operational costs.
The challenge isn’t that organizations lack data.
It’s that analysts often spend hours navigating multiple dashboards, exporting reports, comparing spreadsheets, writing SQL queries, and stitching together information before they can confidently decide whether inventory should be transferred, expedited, reordered, or held.
That’s where the Inventory Decision Assistant comes in.
Its purpose isn’t to replace analysts.
It’s to reduce the time between identifying a problem and taking action.
Throughout the webinar, Prithvi referred to this as reducing decision latency—helping analysts reach the right decision faster by surfacing risks, explaining root causes, and recommending the next best action.
AI Doesn’t Create Value by Making Recommendations
Another important theme emerged as the session progressed.
Generating recommendations has become relatively easy.
Generating recommendations that people trust is much harder.
Prithvi explained that an effective AI product needs several layers working together.
It needs to detect meaningful business signals.
Provide enough context to explain why something happened.
Recommend an action.
Allow users to execute that action seamlessly.
Finally, it needs to learn whether that recommendation actually helped.
Without these layers, organizations often mistake model output for business value.
Recommendations alone create very little impact.
Recommendations that users understand, trust, and act upon are what ultimately improve business outcomes.
Mapping the User Journey Before Writing Any AI
One of the most practical parts of the webinar focused on mapping the decision journey.
Using the inventory assistant as an example, Prithvi broke the process into several distinct stages.
Everything begins with a signal—a stock-out risk, declining inventory, or an unexpected demand spike.
Analysts then begin investigating by opening dashboards, querying databases, reviewing historical demand, and comparing operational reports.
Next comes diagnosis.
Is demand increasing unexpectedly?
Has a supplier delayed shipments?
Is inventory sitting too long in a warehouse?
Is there an imbalance between distribution centres?
Only after answering those questions can analysts make a decision.
Transfer inventory.
Expedite shipments.
Adjust purchase orders.
Escalate operational issues.
Finally comes execution, where those decisions translate into actions across the supply chain.
By visualising every step in this journey, teams can identify precisely where AI removes friction—and where simpler automation or better process design may already solve the problem.
Not Every Business Problem Needs AI
One refreshing takeaway from the session was that Prithvi repeatedly cautioned against applying AI indiscriminately.
Some business problems simply don’t require large language models.
Static reporting rarely does.
Rule-based automation often doesn’t.
Basic workflows can frequently be solved without introducing generative AI at all.
According to Prithvi, AI becomes valuable when decisions are repetitive, time-sensitive, involve multiple signals, or require synthesising complex information into actionable recommendations.
That’s where AI shifts from being a novelty to becoming genuine decision support.
Why Product Analytics Matters After Deployment
Perhaps the biggest insight from the webinar came after the AI product had already been deployed.
Shipping the product isn’t the finish line.
It’s the beginning.
Once users begin interacting with an AI assistant, product teams need to understand how that interaction unfolds.
Did users open the recommendation?
Did they read the explanation?
Did they accept the suggested action?
Did they complete it?
Where did they abandon the workflow?
These behavioural questions are often impossible to answer using traditional business intelligence alone.
Instead, they require product analytics.
Prithvi introduced tools such as Amplitude as examples of behavioural analytics platforms that capture user interactions throughout the product lifecycle. Unlike BI dashboards that explain what happened in the business, product analytics helps explain how users behave inside the product itself—and where trust begins to break down.
Where SQL and Python Fit Into AI Product Analytics
While AI dominated the discussion, the webinar repeatedly returned to two technologies that every data professional already knows.
SQL.
Python.
SQL forms the backbone of product analytics by transforming raw event logs into meaningful user funnels.
How many recommendations were generated?
How many were viewed?
How many were accepted?
How many ultimately resulted in completed actions?
Those simple funnel analyses often reveal where products lose users long before sophisticated machine learning becomes necessary.
Python then extends those insights through cohort analysis, segmentation, experimentation, A/B testing, behavioural modelling, and recommendation analysis.
Rather than replacing traditional analytics, AI product development depends on it.
As Prithvi put it, SQL helps measure the funnel, while Python helps explain the patterns behind it.
Measuring What Actually Matters
The webinar concluded with a useful framework for thinking about AI success.
At the bottom sit model metrics.
Precision.
Recall.
Latency.
Cost.
Groundedness.
These remain important.
Above them sits product adoption.
Are recommendations viewed?
Accepted?
Executed?
Are users providing positive feedback?
At the very top sit the metrics every business ultimately cares about.
Did stock-outs decrease?
Was waste reduced?
Were analysts able to make decisions faster?
Did customer experience improve?
Those are the outcomes that determine whether an AI product has truly succeeded.
Final Thoughts
If there was one message that carried through every section of the webinar, it was this:
Great AI products don’t simply generate better answers. They reduce decision friction.
As AI becomes deeply integrated into enterprise products, data professionals will increasingly find themselves working across product management, analytics engineering, experimentation, behavioural analytics, SQL, Python, and machine learning.
The organizations that succeed won’t necessarily be those with the most advanced models.
They’ll be the ones that understand their users best.
Because in the end, successful AI isn’t measured by what the model produces.
It’s measured by the decisions people make because of it—and the business outcomes those decisions create.
See you next time!