Workday and Databricks Enhance AI Scalability with Unity Catalog
By collaborating with Databricks, Workday is tackling the complexities of AI deployment with a universal data layer, ensuring that trust in AI systems remains intact while optimizing data management across the board.
Key Facts
- Workday's universal data layer reduces integration complexity from M×N to M+N, enhancing scalability.
- Unity Catalog ensures fine-grained access control, mitigating risks of data breaches and compliance failures.
- AI agent deployment across 10,000 organizations improves operational efficiency, driving significant ROI potential.
- Modular design allows rapid onboarding of new agents, reducing technical debt and maintaining governance integrity.
- Workday's focus on interoperability positions it for future growth in complex data environments, enhancing competitiveness.
Summary
Summary
Workday, a leader in enterprise cloud applications for finance and human resources, faced challenges in scaling AI agents across its platform while maintaining data integrity and trust. By partnering with Databricks to implement a universal data layer using Apache Iceberg and Unity Catalog, Workday transformed its data architecture, resulting in a scalable solution that significantly improved operational efficiency and trust in AI outputs.
Background
Workday serves over 10,000 organizations worldwide in the finance and human resources sectors. Before deploying the new AI architecture, Workday struggled with a complex system of independent data stores that led to inefficiencies and governance challenges as it began to scale AI agents across various departments, including procurement, finance, and HR.
Challenge
The primary challenge was the multiplicative complexity created by point-to-point connections between AI agents and their respective data stores. With 100 agents and 100 data systems, the architecture became unmanageable, leading to potential trust issues in AI outputs. Workday's leadership recognized that losing trust in AI could hinder adoption and effectiveness across the organization.
Solution
Workday opted to build a centralized, open data foundation using Apache Iceberg for data management and Databricks Unity Catalog for governance. This "universal data layer" allows AI agents to access governed data without creating additional replicated stores. Iceberg provides transactional guarantees, while Unity Catalog enforces fine-grained access control and auditing, ensuring that agents operate within their authorized data boundaries.
Results
The implementation of the universal data layer and Unity Catalog has enabled Workday to accelerate finance close cycles, identify attrition risks in HR early, optimize supply chain negotiations, and improve sales planning accuracy. The architecture has facilitated a shift from single-agent pilots to coordinated multi-agent deployments without incurring technical debt or governance gaps.
Key Insights
Workday's experience illustrates the importance of centralizing data and governance to manage complexity in AI deployments. A modular design allows for flexibility and rapid onboarding of new agents, while a focus on trust and compliance is critical for successful AI adoption.
Customer Testimonial
"We realized we cannot operate like this... One hundred good things, but one bad thing and the trust erodes." — Phoenix Majumder, Senior Director of AI Engineering and Platforms at Workday.
Entities Mentioned
Companies
Products
People
Key Concepts
Definitions
- Governed Agentic AI
- A framework for deploying AI agents that ensures data governance and trust while scaling across an organization.
- Universal Data Layer
- A centralized data architecture that allows AI agents to access governed data without creating additional data silos.
- Unity Catalog
- An open-source governance solution that manages access controls and data lineage for data and AI assets.
- Apache Iceberg
- An open table format that provides transactional guarantees and supports ACID compliance for data management.
- Modular Design
- An architectural approach that allows components to be easily replaced or upgraded without disrupting the overall system.
Use Cases
- →Finance close-cycle acceleration
- →Early attrition-risk identification in HR
- →Supply chain and procurement contract negotiation optimization
- →Improved retention modeling
- →Sales planning accuracy
Frequently Asked Questions
What is Unity Catalog?
Unity Catalog is the industry's first open-source, universal governance solution for data and AI assets across cloud platforms. It provides a centralized interface for managing access controls and maintaining data lineage.
How does Workday ensure data governance?
Workday ensures data governance through a centralized architecture that utilizes Unity Catalog for access control and auditing. This approach helps maintain trust in AI outputs by preventing unauthorized data access.
What are the benefits of using Apache Iceberg?
Apache Iceberg offers ACID compliance and snapshot isolation, providing a consistent view of data for AI agents. This is crucial for ensuring reliable decision-making when multiple agents query the same datasets.
How does Workday's architecture reduce complexity?
Workday's architecture reduces complexity by transforming the data integration model from a multiplicative to an additive approach. This simplification allows for easier scaling and management of AI agents across departments.
What future priorities does Workday have for its AI systems?
Workday's future priorities include expanding interoperability, establishing communication protocols for agent interactions, and enhancing governance automation to monitor and improve AI agents over time.