NASWA Enhances Labor Market Insights with AWS Generative AI
Discover how NASWA transformed its data processing capabilities by integrating AWS generative AI, enabling real-time labor market insights that drive workforce development and policy decisions.
Key Facts
- NASWA's NLx Hub processes 200M job postings, enhancing labor market insights for 50 states.
- 80% cost reduction achieved via AWS generative AI, improving efficiency and financial sustainability.
- Collaborative approach between AWS and NASWA ensures alignment with workforce development goals.
- Real-time data access empowers policymakers to adapt training programs to evolving job market trends.
- Architectural redesign positions NASWA to scale insights, addressing labor market gaps effectively.
Summary
Summary
The National Association of State Workforce Agencies (NASWA) faced challenges in providing timely and accurate labor market data. By deploying AWS generative AI to process nearly 200 million job postings, they transformed their data extraction capabilities, achieving an 80% cost reduction and significantly improving processing efficiency.
Background
NASWA operates within the public sector, focusing on workforce development across the United States. Prior to the deployment of AWS generative AI, NASWA struggled with extracting actionable insights from the National Labor Exchange (NLx) Research Hub, which contained a vast amount of unstructured job posting data.
Challenge
The primary challenge was to efficiently process millions of job postings to extract meaningful labor market insights that could inform policy decisions and support job seekers. The existing methods were not scalable or cost-effective.
Solution
The solution involved a comprehensive redesign of the data processing pipeline. Initially relying on Amazon Bedrock for batch processing, the NASWA team shifted to a combination of batch and real-time processing using Amazon Nova Lite. They implemented a technique called prompt decomposition, breaking down complex AI tasks into simpler, focused prompts for better efficiency and accuracy. This approach allowed for the extraction of structured information on educational requirements, pay, benefits, remote work status, and full-time classification.
Results
The architectural transformation led to an 80% reduction in processing costs, resolved performance bottlenecks, and maintained high data quality. The new system enabled scalable analysis of millions of job postings, allowing users to access insights on educational trends and job requirements in real time.
Key Insights
Key lessons from this deployment include the importance of early engagement with architectural expertise, understanding data characteristics, distinguishing between extraction and inference tasks, and fostering cross-functional collaboration. Additionally, planning for scalability from the outset is crucial for successful implementation.
Customer Testimonial
“This partnership demonstrated the value of bringing together different perspectives,” says the project team lead. “AWS provided the technical architecture expertise, while the NLx Research Hub team contributed essential domain knowledge about the data and how system stakeholders will leverage this type of labor market information.”
Entities Mentioned
Companies
Products
Technologies
Organizations
Key Concepts
Definitions
- generative AI
- A type of artificial intelligence that can generate new content or insights based on existing data.
- natural language processing
- A field of AI that focuses on the interaction between computers and humans through natural language.
- prompt decomposition
- A technique in AI where complex tasks are broken down into simpler, more manageable components.
- NLx Research Hub
- A resource that contains a vast database of job postings aimed at providing labor market insights.
- batch processing
- A method of processing data in large groups or batches rather than individually.
Use Cases
- →Improving access to labor market data
- →Guiding training programs for high-opportunity careers
- →Assisting job seekers with career pathways
- →Analyzing educational requirements trends
- →Evaluating tools for structured job posting data
- →Enhancing workforce development strategies
Frequently Asked Questions
What is the primary goal of the NASWA project?
The primary goal is to increase the accessibility of labor market information to enable better decisions for workforce and education stakeholders.
How does AWS generative AI improve data processing?
AWS generative AI improves data processing by using advanced models like Amazon Nova Lite and techniques such as prompt decomposition to enhance efficiency and accuracy.
What are the benefits of cross-team collaboration in this project?
Cross-team collaboration ensures that technical decisions align with workforce development goals, leveraging diverse perspectives to create more effective solutions.
What challenges did the NASWA team face during the project?
The team faced performance and cost challenges during the initial processing approach, which required a complete redesign of their data pipeline.
When will the parsed data be available to Research Hub users?
The parsed data, including pay, benefits, and job requirements, will be available to Research Hub users starting in 2027.