AI Engineer

Job Title: AI Engineer Job Category: Information Technology Time Type: Full time Minimum Clearance Required to Start: Secret Employee Type: Regular Percentage of Travel Required: Up to 10% Type of Travel: Local * * * The Opportunity: We are seeking an AI Engineer to build, test, and support AI workflows and agents on an enterprise AI workflow management platform supporting the Department of the Air Force. You will work directly with the AI Platform Architect to turn platform designs and government use cases into tested, production-ready solutions in Azure IL-5 cloud environments, integrated with ServiceNow. You will build within the frameworks, patterns, and tools the Architect establishes, including Microsoft AI Foundry and third-party agentic platforms, and help translate customer mission needs into working prototypes and deployed workflows. Responsibilities: Workflow & Agent Development Build AI workflows and agents using the platform's frameworks, tools, and reference architectures Develop RAG pipelines, including document ingestion, chunking, embedding, vector search, and retrieval tuning Build prompt chains, tool calls, and agent logic for document generation, summarization, and task automation Integrate AI workflows with Azure OpenAI services, ServiceNow APIs (Flow Designer, REST, scripted integrations), and internal data sources Build reusable components, templates, and connectors that both technical and non-technical users can leverage Implement workflow orchestration and business process automation Testing & Quality Assurance Develop and execute test plans for AI workflows, including unit, integration, and user acceptance testing Build evaluation datasets and automated evaluation harnesses to measure the accuracy, relevance, groundedness, and reliability of LLM outputs Define and track success metrics and evaluation criteria for AI-driven workflows Identify, document, and triage defects, hallucinations, and performance issues, and drive root-cause analysis and remediation Perform regression testing when models, prompts, or platform components change Architect Support & Collaboration Partner with the AI Platform Architect to implement designs, evaluate tools, and give feedback on platform capabilities and limitations Prototype solutions and proofs-of-concept for government use cases, and present results to customers and stakeholders Work with ServiceNow developers and Azure cloud engineers to troubleshoot integration issues across Azure, ServiceNow, and government networks Contribute to design reviews, engineering standards, and technical documentation Support technical screening of AI/ML talent when requested Operations & Monitoring Support monitoring, logging, and alerting for AI workflows and agents Manage prompt and model versioning, and support CI/CD deployments of AI services Monitor output quality, model drift, and data quality in production workflows, and report findings Track model and compute usage costs and recommend optimizations Government & Compliance Build and operate within DISA STIGs, IL-5 requirements, and Department of the Air Force security policies Implement CUI handling and data classification controls in AI workflows Provide engineering evidence and documentation to cybersecurity teams for ATO and continuous monitoring . Qualifications: Required:   US Citizenship required. Ability to obtain and maintain a secret clearance. 15+ Years of relevant experience (Bachelor’s Degree in relevant field may be substituted for 5 years of relevant experience). Hands-on experience building LLM applications, including RAG, prompt engineering, and agentic workflows Proficiency in Python, plus experience with LangChain, LlamaIndex, or similar AI frameworks Experience with Azure cloud services (Azure OpenAI, AI Foundry, DevOps, API Management) Experience with containers (Docker) and REST API development and integration Working knowledge of vector databases and embedding strategies Experience with test automation and QA for software or AI systems Experience with ServiceNow integration or scripting (Flow Designer, REST APIs, or development) Ability to translate requirements into working solutions and communicate technical results to non-technical stakeholders Familiarity with workflow orchestration tools and business process automation.   Desired:   Active Secret clearance (Top Secret eligibility preferred) Experience in government cloud environments (Azure Government, AWS GovCloud, or similar) Experience with Kubernetes or Azure Kubernetes Service (AKS) Familiarity with DoD compliance frameworks (IL-4/IL-5, FedRAMP, NIST 800-53, STIGs) Experience with MLOps tools and practices (MLflow, CI/CD for ML, model monitoring) Experience with PostgreSQL and data modeling for AI workloads Experience with ATO processes and security accreditation Experience with ServiceNow Flow Designer or ServiceNow development certification Sec+ certification - What You Can Expect:  A culture of integrity. At CACI, we place character and innovation at the center of everything we do. As a valued team member, you’ll be part of a high-performing group dedicated to our customer’s missions and driven by a higher purpose – to ensure the safety of our nation. An environment of trust. CACI values the unique contributions that every employee brings to our company and our customers - every day. You’ll have the autonomy to take the time you need through a unique flexible time off benefit and have access to robust learning resources to make your ambitions a reality. A focus on continuous growth. Together, we will advance our nation's most critical missions, build on our lengthy track record of business success, and find opportunities to break new ground — in your career and in our legacy. Pay Range : There are a host of factors that can influence final salary including, but not limited to, geographic location, Federa