Senior AI Software Engineer

Type of Requisition: Pipeline Clearance Level Must Currently Possess: Secret Clearance Level Must Be Able to Obtain: Top Secret Public Trust/Other Required: None Job Family: Data Science and Data Engineering Job Qualifications: Skills: Agile Software Development, Back-End Development, Microservices Architecture Certifications: None Experience: 7 + years of related experience US Citizenship Required: Yes Job Description: GDIT is seeking a highly motivated and experienced Senior AI Software Engineer to support the development, modernization, integration, and sustainment of our on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, VA. The selected candidate will design, develop, and integrate software solutions leveraging Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, document processing technologies, and modern AI inference frameworks. This role requires a strong foundation in software engineering, Python development, API integration, containerization, and AI application architecture. The engineer will help transition existing AI services toward a scalable, containerized architecture while introducing new capabilities that enhance system performance, reliability, security, and user experience. The ideal candidate is a hands-on software engineer who can work across the AI application stack, from backend development and model serving integration to data processing, automated testing, and deployment. The individual will collaborate with AI/ML engineers, data engineers, infrastructure specialists, cybersecurity teams, and government stakeholders to deliver secure, reliable AI capabilities in highly controlled, classified, and air-gapped environments. Key Responsibilities 1. AI Application Development & Integration Design, develop, test, and maintain Python-based applications, microservices, and APIs supporting an on prem chatbot. Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems. Develop custom OpenWebUI Functions, Tools, and integrations to support mission-specific use cases. Integrate approved LLMs into chatbot using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama. Develop reusable software components for AI model access, prompt management, response processing, and application workflows. Collaborate with AI/ML engineers to improve model inference performance, response quality, and application reliability. Support multimodal AI use cases, including document understanding, optical character recognition, and vision-language model integration. Develop backend capabilities to support secure enterprise AI workflows. 2. Retrieval-Augmented Generation (RAG) & Document Processing Design, implement, and optimize RAG workflows for enterprise document search and question answering. Integrate document processing technologies such as Docling, Tesseract, and vision-language models into automated ingestion pipelines. Develop document chunking, metadata extraction, embedding generation, and retrieval workflows. Integrate embedding models and reranking models to improve retrieval accuracy and relevance. Work with PostgreSQL and pgvector to support vector search, semantic retrieval, and document indexing. Develop APIs and background processing services for document ingestion and knowledge base management. Collaborate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information. Implement testing and evaluation approaches to measure RAG accuracy, retrieval relevance, and answer quality. 3. AI Model Serving & Inference Integration Integrate and maintain AI inference services using vLLM, Ollama, and compatible model-serving frameworks. Support the deployment and configuration of approved open-weight LLMs, embedding models, rerankers, and vision-language models. Develop software integrations using OpenAI-compatible REST APIs. Assist in optimizing inference configurations, including context length, GPU memory utilization, concurrency, and model serving performance. Support the transition from existing inference frameworks to improved architectures as technologies mature. Collaborate with AI/ML engineers to troubleshoot inference errors, model loading issues, and integration failures. Develop benchmarking and performance-testing utilities for AI workloads. 4. Containerization & Platform Modernization Support the migration of DS Chat components from traditional deployments into containerized services. Develop, configure, and maintain container images using Podman or Docker. Create and maintain Podman Compose configurations defining application services, networking, volumes, and environment variables. Integrate services such as OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis. Develop scripts and automation for repeatable installation, deployment, upgrades, and recovery. Implement health checks, readiness checks, logging, and service dependency management. Troubleshoot container networking, authentication, storage, GPU access, and service connectivity. Collaborate with infrastructure and cybersecurity teams to improve deployment consistency and security. 5. Agentic AI & Future Capabilities Research, prototype, and develop agentic AI capabilities for future DS Chat releases. Implement controlled tool-calling and multi-step AI workflows. Develop integrations that allow approved AI models to interact with authorized enterprise APIs and services. Explore emerging technologies involving AI orchestration, workflow automation, and Model Context Protocol (MCP). Design secure execution boundaries, permissions, audit logging, and approval mechanisms for AI-driven actions. Collaborate with stakeholders to evaluate new AI capabilities for mission relevance, security, and operational feasibility. I