AI Ops Engineer (m/f/d), fully remote in Germany

Job type: Full-time
Experience: 4+ years in DevOps or MLOps
Apply: one online form, button below

Who you work for
With this AI Ops Engineer job you join Insight42, based in Ingolstadt and led by its founder Martin-Peter Lambert. The company won the German Innovation Award for decentralized, risk-eliminating architectures. Your team turns cloud engineering, MLOps and generative AI into secure, resilient systems for European clients.

What you will work on
You design and run the infrastructure behind Insight42’s GenAI and large language model (LLM) workloads. You architect self-hosted GenAI environments, tune GPU/CPU orchestration for performance and cost, and automate model deployment across edge and cloud systems.

Your tasks

  • Design and maintain self-hosted GenAI model deployments in secure cloud environments
  • Build and automate workflows for AI model lifecycle management (training, tuning, and versioning)
  • Build observability and reliability into AI workloads
  • Collaborate with MLOps, DevOps, and Data teams to align infrastructure with model requirements
  • Integrate models into agentic frameworks and autonomous system ecosystems
  • Optimize infrastructure for performance, cost efficiency, and scalability
  • Establish CI/CD and GitOps practices for AI and ML system delivery

What you bring

  • 4+ years of experience in DevOps, MLOps, or AI infrastructure engineering
  • Hands-on experience deploying self-hosted GenAI models (LLMs, multimodal, or diffusion models)
  • Strong understanding of containerization, microservices, and orchestration (Docker, Kubernetes)
  • Familiarity with agentic ecosystems, AI orchestration frameworks, and vector databases
  • Knowledge of GPU orchestration, distributed inference, or on-prem AI serving
  • Solid programming and automation skills (Python, Bash, or Go)
  • Cloud experience with Azure, GCP, or hybrid infrastructure setups
  • Fluent communication in English; German is a plus

Nice to Have

  • Experience with open-source AI frameworks (Ollama, vLLM, FastAPI, LangChain, Haystack)
  • Understanding of AI observability and monitoring tools
  • Exposure to model compression, quantization, and inference optimization
  • Awareness of data privacy, security, and compliance in AI systems

What you get

  • No commute or relocation: work from anywhere in Germany
  • Competitive salary and performance-based incentives
  • Build and run AI infrastructure at scale and grow your expertise
  • A collaborative team with its own products, like Secretary42
Apply as AI Ops Engineer