We are looking for a Lead DevOps/MLOps Engineer to join our Data Science Department and work with the ML Engineering team to build out and harden our on-premises, GPU-backed ML platform. In this role, you will help enable multiple ML and LLM use cases to run simultaneously, reliably, and with low latency, while scaling the platform to support a wider user base across the organization.
Unlock the potential of remote work in Kazakhstan, giving you the flexibility to work from home or access our offices in Astana, Almaty or Karaganda.
Responsibilities
Build and harden the on-premises GPU-backed ML platform to support concurrent ML and LLM workloads
Troubleshoot and resolve daily resource bottlenecks and latency issues across competing production and development use cases
Deploy and maintain the NVIDIA GPU Operator and manage GPU resources across Kubernetes and OpenShift clusters
Implement GPU sharing and isolation strategies to optimize memory and fault isolation across workloads
Scale the GPU environment to accommodate growing compute demands as new H200 servers are added
Maintain and improve platform services such as MLflow and the Agent Gateway
Manage monitoring pipelines using NVIDIA DCGM exporter, Prometheus and Grafana to track GPU utilization and performance
Collaborate with the Deep Learning and Traditional ML teams to support shared infrastructure, including Ray-based training workloads
Ensure platform reliability and security given the confidential and sensitive nature of data handled on-premises
Partner with the Data Analytics team to support the in-house agentic platform as it scales to a broader employee base
Requirements
5+ years of experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises
Practical experience deploying and troubleshooting the NVIDIA GPU Operator, with clear understanding of each component's function
Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilization semantics and scheduling limitations
Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, along with their trade-offs
Solid Linux fundamentals, including cgroups, namespaces, systemd resource controls and process-level resource management
Experience with Prometheus, Grafana and NVIDIA DCGM, including GPU metrics at process or pod level
Working knowledge of the CUDA stack, including drivers, CUDA toolkit and container runtime version compatibility
Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes
Nice to have
Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton
Experience with Ray, KubeRay and GPU-aware schedulers such as Kueue or Volcano
Hands-on work with H100/H200-class hardware and GitLab CI
We offer/Benefits
We connect like-minded people:
We invest in your growth:
We cover it all:
EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Magnum E-commerce Kazakhstan
Алматы
до 5000 EUR
Choco (ТОО Chocofamily)
Алматы
до 5000 EUR
Biometric(Биометрик)
Алматы
до 4000000 KZT