Research Engineer – Generative AI (LLMs)

About the job

What you’ll do • Build and optimize LLM training and inference pipelines on cloud GPUs. • Generate, curate, and maintain datasets for pretraining and finetuning. • Implement and improve transformer architectures (attention, positional encodings, MoE). • Optimize inference using FlashAttention, PagedAttention, KV caches, and serving frameworks like vLLM / sglang. • Collaborate with research and product teams to design experiments, analyze results, and ship improvements. What we’re looking for • Strong Python skills and solid software engineering practices. • Hands-on experience with LLM training and inference. • Proficiency with PyTorch or JAX. • Experience with Hugging Face libraries: transformers, trl, accelerate. • Experience training on cloud-hosted GPUs and with distributed / mixed-precision training. • Strong understanding of transformer internals: attention, positional encodings, MoE. • Familiarity with writing prompts, tool definitions, and managing context for LLMs in real applications (langchain, pydantic, smolagents). Nice to have • RL for LLMs (RLHF, PPO, GRPO). • CUDA / GPU kernel or systems-level performance work. • Experience with training infrastructure: monitoring, checkpointing, networking / distributed systems.

Requirements

• LLM training
• Inference pipelines
• Python
• PyTorch
• Hugging Face

Preferred Technologies

• LLM training
• Inference pipelines
• Python
• PyTorch
• Hugging Face

About the company

Abacus.AI is a leading Generative AI company building a future where AI assists and automates most work and business processes for enterprises and professionals.