Machine Learning Engineer — AI Architecture Research

About the RoleWe’re looking for a Machine Learning Engineer focused on AI architecture research to help design, prototype, and validate next-generation model architectures. You’ll work at the intersection of research and production — turning new ideas into scalable, real-world systems.This role is ideal for someone who enjoys questioning architectural assumptions, experimenting with novel model designs, and pushing beyond standard Transformer-style approaches.What You’ll Work OnResearch and develop new neural network architectures (e.g. alternatives or extensions to Transformers, recurrent / hybrid models, long-context systems)Design and run architecture-level experiments (scaling laws, memory mechanisms, compute trade-offs)Prototype models end-to-end — from research code to training-ready implementationsCollaborate with inference and systems engineers to ensure architectures are deployable and efficientAnalyze model behavior, failure modes, and inductive biasesRead, reproduce, and extend cutting-edge research papersContribute to internal research notes, benchmarks, and open-source efforts (where applicable)What We’re Looking ForStrong background in machine learning fundamentals and deep learningHands-on experience implementing model architectures from scratchSolid understanding of:Attention mechanisms, RNNs, state-space models, or hybrid architecturesTraining dynamics, scaling behavior, and optimizationMemory, latency, and compute constraints at the model levelComfortable working in PyTorch or JAXAbility to move fluidly between theory, experimentation, and engineeringClear communicator who can explain architectural trade-offsNice to HaveExperience with non-Transformer architectures (RNN variants, SSMs, long-context models)Background in research-driven startups or open-source ML projectsExperience with large-scale training or custom training loopsPublications, preprints, or notable research contributionsFamiliarity with inference optimization and deployment constraintsWhy JoinWork on core model architecture, not just fine-tuningDirect influence on the technical direction of a Series-A companySmall, high-caliber team with fast feedback loopsOpportunity to ship research into productionCompetitive compensation + meaningful equityOriginally posted on Himalayas