Description
Transformers, Foundation Models and Scientific AI
About SPOC Biosciences
SPOC Biosciences is developing an on-chip experimental platform that generates high-resolution biological data at scale. Our goal is to combine large-scale experimentation with advanced AI models to improve how biological molecules are designed, evaluated and optimized.
We are looking for an AI Research Engineer Intern with strong foundations in machine learning, transformers and modern deep learning systems. Prior experience in biology, protein design or drug discovery is not required. We are primarily looking for candidates with deep technical ability, strong research instincts and an interest in building foundational AI models for complex scientific data.
What You Will Work On
The intern will contribute to development of transformer-based and multimodal foundation models trained on large, structured experimental datasets.
Potential projects include:
Designing and training transformer architectures for sequence, numerical and multimodal data
Developing representation-learning and self-supervised learning methods
Building models that integrate sequence information with experimental measurements and metadata
Adapting language-model architectures to scientific and biological datasets
Developing embedding, ranking, prediction and generative modeling approaches
Fine-tuning and evaluating open-source foundation models
Improving model training efficiency, inference performance and scalability
Building data pipelines, training workflows and evaluation frameworks
Exploring model interpretability, uncertainty estimation and active learning
Contributing to research that may lead to publications, patents and production AI systems
The exact project will be matched to the candidate’s background and interests.
Requirements
Ideal Background
We welcome applications from undergraduate, master’s and PhD students in computer science, artificial intelligence, electrical engineering, applied mathematics, statistics or related fields.
Strong candidates will have experience in several of the following areas:
Transformers, attention mechanisms and large language models
Deep learning using PyTorch, JAX or similar frameworks
Representation learning, self-supervised learning or contrastive learning
Generative models, diffusion models, autoregressive models or graph neural networks
Distributed training and GPU-based model development
Training and evaluating models on large or complex datasets
Strong Python programming and software engineering practices
Mathematical foundations of machine learning, optimization and probability
What We Value
Strong understanding of core AI concepts rather than experience applying existing APIs
Ability to implement and modify model architectures from research papers
Curiosity about how foundational models can be extended beyond natural language
Comfort working on open-ended research and engineering problems
Ability to move between mathematical reasoning, experimentation and production-quality code
Independent thinking and willingness to challenge existing approaches
Experience in computational biology, protein design or drug discovery is welcome but not required. We expect to provide the necessary scientific context.
Internship Structure
Type: Paid internship
Duration: Approximately 3-6 months, with potential for extension
Full Time Position: Potential for conversion to a full time position during or at the end of the internship
Commitment: Full-time preferred; part-time arrangements may be considered during the academic year
Location: Scottsdale, Arizona. Remote arrangements may be considered for exceptional candidates.
Start date: Flexible
Interns will work directly with SPOC’s technical and scientific leadership and collaborate with researchers developing experimental datasets and AI-enabled discovery systems.
