Projects

COBS studies the block-selection problem in hardware-friendly sparse attention. We formalize selection as ranking blocks by attention mass, then use a cumulant expansion to explain why existing first-order selectors lose information.

The resulting method stores a compressed second-order statistic for each block. On the 32k RULER long-context retrieval benchmark, COBS closes about 86% of the gap between the Native Sparse Attention baseline and dense attention while using 15.15 times less KV-cache read traffic than dense attention.

This work was coauthored with Aditya Ghai, Sanjit Neelam, Zaal Vasania, and Akshay Mishra at MatX.

A contracted project with Amazon Annapurna Labs optimizing video diffusion model inference for AWS Trainium 2 chips. The work centered on developing fused NKI kernels for attention and 3D rotary positional embedding modules, performing roofline analyses to identify bottlenecks, and making MFU (model FLOP utilization) improvements.

The goal was to close the gap between theoretical hardware performance and what the model actually achieves in practice — which comes down to understanding memory movement, kernel fusion opportunities, and how operations map onto the accelerator’s compute units.

Altair is a UCI chess engine I built from August 2022 through December 2025. At its peak it was ranked top-30 among over 1000 engines worldwide, with a rating above 3600 — well into superhuman territory. It has been downloaded over 3000 times on GitHub and competed in the Chess.com Computer Chess Championship (CCC) and the Top Chess Engine Championship (TCEC).

The engine uses an exhaustive alpha-beta search bolstered by neural networks trained on 5 billion self-generated data points. Performance was pushed further through statistical testing, profiling, SIMD intrinsics, and simultaneous multi-processing. Building Altair was my first serious dive into low-level optimization and shaped a lot of how I think about search, evaluation, and performance engineering.

A mechanistic interpretability study of YOLO-v11, focused on identifying which internal circuits are responsible for object detection. The approach used pixel-based optimization of activations to discover these circuits, drawing on ideas from feature visualization and circuit analysis.

As part of the project I developed a novel importance-weighting algorithm for automatically identifying kernels relevant to specific object classes, and experimented with receptive fields and feature sensitivity across different layers of the network. The goal was to understand not just what the model detects, but how it does so internally.

A modular, CPU-based C++20 neural network framework written from scratch using only a linear algebra library — no PyTorch, no autograd. The framework supports a range of components from convolutional layers to masked multi-headed self-attention.

As a test of the implementation, I built a GPT-2 styled decoder-only LLM on top of it and matched the training loss of a PyTorch-based baseline on the tiny-shakespeare dataset. The project was primarily about understanding exactly what happens under the hood: gradient flow, layer implementations, and the details that frameworks usually abstract away.

A simulator of orbital mechanics and the N-body gravitational problem, capable of handling 100,000 objects simultaneously. The implementation uses numerical integration, asymptotic algorithms (Barnes-Hut style approximations), and parallel processing with Numba to reach that scale.

A significant part of the work was understanding and reducing approximation error — balancing the accuracy tradeoffs that come with treating clusters of bodies as point masses, and optimizing the gravitational force calculations to keep the simulation tractable.