Mixture-of-Experts (MoE) Models: Structure and Trade-offs

Overview of Mixture-of-Experts (MoE) transformers: sparse routing with gating networks and experts, training and inference trade-offs, and recent Mistral-8x7B-MoE.

OpenAI Sora: a new AI model for video generation

Technical overview of OpenAI's Sora and its video generation capabilities, core machine learning foundations, and potential impacts on production workflows and society.

Building Graph Neural Networks with PyTorch

Overview of graph neural networks, graph basics and NetworkX graph creation, GNN types and challenges, plus a PyTorch spectral GNN example for node classification.

Sora: OpenAI's Video Model Architecture and Use Cases

Overview of Sora, OpenAI's video diffusion model using a spatiotemporal autoencoder and DiT Transformer to generate high-resolution, minute-long videos.

AI Server PCB Hardware Breakdown

Teardown analysis of NVIDIA DGX A100 AI server PCBs: PCB types, area and per-system value breakdown for GPU board assembly, CPU motherboard, substrates and accessories.

Optimizing Deep Learning Models

Practical deep learning tuning guide covering learning rate selection, batch size effects, weight initialization, optimizers, regularization, data augmentation and training tips.

Understanding the Transformer Neural Network Model

Overview of the Transformer architecture: self-attention, multi-head attention, positional encoding, encoder-decoder stacks, and implications for distributed model training.

How Computer Systems Address Challenges of Large AI Models

Dr. Mao Yang examines how computer systems must evolve - ultra-large-scale computing, cloud redesign, and distributed systems - to support large models and next-gen AI

AI Infrastructure Drives Three Waves of Industrial Change

Technical overview of AI 2.0: how generative AI drives demand for large-scale compute, data pipelines, and Model-as-a-Service (MaaS) to enable industry deployments.

Popular LLM Inference Stacks and Setups

Survey of LLM inference stacks covering throughput, latency and cost; explains hardware constraints, KV cache, quantization, paged/grouped attention, and practical optimizations.

Practical Tricks for Reward Models

Technical overview of methods to improve reward model robustness for RLHF: quantify preference strength, flip/soften labels, apply adaptive margins, contrastive learning and MetaRM

Visualization Techniques for Machine Learning

Guide to machine learning visualization techniques covering model structure, performance plots (ROC, confusion matrix), feature importance, and practical analysis.

Survey of Deep Metric Learning

Survey of deep metric learning: formulations, sample selection and metric loss functions (contrastive, triplet, N?pair), architectures and applications in vision, audio, and text.

12 Core Machine Learning Interview Questions

Overview of 12 deep learning interview questions highlighting core concepts and training considerations, including batch normalization and practical model evaluation.

Can LLMs Reason and Plan?

Analysis of recent research evaluating whether LLMs can plan or reason, showing limited autonomous planning and that apparent emergent capabilities stem from in-context learning.

F-Learning: Forget-then-Learn for LLM Knowledge Updates

F-Learning: a parameter-based fine-tuning paradigm that subtracts knowledge parameter deltas to forget outdated facts, then fine-tunes (LoRA or full-model) to update LLM knowledge.

Using AI Tools to Improve IC Design Efficiency

Synopsys.ai and Microsoft extend Copilot into EDA with Azure OpenAI, adding GenAI features for RTL generation, formal verification assertions, and validated design workflows.

Implicit Harmful Outputs in Large Language Models

Tsinghua study analyzes implicit harmful content in LLMs, showing how SFT and RLHF can induce subtle abusive outputs that evade harmful-content detectors.

RL-Designed GPU Arithmetic Circuits at NVIDIA

PrefixRL uses deep reinforcement learning to optimize parallel prefix circuits, producing smaller, lower-latency adders and mapping Pareto trade-offs between area and latency.

Retrieval-Augmented In-Context Learning for Domain Adaptation

DAICL: a retrieval-augmented in-context learning framework for unsupervised domain adaptation that retrieves target examples and jointly optimizes task and LM.