This paper introduces refined variants of the Local Learning Coefficient (LLC) to study the differentiation and specialization of attention heads in transformer language models during training. The findings reveal how attention heads evolve into distinct functional roles, analyze their specialization based on data types, and uncover a novel multigram circuit, contributing to a deeper understanding of model complexity and interpretability.

This paper presents Dualformer, a Transformer model that integrates both fast and slow reasoning modes to enhance performance in reasoning tasks while reducing computational costs. The model is trained using randomized reasoning traces, allowing it to adaptively switch between modes during inference. The effectiveness of Dualformer is demonstrated through experiments on maze navigation tasks and its application in fine-tuning large language models for math reasoning.

This paper introduces ToolGen, a framework that integrates tool retrieval and execution in large language models (LLMs) by embedding tool-specific virtual tokens into the model’s vocabulary. The framework transforms tool interaction into a generative task, allowing LLMs to efficiently retrieve and execute tools in real-world scenarios. ToolGen's three-stage training process enhances the capabilities of AI agents, setting a new benchmark for scalable and efficient tool usage.

This paper introduces SFTMix, a novel approach to instruction tuning for large language models (LLMs) that leverages training dynamics to identify data subsets of varying confidence levels and incorporates a Mixup-based regularization technique. The proposed method aims to enhance instruction-following capabilities without relying on well-curated datasets, demonstrating significant improvements over conventional methods across various tasks and LLM families.

In this work, we present MatMamba, which is a way to impose a nested Matryoshka structure on a Mamba2 state space model. It brings together the best of both Mamba-style models (faster inference times, especially for longer sequences) and Matryoshka-style learning. A single MatMamba model contains hundreds of nested and accurate submodels that can be flexibly extracted for inference.

This work explores the in-context exploration capabilities of large language models (LLMs) in bandit environments, introducing BanditBench as a benchmark for evaluating their performance. The study finds that LLMs struggle with exploration when relying solely on raw interaction history, but performance improves significantly with inference-time support and algorithmic guidance. The authors propose methods to integrate optimal algorithms into LLMs through both algorithm-guided support and algorithm distillation, demonstrating that smaller models can outperform larger ones in decision-making tasks.

This article investigates the reasoning capabilities of large language models (LLMs) and introduces GSM-Symbolic, a novel benchmark designed to provide deeper insights into their mathematical reasoning abilities. The study reveals significant performance variability across different instantiations of the same question, challenging the reliability of current evaluations on GSM8K. It highlights that while LLMs show some robustness to changes in proper names, they are more sensitive to variations in numerical values, and their performance deteriorates as question complexity increases. The introduction of GSM-NoOp exposes critical flaws in LLMs' understanding of mathematical concepts, indicating that their reasoning resembles sophisticated pattern matching rather than true logical reasoning.

This paper argues that representations in AI models, particularly deep networks, are converging towards a shared statistical model of reality. It surveys examples of convergence across different domains and modalities, hypothesizing that this trend is driven by scaling and performance improvements. The authors discuss the implications of this convergence, its limitations, and counterexamples, while also exploring how different models align with biological representations in the brain.

This article discusses the challenges and advancements in quantizing large language models (LLMs) to reduce their size while maintaining performance. It introduces Vector Post-Training Quantization (VPTQ) as a new method that aims to achieve state-of-the-art accuracy with extremely low-bit quantization, addressing issues related to traditional scalar quantization methods and exploring the benefits of vector quantization.

This paper presents two improvements to traditional biencoder models for generating embeddings, focusing on contextual training strategies and a new corpus-aware architecture that enhances performance in text retrieval tasks.

This paper revisits traditional recurrent neural networks (RNNs) such as LSTMs and GRUs, proposing minimal versions that remove hidden state dependencies to enhance training efficiency. The authors demonstrate that these minimal models can be trained in parallel, achieving performance comparable to modern sequence models while maintaining computational efficiency.

This article presents Movie Gen, a cast of foundation models developed by Meta that generates high-quality videos with various capabilities including text-to-video synthesis, video personalization, and precise video editing. The paper discusses the architecture, training methods, and performance benchmarks of these models, aiming to advance the field of media generation.

This paper presents the UniAudio system, a unified model leveraging large language model techniques to generate various types of audio including speech, sounds, music, and singing. It aims to address the emergent needs in audio generation by providing a foundation model that can seamlessly support multiple audio generation tasks with competitive performance across them.

The paper investigates the phenomenon of grokking in neural networks through the lens of Minimal Description Length (MDL), offering an information-theoretic perspective on sudden generalization. The authors propose a method to estimate and track MDL during training using weight pruning techniques. Experiments on modular arithmetic and permutation tasks reveal a strong connection between MDL transitions and grokking points, with varying dynamics across different tasks.

This paper introduces Emu3, a new series of multimodal models that excel at multimodal generation and perception through next-token prediction. By tokenizing images, text, and videos into a discrete space and training a single transformer from scratch, it eliminates reliance on diffusion and compositional methods while surpassing established task-specific models. The results provide compelling evidence that next-token prediction can serve as a powerful paradigm for multimodal models, scaling beyond language models and delivering state-of-the-art performance across diverse tasks, including challenging video generation.

The article critiques the prevailing 'bigger-is-better' paradigm in AI, arguing that this focus on scale leads to unsustainable practices, environmental concerns, and a concentration of power among a few large players. It emphasizes the need for a shift towards smaller, more efficient models that can address a wider range of applications without the excessive resource demands associated with large-scale systems.

This paper presents MaskBit, a novel embedding-free image generation model that utilizes bit tokens for class-conditional image generation. The authors systematically study and modernize the VQGAN architecture, leading to significant improvements in performance and accessibility. They demonstrate that their approach achieves state-of-the-art results on the ImageNet benchmark while addressing limitations of previous models.

The paper proposes a hybrid algorithm called ARES that alternates between reinforcement learning and supervised fine-tuning to enhance multi-modal rationale reasoning for tasks like ScienceQA and A-OKVQA. It leverages advanced AI feedback to improve the quality of generated rationales and inference accuracy, addressing challenges in traditional reinforcement learning methods.

This article evaluates the planning capabilities of large language models (LLMs) and introduces OpenAI's new Large Reasoning Model (LRM), o1, using the PlanBench benchmark. It discusses the performance improvements of o1 over previous models while highlighting ongoing challenges in achieving robust planning abilities.

This paper introduces a new inference pattern called Writing in the Margins (WiM), which enhances the performance of Large Language Models (LLMs) in handling long input sequences for retrieval-oriented tasks. The WiM approach utilizes chunked prefill of the key-value cache to perform segment-wise inference, improving accuracy and efficiency without requiring fine-tuning. The study demonstrates significant performance boosts across various benchmarks, emphasizing the method's compatibility with existing transformer models and its potential for enhancing user experience through increased transparency in context processing.

This study addresses challenges in text-to-music generation by introducing a quality-aware masked diffusion transformer (QA-MDT) that enhances music generation through improved data quality and alignment between textual descriptions and audio signals. The proposed method demonstrates state-of-the-art performance on benchmark datasets, showcasing its effectiveness in generating high-quality music from textual inputs.

This position paper argues that while Large Language Models (LLMs) cannot independently perform planning or self-verification, they can serve as valuable cognitive tools within a framework called LLM-Modulo. This framework integrates LLMs with external model-based verifiers to enhance planning tasks, addressing misconceptions about LLM capabilities and proposing a structured approach to leverage their strengths effectively.

This paper introduces CLAIR_A, a simple and interpretable domain-specific LLM-based measure for audio captioning. The authors demonstrate that this approach aligns well with human judgments and is significantly more interpretable to downstream users. They aim to inspire further research into the alignment of LLMs with human judgment in various audio domains.

This paper introduces Score Forgetting Distillation (SFD), a novel approach to machine unlearning in diffusion models that allows for effective forgetting of undesirable information while maintaining generative capabilities. The method is data-free and demonstrates significant improvements in both performance metrics and generation speed compared to traditional methods.

This work analyzes the roles of autoregression and recurrence in neural models, demonstrating that recurrence enhances computational depth. It explains how Chain of Thought (CoT) approximates recurrence in Transformer-based autoregressive models and discusses the implications for model design and performance across various tasks.

This article analyzes the effectiveness of chain-of-thought (CoT) prompting in large language models, particularly focusing on its performance in mathematical and symbolic reasoning tasks. Through a meta-analysis of existing literature and experimental evaluations, the authors find that while CoT can enhance reasoning capabilities, its benefits are primarily observed in tasks requiring mathematical or logical reasoning, suggesting a need for more sophisticated approaches beyond traditional prompting methods.

This paper explores the effectiveness of the Chain of Thought (CoT) approach in enhancing the performance of transformer models, particularly in solving inherently serial problems. It provides a theoretical framework for understanding how CoT improves the expressiveness of transformers by enabling them to perform serial computations that are typically challenging for standard architectures. The authors present empirical results demonstrating the advantages of CoT in various tasks, alongside discussions on related works and future directions for research.

This paper proposes a novel approach to address domain shifts by separating them into measurable quality and style components. We introduce the Relative Quality loss that integrates quality scores into the training loss function, adding a quality prior to the classification model. Further, we present DiffFAS, a versatile generative model for high-fidelity cross-domain and cross-attack generation, addressing the lack of labeled data for novel attack types.

This paper introduces reflective augmentation (RefAug) for enhancing mathematical reasoning in language models. It incorporates reflection into training problems, complementing existing data augmentation techniques. Extensive experiments demonstrate RefAug's efficacy in improving both basic problem-solving skills and complex reflective reasoning tasks. The method is also validated in code generation tasks, showing scalability and robustness through various analyses.

Common self-improvement approaches for large language models (LLMs), such as STaR, iteratively fine-tune LLMs on self-generated solutions to improve their problem-solving ability. However, these approaches discard the large amounts of incorrect solutions generated during this process, potentially neglecting valuable information in such solutions. To address this shortcoming, we propose V-STaR that utilizes both the correct and incorrect solutions generated during the self-improvement process to train a verifier using DPO that judges correctness of model-generated solutions. This verifier is used at inference time to select one solution among many candidate solutions. Running V-STaR for multiple iterations results in progressively better reasoners and verifiers, delivering a 4% to 17% test accuracy improvement over existing self-improvement and verification approaches on common code generation and math reasoning benchmarks with LLaMA2 models.

The article discusses the comparison between outcome supervision and process supervision in training large language models, particularly in mathematical reasoning tasks. It highlights the advantages of process supervision, such as providing precise feedback and improving AI alignment. The study finds that process supervision significantly outperforms outcome supervision, especially when scaled up, and introduces active learning to enhance data collection efficiency. The authors release a dataset, PRM800K, to support further research in this area.

Large Language Models (LLMs) have shown remarkable capabilities in natural language tasks requiring complex reasoning, yet their application in agentic, multi-step reasoning within interactive environments remains a difficult challenge. Traditional supervised pre-training on static datasets falls short in enabling autonomous agent capabilities needed to perform complex decision-making in dynamic settings like web navigation. Previous attempts to bridge this gap through supervised fine-tuning on curated expert demonstrations often suffer from compounding errors and limited exploration data, resulting in sub-optimal policy outcomes. To overcome these challenges, we propose a framework that combines guided Monte Carlo Tree Search (MCTS) search with a self-critique mechanism and iterative fine-tuning on agent interactions using an off-policy variant of the Direct Preference Optimization (DPO) algorithm. Our method allows LLM agents to learn effectively from both successful and unsuccessful trajectories, thereby improving their generalization in complex, multi-step reasoning tasks. We validate our approach in the WebShop environment, a simulated e-commerce platform—where it consistently outperforms behavior cloning and reinforced fine-tuning baseline, and beats average human performance when equipped with the capability to do online search. In real-world booking scenarios, our methodology boosts Llama-3 70B model’s zero-shot performance from 18.6% to 81.7% success rate (a 340% relative increase) after a single day of data collection and further to 95.4% with online search. We believe this represents a substantial leap forward in the capabilities of autonomous agents, paving the way for more sophisticated and reliable decision-making in real-world settings.

The article introduces Quiet-STaR, a method for training language models to generate rationales at each token to improve predictions. It addresses challenges such as computational cost and the need for predicting beyond individual tokens. The approach shows improvements in zero-shot reasoning tasks without fine-tuning on specific datasets.