MIRA: Memory-Integrated Reconfigurable Adapters – A Unified Framework for Settings with Multiple Tasks

Susmit Agrawal1,2,3,∗,†, Krishn Vishwas Kher1,∗, Saksham Mittal1, Swarnim Maheshwari1, Vineeth N. Balasubramanian1,4
1IIT Hyderabad 2University of Tübingen 3Tübingen AI Center 4Microsoft Research India
∗Equal contribution, †Majority of work done at IIT Hyderabad
Paper (PDF) arXiv (soon) Code (coming)
MIRA overview diagram (associative memories + adapters)

MIRA augments a frozen backbone with Hopfield-style associative memories that store adapter weight updates and retrieve an input-conditional affine combination at inference—enabling rapid task switching and strong retention across DG, DIL, and CIL.

Abstract

We propose Memory-Integrated Reconfigurable Adapters (MIRA), a unified framework that layers Hopfield-style associative memories over a shared backbone. MIRA stores adapter weight deltas as values and learns keys post hoc to retrieve sample-wise affine combinations of adapters for any task/domain. With only objective changes across settings, MIRA jointly addresses domain generalization (DG), domain-incremental (DIL), and class-incremental (CIL) scenarios, yielding state-of-the-art or competitive results on standard benchmarks while mitigating catastrophic forgetting.

Beyond accuracy, ablations show both the necessity of explicit associative memory storage and the benefit of key refinement. The design closely mirrors neuro-inspired task modulation with a single substrate dynamically reconfigured by memory-guided adapters.

Highlights

  • Unified architecture for DG, DIL, and CIL with a single backbone + memories.
  • Associative memory retrieves per-sample adapter ensembles via learned keys.
  • SoTA or competitive results on PACS, VLCS, OfficeHome, DomainNet, iDigits, CORe50, ImageNet-R, DN4IL, and CDDB-hard.
  • Simple two-stage recipe: Adaptation (train adapters) → Consolidation (learn keys & query modules).

Method at a Glance

Each transformer layer attaches a memory unit that stores task adapters. A lightweight query module maps activations to memory queries; the memory returns an affine combination of stored adapters, which modulates the layer weights on-the-fly. Consolidation fine-tunes keys (and queries) to align retrieval with task performance—no gradients at inference.

Two Stages

  1. Adaptation: Train LoRA-style adapters per task/domain; write adapters into layer-wise memories with placeholder keys.
  2. Consolidation: Freeze adapters; train keys and query modules to optimize retrieval (cross-entropy) on seen data.

Related Links

Parameter-efficient fine-tuning and memory mechanisms underpin MIRA. See LoRA/adapter methods, Hopfield-based memories, and unified continual learning.

Representative resources: AdapterHub, LoRA, Modern Hopfield Networks, Continual Learning.

BibTeX

@inproceedings{agrawal2025mira,
  title     = {MIRA: Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks},
  author    = {Agrawal, Susmit and Kher, Krishn Vishwas and Mittal, Saksham and Maheshwari, Swarnim and Balasubramanian, Vineeth N.},
  booktitle = {NeurIPS},
  year      = {2025}
}