What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers
arXiv:2608.18399v1 Announce Type: new Abstract: Vision transformers (ViTs) trained to copy a pretrained teacher's attention maps recover most of fine-tuning's in-distribution accuracy yet fall measurably short of it under distribution shift, as recent work has shown. What the copy delivers has…