Реклама

What Does Attention Transfer Transfer? Attention Structure and Robustness in Vision Transformers

#What #Does #Attention #Transfer #Structure

arXiv:2608.18399v1 Announce Type: new Abstract: Vision transformers (ViTs) trained to copy a pretrained teacher's attention maps recover most of fine-tuning's in-distribution accuracy yet fall measurably short of it under distribution shift, as recent work has shown. What the copy delivers has…