I have described the problem here:
and nobody could answer, but one person suggested that this could be a VS bug, as he seen this problem on built-in adornments. Although in my simple VS plugin this happens very often and I think I don't use the vertical scrollbar at all as I navigate through keyboard (arrows, PgDn, Ctrl+F or F12). I have no idea what causes this behavior. Also there is no point in sending you the plugin as it uses our (really big) database.