Ok, I solved it! Sort of. You can ignore the message above.
Found a thread in the German part of the forum where the same problem was described, and while my German is not top notch (and Google Translate only helps so far), I saw the Agnus mentioned. And I thought, Agnus does stuff with the clock so why not.
And as by chance, I bought an entire A500+ chipset for pretty cheap some time back. So I switched to the 8372A and poof, problem gone. Can even attach my oscilloscope without the mouse jumping around!
There seems to be a bit of jitter on the 7MHz with the 8371, but not with the 8372A. I have no idea if that's the issue, though. Looks like there's a tiny bit less noise too. See attached pictures.
7MHz line, measured on Indivision, with 8371:
7MHz line, measured on Indivision, with 8372A:
Saw you talked about power supplies too. This is with the Vampire and a non-recapped original power supply. Also works with an A-Power supply I usually use with my 1200. I think fixing power supplies will be my next project after this one.