I did some tests this weekend.
I first swapped the VIC with that of one of my real C64s. The C64R got back to work. I was able to run a few games and demos. So I thought the VIC was faulty and started testing the real C64. But it works flawlessly and I ran the Dead Test 781220 cartridge for 8 hours with no issues.
There, I thought to myself that there must be something weird in the C64R. I ran the Dead Test cartridge on it and saw that maybe I have a problem with one of the CIA chips. After a few cycles, the test no longer displays the same time. At first it was after 2 or 3 cycles and after cleaning the chips pins, I can go up to an hour.
So I'm not going to send the card back now. I will first fix this problem with the CIA chip before.