I am using https. The http://url is where UniversalViewer is looking for the tiles.
I have set the viewer back to TileViewer, cleared the cache, reprocessed media, rebuild search and sort values and somehow the images are loading properly for now, also for new images.
It is almost like something got stuck and popped loose after changing viewers.