With the introduction of the popup interface to vote on pictures in Gran Torneo, I was thinking that the same type of interface could be used to see all upvoters and downvoters of a regular forum post. Perhaps clicking on the vote score could bring up this window, and of course, clicking away from it (or adding a close button) would close the window. This way, you still have the initial performance of loading threads without having to look up every voter on each post, yet still have a way to view all voters if you want to.