Now, I usually don't post here, (Though that might have to do with the general low quality of the forum community.) but I kinda want to bring this topic up anyway, since it seems no one else is doing it.
Locking the test servers behind a pay wall is kind of dumb, and in a way, prevents testing to actually be done.
Now, if anyone have any preconceptions that having people pay you to play test your game is a valid way of doing things: I suggest you actually consider the implications of that.
Now, the problem with locking play testing behind a pay wall is that you limit the amount of players, and what kind of players, that can test your content. There is a very small user base that is allowed on the test server, and it's limited to those who have spent any money on the game in the latest few months. That is running on a lot of assumptions, for example, that they are the most competent ones at providing test feedback, that they are a wide variety of players that have a wide variety of opinions and points of view, etc.
Don't get me wrong, I understand the reasoning with the choice:
A. You want players who are dedicated to the game to the be the ones to test.
B. You also, ideally, want a rather small selection of players to test your game in order to keep track of all the data.
However, I don't agree with that reasoning, nor the solution. First of all, the people who pay money in a game like this comes in two types: Those who value their time more than their money, and those who wants to support the developer. I personally fall into the latter, but the former is by far the most common. Although, to the majority it's a mix of both, to some degree. As such, I don't think listing people who have payed in the recent times is any proof of dedication. In fact, I'd go as far as to say that if someone has put any real money into the game is irrelevant, and is no proof that they are dedicated to it.
I would also like to point out that listening to feedback from only dedicated players in the first place is fundamentally flawed. The game is clearly advertised and promoted in a way that is to bring in new players who may be inexperienced with the genre, or games in general. As such, feedback from players who are yet not dedicated to the game is of great value! Or to sum up: Inexperience =! irrelevance. (Though, getting test feedback from this demographic is by far the hardest. But almost entirely locking them out is not exactly helping.
The problem of not wanting to get flooded with data... well, when you run a continuously updated game, people are going to have very strong opinions on it! But it's also important to take into account that people don't always know what they want, and they don't always know what data YOU need. The forum approach that you're running today works, but it's primitive. Anyone who takes up game design today knows that making sure you get the right feedback is very important. Surveys are a great way to do that, (though, not the only way, as should be obvious!) as long as you still leave room for players to provide their own thoughts and opinions.
That is all! ~Lunamon
(And now we wait for people to make completely irrelevant replies with argumentation and reasoning that is not based on any logic in this realm of existence!)
Now, I usually don't post here, (Though that might have to do with the general low quality of the forum community.) but I kinda want to bring this topic up anyway, since it seems no one else is doing it.
This actually comes up all the time. Heck, I knew what your complaint would be when I saw the subject.
But if you don't regularly post here, that can be excused.
Here's the last topic about this, if you want to see the replies there
http://forums.spiralknights.com/en/node/100140
(And now we wait for people to make completely irrelevant replies with argumentation and reasoning that is not based on any logic in this realm of existence!)
So you come here, preinsult everyone here, and complain about the low quality of the forum community.