Reviews & Download Counts

Trying to find a positive of the review-gate I decided this was a great opportunity to find how positive ratings affects download counts.

Be warned: while I like numbers, statistics isn’t a strong area of mine; and while I tried to be objective about these numbers it’s sorta impossible when my intent is to prove a point.  So take the data that you can find on the stat’s page of Sprout Invoices (and any others) and let me know what I got wrong/right, I don’t mind.

Question 1: Do negative reviews (sub 2.5 stars) affect download counts?

On June 28th and 29th SI had a rating average of 2.2: downloads counts for those two days were down ~30 & ~18% respectively.

Notes:

• I used another invoicing plugin in the .org repo to act as a control. So that the decrease could be applied to a WordPress user deciding to download X instead of Y based on the difference in reviews.
  • The numbers show that the percentage of downloads decreased for SI while the percentage of downloads increasing for the other plugin. The trend line looks like an x, displacing downloads from one to the other and the assumptive cause is the reviews.
• I understand this is an extremely short sample size and since SI received a single positive review on the 1st of July I can’t extend this any further.

I’m fairly confident that the answer is “yes”, that’s a no brainer. However, I don’t feel confident providing a percentage, other than to say it must be more than the percentage decreased based on the second question…

Question 2: Do more positive reviews affect download rates?

Since the 28th (until yesterday) I’m finding a 15% decrease in downloads.

Again, this is controlled by the average download count ¹ that uses a similar plugin to see if the downloads went to the plugin with more or less positive reviews. In this case SI had one single 5 star review, the other had on average 26 reviews with a 4.9 rated average.

Question 3: Does it matter if you’re selling premium upgrades?

At this point, I don’t know. In order to draw any conclusions I’d first need to know the time it takes for a free user to convert to the paid version, and I’m not going research that anytime soon. More importantly, the data set would need to be large enough to create a median so that poor/good conversion rates don’t skew the results.