Let me preface this post by mentioning that this was difficult to write, I’m still frustrated but I’ve accepted the immediate outcome . That said…
Last week I received an email from the WordPress.org plugin review team that Sprout Invoices “has been found to be in violation of the repository guidelines, found at https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/“.
Turns out this was the issue:
What did .org do?
Since I was accused of compensating reviews through “bribery”: Sprout Invoices was immediately pulled out of the repo until that link (and the page at the other end of “send us the link”) were removed.
Also all 5-star reviews were deleted; all of them…
“The reviews that have been associated with this method have been removed from the system in order to restore equilibrium to the reviews.”
Was I compensating reviewers?
Obviously yes; through a discount of a pro license, I figured it was alright to compensate their time.
Was it “Bribery” or did I “Pay Reviewers”? Why does that distinction matter?
In no way was this “bribery”!
Semantics are important, as are my ethics (business and personal) . In no way did I try to persuade a reviewer to provide a good review by offering a discount. Never did I tell a reviewer that submitted for a discount that s/he should change their review. Never did I pay for a review. In fact, the entire system was automated: form submitted, auto-reply sent.
The .org team telling me that I “bribed” users and attempted to “defraud a system”
is was is insulting. It’s too bad they continue to use that type of language instead of broadening their understanding.
Did I know that this type of compensation was against any guidelines?
Should I have known?
No, since there’s no guideline.
What do you mean there’s no guideline against compensating reviews?
There isn’t. Check for yourself – https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/
Nor is any of this mentioned in the TOS.
Although, this was offered in our conversation, “If it’s not clear enough, we’re serious. We even posted on make/plugins – https://make.wordpress.org/plugins/2016/05/31/reminder-do-not-compensate-reviewers/“; a Make blog post shouldn’t be required reading for plugin authors.
Does it matter to them if I knew or not?
No, since the reviews are considered spam.
“The question I have in such a case is: Why do we care what you knew? In our view, you spammed us.”
Does it matter to me (and by extension us)?
Yes. Otherwise any wrong doing can be concluded by opinion, regardless of community feedback.
It’s why the best thing to do IMO (in my case and others) was to reach out first, letting me know the link could be judged as spam inducing.
If the Make blog post was the team’s way of reaching out to everyone: fine.
Are these reviews spam?
This is where opinions diverge.
I agree that they can judge all SI reviews as spam, they can set the conditions for “spam” to apply to any review in the system.
However, were they spam two years ago when there was no guideline, no Make blog post, and no public conversation was had about compensating for a review (regardless of rating)? IMHO, no.
How should they have handled it then?
They could have pulled the plugin from the repo until I fixed the link, explaining that compensating for reviews is deemed as spam and against a guideline (that’s “yet to be published” and then publish something). Warning me to never ask for reviews in this manner again.
So, reviews shouldn’t have been deleted?
No, and for two reasons:
- The “guideline” that I was breaking isn’t a guideline at all. So for me to say that I was unknowingly breaking a guideline for two years is perfectly valid.
- They mention there is precedence and “I’m not the first” but I’ve never heard of this happening before and I’ve been around for a very long time.
- The precedence I see while searching “bribery” and “compensation reviews” in the Slack channels show incidences where only a single review was removed, even though it seems like the plugin author was a repeat offender.
- Their intent was: “the reviews that have been associated with this method have been removed…”. Instead they deleted all reviews because there was no way for them to know if a review had received a discount.
Is there a way to find out if a review was associated with a discount?
Yes, and I provided it to them (since it’s a basic list of form submissions, including their review link). Since November of 2014 — 34 submissions were made, 8 of those submissions didn’t review the plugin (although they still got a discount).
I asked that instead of deleting all 110+ reviews the 26 (that can be associated) be removed instead, they disagreed 😕.
Did they overreact?
I get that the .org teams put up with a lot of shit, and they need to be dogmatic. I also (honestly) respect their time and effort.
I just wish they reached out to me first. I can’t imagine their original thought was “Dan is spamming our system” with reviews. I had around 120 reviews in two years and thousands of active installs, those numbers prove I’m not a spammer trying to just sell a pro product.
The reviews are from real people that shared their experience.
“We’re basically at odds with people trying to use our directory to sell their wares here, because we’re not a place to sell things in the first place. This is why we have such strong opinions on the matter. This is why we require GPL compatible everything. Because we’re all about community, and the community frequently is hurt by competition, but benefits from collaboration.”
These specific reviews (26 of them) for Sprout Invoices were not spam, they were reviews from SI users that wanted to pay for the pro version because they liked the free version (after using it). The free version in the repo is by no means a trojan horse to force users to upgrade, active installs bear that out. These reviewers are pieces of this community too, as am I.
Update: The reviews are still in the DB but they can’t be updated/edited/published.
I was contacted by one of my users that wanted to edit his review. I asked the .org team and the response was: “No, those reviewers can never leave reviews ever again [on that plugin]. Reviews are a one-time deal.
Again, this is why we tell people not to buy reviews. Not only are they all deleted, but the people who left reviews now cannot do so again [on that plugin]. They left a review, it got deleted, and now, they have lost their opportunity.”
This is rather sad; I can easily explain why their review was deleted, “I unknowingly broke a guideline and offered a discount for reviews…the .org team couldn’t conclude if your review was one of them so all were deleted…”, I can’t easily explain how that relates to them now that they want to change/edit or resubmit a new review.
I’ve tried. I made my points clear, maybe not concise, but I was nice and respectful throughout (except at one point when I said the response was “vitriolic” but I quickly apologized and pointed out the hypocrisy since I had/have such an issue with “bribery”). I shared that I would respect their decision but I would like for the decision to be fair.
I’ve been told there’s a “zero tolerance policy” for compensated reviews. Regardless if the policy in question is undefined and it’s unknowingly broken. It’s [scare quotes]their system[/scare quotes], which I get — to a point.
I just don’t think the result is fair, even if my opinion is that some reviews can now be deemed spam.
And as a plugin author I shouldn’t be nervous that something I’m currently doing is later deemed malicious, i.e. asking for a review at all or advertising a pro version.
What would have been the result if I read the Make blog post from a month ago?
Nothing! This is the most depressing part of this entire incident: the thought nothing would have happened if I learned about this before last Tuesday is maddening. Sigh.
- I somehow stay proud of all the hard work I’ve poured into the project — which those reviews proved but are now deemed “dishonest”.
- Track whether ratings affect for download counts; spoiler: they do and I’ll write about that later.
- I somehow prove that some changes need to be made (including the process) and my suggestions below are considered.
What I’d like to see (i.e. suggestions):
- The guidelines updated to cover compensated reviews.
- Clearly defining what is and what isn’t considered.
- Possibly the repercussions; not a first/second offense plan but a “we will delete all reviews if we find you’ve broken these rules, and without warning.“.
- Expectations for plugin authors to keep up to date should be provided, unless a written warning is to be expected now.
- A disclaimer for all below every review submission to not review the plugin if: “You have or will be compensated in anyway”.
- Other disclaimers that all plugin authors have been begging for can also be added, i.e. “A review is a place to ask for support, or demand a feature not present”.
This entire incident still really bums me out, to the point of wanting to throw my hands up and walking away; is hard to find joy in my work at the moment.
Should you (and I) be outraged?
Haha, no. I’m upset by their decision and tact but that doesn’t mean much more than I hope I prove a point so things can change for the better. I also care about people and can see past our disagreements to enjoy a beer in the (hopefully near) future.
So don’t make this out to be fuel for your own personal dumpster fire against .org, or the community. This is my rant to try to improve things for everyone.