What Now? No Way? Huh?
Let me preface this post by mentioning that this was difficult to write, I’m still frustrated but I’ve accepted the immediate outcome 1. That said…
Last week I received an email from the WordPress.org plugin review team that Sprout Invoices “has been found to be in violation of the repository guidelines, found at https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/“.
Turns out this was the issue:
What did .org do?
Since I was accused of compensating reviews through “bribery”: Sprout Invoices was immediately pulled out of the repo until that link (and the page at the other end of “send us the link”) were removed.
Also all 5-star reviews were deleted; all of them…
“The reviews that have been associated with this method have been removed from the system in order to restore equilibrium to the reviews.”
Was I compensating reviewers?
Obviously yes; through a discount of a pro license, I figured it was alright to compensate their time.
Was it “Bribery” or did I “Pay Reviewers”? Why does that distinction matter?
In no way was this “bribery”!
Semantics are important, as are my ethics (business and personal) . In no way did I try to persuade a reviewer to provide a good review by offering a discount. Never did I tell a reviewer that submitted for a discount that s/he should change their review. Never did I pay for a review. In fact, the entire system was automated: form submitted, auto-reply sent.
The .org team telling me that I “bribed” users and attempted to “defraud a system” is was is insulting. It’s too bad they continue to use that type of language instead of broadening their understanding.
Did I know that this type of compensation was against any guidelines?
No.
Should I have known?
No, since there’s no guideline.
What do you mean there’s no guideline against compensating reviews?
There isn’t. Check for yourself – https://developer.wordpress.org/plugins/wordpress-org/detailed-plugin-guidelines/
Nor is any of this mentioned in the TOS.
Although, this was offered in our conversation, “If it’s not clear enough, we’re serious. We even posted on make/plugins – https://make.wordpress.org/plu
Does it matter to them if I knew or not?
No, since the reviews are considered spam.
“The question I have in such a case is: Why do we care what you knew? In our view, you spammed us.”
Does it matter to me (and by extension us)?
Yes. Otherwise any wrong doing can be concluded by opinion, regardless of community feedback.
It’s why the best thing to do IMO (in my case and others) was to reach out first, letting me know the link could be judged as spam inducing.
If the Make blog post was the team’s way of reaching out to everyone: fine.
Are these reviews spam?
This is where opinions diverge.
I agree that they can judge all SI reviews as spam, they can set the conditions for “spam” to apply to any review in the system.
However, were they spam two years ago when there was no guideline, no Make blog post, and no public conversation was had about compensating for a review (regardless of rating)? IMHO, no.
How should they have handled it then?
They could have pulled the plugin from the repo until I fixed the link, explaining that compensating for reviews is deemed as spam and against a guideline (that’s “yet to be published” and then publish something). Warning me to never ask for reviews in this manner again.
So, reviews shouldn’t have been deleted?
No, and for two reasons:
- The “guideline” that I was breaking isn’t a guideline at all. So for me to say that I was unknowingly breaking a guideline for two years is perfectly valid.
- They mention there is precedence and “I’m not the first” but I’ve never heard of this happening before and I’ve been around for a very long time.
- The precedence I see while searching “bribery” and “compensation reviews” in the Slack channels show incidences where only a single review was removed, even though it seems like the plugin author was a repeat offender.
- Their intent was: “the reviews that have been associated with this method have been removed…”. Instead they deleted all reviews because there was no way for them to know if a review had received a discount.
Is there a way to find out if a review was associated with a discount?
Yes, and I provided it to them (since it’s a basic list of form submissions, including their review link). Since November of 2014 — 34 submissions were made, 8 of those submissions didn’t review the plugin (although they still got a discount).
I asked that instead of deleting all 110+ reviews the 26 (that can be associated) be removed instead, they disagreed 😕.
Did they overreact?
Absolutely.
I get that the .org teams put up with a lot of shit, and they need to be dogmatic. I also (honestly) respect their time and effort.
I just wish they reached out to me first. I can’t imagine their original thought was “Dan is spamming our system” with reviews. I had around 120 reviews in two years and thousands of active installs, those numbers prove I’m not a spammer trying to just sell a pro product.
The reviews are from real people that shared their experience.
“We’re basically at odds with people trying to use our directory to sell their wares here, because we’re not a place to sell things in the first place. This is why we have such strong opinions on the matter. This is why we require GPL compatible everything. Because we’re all about community, and the community frequently is hurt by competition, but benefits from collaboration.”
These specific reviews (26 of them) for Sprout Invoices were not spam, they were reviews from SI users that wanted to pay for the pro version because they liked the free version (after using it). The free version in the repo is by no means a trojan horse to force users to upgrade, active installs bear that out. These reviewers are pieces of this community too, as am I.
Update: The reviews are still in the DB but they can’t be updated/edited/published.
I was contacted by one of my users that wanted to edit his review. I asked the .org team and the response was: “No, those reviewers can never leave reviews ever again [on that plugin]. Reviews are a one-time deal.
Again, this is why we tell people not to buy reviews. Not only are they all deleted, but the people who left reviews now cannot do so again [on that plugin]. They left a review, it got deleted, and now, they have lost their opportunity.”
This is rather sad; I can easily explain why their review was deleted, “I unknowingly broke a guideline and offered a discount for reviews…the .org team couldn’t conclude if your review was one of them so all were deleted…”, I can’t easily explain how that relates to them now that they want to change/edit or resubmit a new review.
Indefensible?
Apparently not.
I’ve tried. I made my points clear, maybe not concise, but I was nice and respectful throughout (except at one point when I said the response was “vitriolic” but I quickly apologized and pointed out the hypocrisy since I had/have such an issue with “bribery”). I shared that I would respect their decision but I would like for the decision to be fair.
I’ve been told there’s a “zero tolerance policy” for compensated reviews. Regardless if the policy in question is undefined and it’s unknowingly broken. It’s [scare quotes]their system[/scare quotes], which I get — to a point.
I just don’t think the result is fair, even if my opinion is that some reviews can now be deemed spam.
And as a plugin author I shouldn’t be nervous that something I’m currently doing is later deemed malicious, i.e. asking for a review at all or advertising a pro version.
What would have been the result if I read the Make blog post from a month ago?
Nothing! This is the most depressing part of this entire incident: the thought nothing would have happened if I learned about this before last Tuesday is maddening. Sigh.
What next?
- I somehow stay proud of all the hard work I’ve poured into the project — which those reviews proved but are now deemed “dishonest”.
- Track whether ratings affect for download counts; spoiler: they do and I’ll write about that later.
- I somehow prove that some changes need to be made (including the process) and my suggestions below are considered.
What I’d like to see (i.e. suggestions):
- The guidelines updated to cover compensated reviews.
- Clearly defining what is and what isn’t considered.
- Possibly the repercussions; not a first/second offense plan but a “we will delete all reviews if we find you’ve broken these rules, and without warning.“.
- Expectations for plugin authors to keep up to date should be provided, unless a written warning is to be expected now.
- A disclaimer for all below every review submission to not review the plugin if: “You have or will be compensated in anyway”.
- Other disclaimers that all plugin authors have been begging for can also be added, i.e. “A review is a place to ask for support, or demand a feature not present”.
Personally…
This entire incident still really bums me out, to the point of wanting to throw my hands up and walking away; is hard to find joy in my work at the moment. It’s hard to reconcile that I can still be proud of all the hard work if all these opinions/reviews are labeled dishonest from a respected team. Being accused of “bribing” my users, and “defrauding a system” that I respect and I feel I’ve contributed to, is rough.
Should you (and I) be outraged?
Haha, no. I’m upset by their decision and tact but that doesn’t mean much more than I hope I prove a point so things can change for the better. I also care about people and can see past our disagreements to enjoy a beer in the (hopefully near) future.
So don’t make this out to be fuel for your own personal dumpster fire against .org, or the community. This is my rant to try to improve things for everyone.
Notes:
- although I understand that the larger repercussions are yet to come ↩
I feel you man… I just can’t get the approach of taking your plugin down first, and then contacting you. My plugin was pulled out from the repo three times since 2011, all of them related to things that I was just not aware of (and I did read the guidelines).
Thanks for sharing this Dan! I’m so sorry to hear about this. It’s shocking and quite scary. I sincerely hope that you are given some of the reviews back. Deleting your reviews seems an unreasonably excessive reaction IMO.
have thought logic would indicate this isn’t going to be allowed.
How so?
I’m sorry you have had this kind of set back Dan.
That being said, I think that given what you did the punishment is justified.
You can argue the semantics about whether there was in a rule in place preventing this or not, but WordPress.org cannot have a rule in place for every possible situation.
You gained an unfair advantage over competitors in order to get higher rankings, more traffic, and more downloads. This ultimately led you to making more money.
Do you think it is fair that your technique helped you get ahead in front of other plugin developers?
The fact is that a reviews system is based on quality and integrity. If developers are allowed to manipulate reviews then the whole review system becomes pointless.
I don’t enjoy the fact you have had this setback, but I’m also surprised by how you reacted. I think it’s silly to expect WordPress.org to have rules for every single possibility.
Next we will see malware spammers pleading innocence because they didn’t realise putting injections into databases was against the rules.
Perhaps that’s an extreme example, but I hope you can see my point.
It’s no different to Google applying penalties to websites that use black hat SEO. What you did manipulated the review system and hurt its integrity.
Again, I stress that I don’t enjoy seeing your business suffer as a result of this, but when I think about all of this impartially I feel that you forced WordPress.org into a corner by doing this.
Perhaps you are right that their response was a little heavy handed and a lighter punishment could have been applied. However, I’m not sure what they could have done to undo all the five star reviews you helped generate.
Kevin
I don’t expect the .org team to do anything but be fair and respectful.
I also explained that the type of compensation I provided for those 28 reviews was different much different than me “paying” for a review, or “free kittens”; the fact is that the people reviewing the free plugin used it, wanted to upgrade to the pro version because they liked it, and provided a review. The technique helped push someone that liked the plugin to review, simple as that, nothing more. There was no “bribery” or “gaming the system”.
I’ve also shared that the most upsetting part of this is that real people had their reviews lost, a few have reached out to me and I don’t have an good answer (which this post is trying to do).
How was there no bribery? How were you not gaming the system?
You were offering a discount to anyone who gave you a review. That kind of marketing almost guarantees five star reviews. That is highly unfair to other plugin developers.
I agree that legitimate reviews should not have been lost, however it is difficult to determine which are real reviews and which are not.
Think about how Google handles websites that use black hat SEO techniques. Do they simply remove the thousands of artificial links that were generated? No. What they do is apply a manual penalty so that their pages are displayed on page 50 of the SERPs instead of page 1.
You are expecting the .org team to be fair and respectful. In my opinion you were not doing the same to fellow plugin authors when you gave an incentive for a review.
I do realise you were not looking at things that way and were not trying to hurt competitors; you were simply trying to encourage more positive reviews. I do still think that you gained an unfair advantage in the marketplace because of what you did and you should have expected a response.
How wrong and short sighted this comment is.
I am very familiar with this topic and very opinionated too. Just for the record, I released a small library to easily ask for a review from users. I added an automatic EDD coupon code generation in the process. Immediately after that, I suffered the wrath of .org team :p (https://github.com/julien731/WP-Review-Me/issues/3).
As I said in the issue linked above, I understand their point of view and comply with the guidelines. However, there are two things that bother me.
I saw in the Make blog post the following sentence (in the comments): “[…] we’re hoping that appealing to the humanity of things and ‘don’t ruin this for everyone’ will be more beneficial”.
While this is definitely true, it can be interpreted both ways. Why couldn’t people appeal to their humanity by being honest, and at the same time benefit a small “reward”? Basically, it’s saying “if we allow that, people will f#@k us” instead of saying “if we allow that, people will use it wisely”. It’s a matter of how one sees “the humanity”.
I realize this argument might be quite controversial, though 😉
The second thing, which I think is harder to debate, is that it all starts with “Reaching out to new people and saying ‘please try and review’ inflates the number of reviews in an unnatural manner”.
How is that reaching out to new people? We’re talking about asking *actual* users for a favor. I insist on *actual* because the review prompt is displayed in the WordPress dashboard, by the plugin itself. This means it’s targeted directly to actual users that have installed the plugin and are currently using it.
And if you ask “what about users who were just giving the plugin a try” I’ll answer “so what?”.
If the user is trying the plugin and doesn’t like it, why the hell would s/he want a discount code on the pro version or on an addon? What’s to gain? Who wants a discount on something s/he has no intention to buy?
Just imagine the scenario: “Well, this plugin is crap, I definitely won’t use it and certainly won’t pay for it. Let’s leave a review to get a discount”. How stupid is that.
To conclude, I respect the .org opinion and how they want things to work, but it just doesn’t make sense to me.
Also, I personally find it a little diminishing. When you’re putting hours of honest work in a free product and some tells you “you can’t do that, we’ll delete a part of your work”, well, you feel like shit.
I know the .org team puts a lot more free time than me into things, but does it justify it all?
^^^
I agree with our views and I find the “reward” system as a good approach to ask users to review a plugin.
Honestly, I use a lot of plugins and am happy with many of them. But, I would hardly go and write a review about the plugin. There are two things
1. It’s already a popular plugin – how would my review help anyways?
2. The plugin isn’t extra-ordinary or I am too lazy.
Getting genuine reviews from users is a big challenge for plugin and theme developers. I have seen a lot of people misuse the review system to share their hatred against a theme / plugin when they do not understand it completely.
So, more people tend to share negative reviews as compared to the positive ones.
A similar discussion came in 2 years back on AWP facebook group that plugin’s install / download count increases over the time but, reviews are hardly 1% of the total count.
Here’s a post that I wrote https://iampuneet.com/free-plugin-theme-review/
As far as incentives are considered, I don’t see them as a case of “bribery”. Julie makes a valid point there, only those who like your plugin will review it to upgrade for a discount.
Oh, well, let’s not talk about how Jetpack does everything!
Well, that’s some bullshit right there, Dan. I’m sorry that happened. Rather than seeing this as an opportunity to improve their systems, the team came down on you instead. No bueno.
SI powers my business, so I can tell you and anyone else who will listen that it’s a well-built product.
You have a good take on this, though. I admire that. Keep up the good work, buddy.
Dan… I am so sorry to hear about this and agree with Alex 100%. You have a fantastic perspective on this and even in your frustration (which is equally shared) it comes across as an opportunity to improve communication within the .org and the broader WP community. It’s amazing the power of communication and how when it’s done properly everyone benefits (not just one party).
[…] the professional license of his plugin for customers who gave a review on WordPress.org. In a post expressing his frustration with the way the situation was handled, Cameron said he “figured […]