Escape from Bing Jail
My post about Bing Jail gained a bit of attention (a lot!). With the traction, there is hate, but I could expect that as well. There was a bit of conversation in the comments on Hacker News which gives me valuable feedback.
This short, post scriptum post is not an explanation but rather my experience on the following days. My site reappear in Bing search for one day to be manually removed (blocked) the day after where Bing Support, who surprisingly responded, start looking into the case.
Just 24 hours after my post I find out that my site start appearing in Bing and consequently DuckDuckGo once again.
From a typical (small) number of users visiting my website daily, I gained over 7.79K in just a couple of evening hours (UK time) on a post date followed by over 9.32K a day after. This traction has some force!
Blocklists
Some users, in Hacker News comments, pointed out that my site is blocked by some adblockers. Once I investigated that, it all went into Polish Annoyance Filters where my site was added at some point in 2019. This probably does not have a direct impact on Bing being blacklisted three years later but since 2019 lot has changed. There were other filters where my site was added, or simply blindly copied from other lists without checking. I am not surprised if Bing Team do the same.
In 2019 my site was WordPress based and been serving some Google Ads. Since I moved to Hugo in late-2020 a lot on the website itself changed. It become much clearer that the content and user experience are more important than scraps that I received from advertisements, the scraps that overall kill users’ engagement.
Resignation from serving ads was also because my site is served now on Netlify which does not cost me as much as WordPress hosting in the past.
I have contacted the maintainers of the lists in a couple of repositories on GitHub and worked together to remove them.
Even after removal, not all the changes will happen straight away but after just a couple of days, I shall observe change, especially on AdGuard list 237 which was updated on the 25th of April 2023 with my website not been there.
AdBlock Team upon contact been also helpful to point me in the right direction to get my site removed from the PAF list.
At that time I thought that I escaped from Bing Jail as my site reappeared in full, but did I?
As I have no response from Bing Team, my post must reach somebody who knew somebody who decided to check that. Possibly Bing is using AdBlock filter lists (I doubt it) to filter their search results from mid-January 2023, especially for non-English websites. Without any feedback, we will never know what’s going on here.
When I inspected my homepage (Polish) in Bing Webmaster Tools URL Inspection, Bing Index do not report an issue like previously, instead returned green ticks stating Indexed successfully.
The first page that appears back in Bing was indexed at 18:44 UK time on 24th April 2023. That’s just not long after a team from Polish Annoyance Filters published their repo with my website removed from their list. Coincidence?
Yes, this may be a coincidence, but as we know Bing went through some changes in the middle of January 2023 so maybe one of the changes (not acknowledged) was the use of popular AdBlock filters to clean their indexed pages.
Finding if your site has been added to any block lists over the Internet is like searching for a needle in hay stag. Only through some helpful people who left comments on Hacker News posts, I have been able to act upon it. But how others can fight that?
On the next day, the 25th, I noticed that a lot more pages starts appearing in Bing results as well. The chart in Webmaster Tools remains at zero, but, similar to Google Search Console charts, the update is typically a couple of days behind.
It doesn’t look like a coincidence when AdGuard list 237 was updated (
TimeUpdated: 2023-04-25T01:01:45+00:00) not far before more results start reappearing.
On the morning of 25th of April 2023, at 7:25 UK time Bing reported back 2,280 results when searched strictly by my site URL and on desktop, the results were fulfilled with rich snippets on a side.
Things (initially) changed thanks to all of you who read and shared my post and through the power of Hacker News itself.
Sadly, there are more stories about the same practices without any acknowledgement either in Bing Webmaster Tools or directly from Bing Team upon contact.
I am starting to think what is the sense of having a tool that does not display useful information, especially when you are rejected.
The whole story reminds me of what I learned when I first come to live in the United Kingdom back in 2010. In this country if you want to achieve something you need to get yourself heard, you need to ask, you need to speak and you need to complain!
A bit of optimism
On the morning of the 27th of April Bing Search Performance chart finally refreshed and showed what I had been seeing through earlier days.
If you look closer to the chart you will say that it doesn’t look normal. It’s not possible to spike from zero to 2.3K indexed pages overnight and you are right, it’s not possible unless the pages have been there but have been simply blocked.
The chart clearly shows that someone or something has been blocking my site from Bing and mentioned blacklisting exists. Looks like my post, through all of you, reached someone and I escaped from Bing Jail (once again, did I?), but there are a lot of other inmates (website) still there.
Nicholas A. Ferrell has been inspired by my Bing Jail post and he decided to share his story as well in post The story of our Bing blacklisting (take some time for reading in full, it’s worth it). I only hope that he will have his attention as I have and his website reappear in Bing in no time. Please share his story and help in the same way you helped me.
Failed prison break
I repeated twice the word “Did I?” and here is the answer.
My Prison Break ended in captivity again!
As I thought that issue had been sorted, I just checked on 27th April 2023 once again does Bing show any results for my site? Just a couple of hours earlier there were 2,280 results and today morning I was presented with information that Some results have been removed and my site disappeared once again from Bing results. Zero, nada!
I am angry again!
This time I was presented with a link to documents on how Bing delivers search results. Off course this link wasn’t really useful without any feedback from Bing Team.
That was before noon on 27th April 2023.
At 2 pm the hell freezes over.
Blacklisting confirmed
At 2:14 pm on 27th April 2023, my inbox landed with a response from Bing Team to my query REQ00056361. It looks like the attention gained by my Bing Jail post went in the right direction and reached the right people (did it?).
I already start giving up when I saw that my site disappeared from Bing the day after it bounced back.
But let’s get to the email response…
Straight away when I read the first sentence I shouted inside - Bing got hidden blacklisting in place!. Email state the following:
“Firstly, I am happy to provide you information that our Product Review Group succeeded in resolving the issue which prevented your site from showing in our index. After submitting your site to be reviewed, the team has decided to lift the block. Allow up to two to three weeks for your site to be crawled, indexed, and serving again.”
Straight away when I read it I composed a response email just to verify, is this, what I saw yesterday, was when the block was lifted? What happens today that my site is gone once again? Am I being blocked for a second time or it’s just how it’s working and it will take mentioned “up to two to three weeks for your site to be crawled, indexed, and serving again”?
I will need to monitor it further and patiently wait but I cannot believe that Microsoft indeed doing blacklisting without any acknowledgement. They have in their Webmaster Tools section called Security & Privacy with subsection Copyright Removal Notices. Why not add there any block information and violations so users will be aware and, if indeed the blocks are valid, they will be able to act upon them?
But are the blocks valid?
Reading the email further we got the following:
“I am unable to provide you the specifics of the block, as our Product Review team does not share the details of the block. However, I wish to help you with a few links and help docs, which will not only help in avoiding blocks, but also help you to rank better in the Bing index.”
Of course, they cannot “provide you the specifics of the block”, why would they? As I mentioned in my initial post, if that would happen to somebody with a lot of money to spare, this would finish in a lawsuit. I understand that Bing Index is a product that is offered for free, but even free products are bonded with some terms of use and some rules to obey by both parties. Implementing shady practices like this kind of block is unacceptable!
I have been provided with the following link which I decided to review to see if they will give me any clue of why this happened in the first place. I doubt that they will (like pointing to Bing Webmaster Guidelines) but will try.
- Why is My Site Not in the Index?
- 10 SEO Myths Reviewed
- Is SEO The Future? No, And Here’s Why
- Building Authority & Setting Expectations
- The Role of Content Quality in Bing Ranking
Before I will do that I looked at the next sentence in their response:
“When Bing crawls and indexes pages on the Internet, it makes decisions about which pages will most likely resonate with searchers, so we look at these signals relative to all other sites on the web as part of our selection criteria. Always ensure you follow webmaster guidelines, especially the section Things to Avoid.”
Seriously Bing? You think that my content does not “resonate with searchers”. Google Search users and results from analytics of my website will disagree!
I will look at the link under Things to Avoid once again later, but I doubt that I will find anything explaining this whole situation. Overall this points back to Bing Webmaster Guidances that I went through already.
Email finishes with:
“Let me know if you have any follow-up questions or concerns regarding this issue and I would be more than happy to assist. Have a wonderful day ahead!”
Of course, I got a follow-up question which I sent.
I didn’t expect a follow-up answer but, to my surprise, it arrived in my inbox at 9:50 (UK Time) on 28th April 2023 stating:
I am happy to inform you that the issue that prevented your site https://dariusz.wieckiewicz.org/en/ from showing in our index has been resolved.
It may take up to 2-3 weeks for your site to be crawled indexed and serving again. Additionally, you can submit your URL using the URL Submission feature to get them recrawled faster.
Please review our Webmaster Guidelines, especially the section Things to Avoid, to avoid this in the future.
Looks like I have been treated differently, as I got somebody responding to my emails.
This email however doesn’t change the fact that when I search my website through Bing still got notice that Some results have been removed and zero results.
It has been suggested to “submit your URL using URL Submission feature to get them recrawled faster” which I of course do to see if anything changes. No changes whatsoever!
At the time of checking, 28th April 2023 11:38 UK Time, the site (the /en/ as mentioned in the email) still showing as present in Bing Index despite not being displayed in search.
I pressed Request indexing button as well.
Once again I have been pointed to Things to Avoid section in Bing Webmaster Guidelines, “to avoid this in the future” which I will go into dept then to see if there are any signs of explanation.
I once again followed up with an email response, stating that “you can submit your URL using URL Submission feature to get them recrawled faster” was another nonsense. Crawling and indexing is nothing if you are still manually blocked.
There are 2.3k pages indexed but none of them appears in search as they are manually blocked. When you try to do your search you still got the following output:
“We request you to please wait for up to 2-3 weeks for your site to be crawled indexed and serving again.”
This was the last response I got from Bing Team on 29th April 2023. It looks like, independently of what I will do, I need to let it go for this period.
Nonsense Crawl Errors
The interesting thing was to see that together with a return of my pages to the index (despite not being displayed in search), there was a sudden spike in crawl errors. Even looking into 6 months of data, I never saw such a spike and I decided to look into that as well.
There is no easy, intuitive way of exploring the crawl errors in Bing Webmaster Tools. Instead, you need to go to Site Explorer and from Filter by choose one of two URLs with Crawling Issues option.
When I explored dead links there I noticed some links to /professionals/landscape-contractors or /products/bathroom-faucets.
Interestingly was that I never got this kind of page. My website wasn’t hacked either. The date when this page was last crawled states 25 Apr 2023 which is utter nonsense unless Bing got an issue with Netlify CDN.
Apart from that, I couldn’t get any more information, hence these “crawl errors”, despite that are “apparently” there, if you cannot get any feedback on what they are, you cannot act upon them. Nonsense!
Of course, I added these links to Block URLs to prevent them from relating to my website. This looks more like spam websites that point things to my site which shall not be a factor to penalise you in Bing Index. It looks like Bing got a big issue with that where Google is simply ignoring spam links to our websites.
Once I have been waiting for these “up to 2-3 weeks”, I decided once review some of the links mentioned earlier starting with Things to Avoid.
Review of Things to Avoid
At the beginning, we read:
“Following the guidelines will ensure your site plays by the rules and is not found to be spammy, which could lead to demotion or even delisting your website from Bing search results.”
I go through the page and when I reached Help Bing understand your pages I stopped at the Content paragraph that states:
“Content: Websites that are thin on content, show ads or affiliate links primarily or redirect visitors away to other sites quickly tend to drop in rankings on Bing. In some cases, it may not get indexed at all. Your content should be easy to navigate, rich and engaging to the website visitor, and provide the information they seek.”
My site is not content-thin and does not show ads or affiliate links. You may think that some links that I promote on the top of my Polish page are affiliate links but in reality, they are just promo links to my posts.
I am not using any shady redirects and posts are easy to navigate, rich and engaging to the website visitor. That is the main purpose of my website.
The problem is with Bing, as going through their Site Explorer I see that they are still, almost 3 years after, referring to some WordPress-specific pages that are no longer there.
My content is created for users, not search engines. I am trying to be unique and have my content-rich as much as possible. My images are optimised and filled with alt tags. The overall site is working with and without JavaScript and I am optimising it also with accessibility on my mind.
I am using all relevant HTML Tags and Semantic markups.
All is good here.
I reached the Abuse and Examples of Things to Avoid section.
I go through this and in my opinion, I do not violate any of the points there. The only part that brings my attention was the section Keyword Stuffing then I reviewed my <head> section of my website where I remember that I have such a thing as <meta name="keywords".
None of the valuable search engines using this anymore including Bing hence decided to remove this outdated code not only from my website (that is Blacklisted by Bing) but also from all others that I am maintaining (that don’t have an issue with Bing).
Apart from that I haven’t learned anything, especially any answers suggesting why Bing Ban was applied on my site in the first place.
Email from Bing Support may suggest that I concentrate too much on the SEO aspect rather than content, but in my opinion, it’s not the case (content is always what I am concentrating on). The problem is that Bing is still analysing websites based on technologies that have been used on them in the past and not what is currently there. Moving from WordPress to Hugo changes a lot, sadly Bing robots don’t have a clue and don’t tend to re-evaluate websites in that aspect very quickly. 3 years and still referring to WordPress links is a bit crazy!
I looked at the other links from the first Bing Support email but they also didn’t make anything clear.
At this stage, there is nothing else that I can do but wait this 2 to 3 weeks as suggested. In the meantime, I will be monitoring my presence in Bing Search and for any responses from Support. After 3 weeks if nothing will change I will ask a question once again hoping to get some more clues as to why I have been blacklisted in the first place.
Nicholas A. Ferrell, the editor of The New Leaf Journal, which is still suffering from Bing Jail, has created a helpful collection of articles about Bing Jail in his Not-So-Awesome Bing Search Bans and De-Indexing GitHub repository. It is worth taking a look.
