How to import from RSS feeds protected with CloudFlare

WP Automatic can import from RSS feeds to WordPress posts but sometimes sites use CloudFlare protection which makes WordPress get a CloudFlare page instead of the real feed items. Fortunately there is a workaround with the help of the multi-page scraper module and the apify service

for example, if we try to normally post from this RSS feed https://coinfomania.com/feed/, it returns this error

A feed could not be found at `https://coinfomania.com/feed/`; the status code is `403` and content-type is `text/html; charset=UTF-8`

to know what the plugin get exactly when trying to load this feed URL, we can create a multi-page scraper campaign, add the feed URL and click the locate button to load the visual selector

here is what we get, a cloudFlare blocked page

The workaround to this issue is to:-

1- Create the multi-page scraper campaign, set it to extract by REGEX

<item>.*?<link>(.*?)</link>

2- Make sure that you have activated using APIFY option and set up your APIFY token on the plugin settings page if you did not already set it up, check this tutorial on how to set it up

3- If the site shows a cloudFlare page for single post as well, you will need to enable this option

our test result for this feed

 

Leave a Reply

Your email address will not be published.