{"id":2782,"date":"2019-10-23T12:32:00","date_gmt":"2019-10-23T04:32:00","guid":{"rendered":"https:\/\/www.mondoze.com\/guide\/?post_type=kb&#038;p=2782"},"modified":"2022-10-05T08:01:42","modified_gmt":"2022-10-05T00:01:42","slug":"how-do-i-block-bots-and-crawlers","status":"publish","type":"kb","link":"https:\/\/www.mondoze.com\/guide\/kb\/how-do-i-block-bots-and-crawlers","title":{"rendered":"How do I block bots and crawlers?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"2782\" class=\"elementor elementor-2782\" data-elementor-settings=\"[]\">\n\t\t\t\t\t\t<div class=\"elementor-inner\">\n\t\t\t\t\t\t\t<div class=\"elementor-section-wrap\">\n\t\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-36d8becd elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"36d8becd\" data-element_type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t\t\t<div class=\"elementor-row\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-28c4cb74\" data-id=\"28c4cb74\" data-element_type=\"column\">\n\t\t\t<div class=\"elementor-column-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t<div class=\"elementor-widget-wrap\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-7cd16ac elementor-widget elementor-widget-heading\" data-id=\"7cd16ac\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">How To Block Bots and Crawlers<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2c171006 elementor-widget elementor-widget-text-editor\" data-id=\"2c171006\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t<div class=\"elementor-text-editor elementor-clearfix\">\n\t\t\t\t<p>Cloudflare&#8217;s data sources will help reduce the number of bad bots and crawlers hitting your site automatically block bots and crawlers (not all). If you would like to place a block of your own for a bad bot or crawler, you can block them by IP in your Firewall app.\u00a0<\/p><p>In addition, customers that are on a paid plan can look at turning on Cloudflare&#8217;s\u00a0Web Application Firewall (WAF)\u00a0to further help reduce the threat of bad bots and crawlers that don&#8217;t follow good behavior guidelines.<\/p>\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>How To Block Bots and Crawlers Cloudflare&#8217;s data sources will help reduce the number of bad bots and crawlers hitting your site automatically block bots and crawlers (not all). If you would like to place a block of your own for a bad bot or crawler, you can block them by IP in your Firewall &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/www.mondoze.com\/guide\/kb\/how-do-i-block-bots-and-crawlers\"> <span class=\"screen-reader-text\">How do I block bots and crawlers?<\/span> Read More \u00bb<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-gradient":""}}},"kbtopic":[53],"kbtag":[110],"mkb_version":[],"_links":{"self":[{"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kb\/2782"}],"collection":[{"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kb"}],"about":[{"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/types\/kb"}],"author":[{"embeddable":true,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/comments?post=2782"}],"version-history":[{"count":6,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kb\/2782\/revisions"}],"predecessor-version":[{"id":18845,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kb\/2782\/revisions\/18845"}],"wp:attachment":[{"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/media?parent=2782"}],"wp:term":[{"taxonomy":"kbtopic","embeddable":true,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kbtopic?post=2782"},{"taxonomy":"kbtag","embeddable":true,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/kbtag?post=2782"},{"taxonomy":"mkb_version","embeddable":true,"href":"https:\/\/www.mondoze.com\/guide\/wp-json\/wp\/v2\/mkb_version?post=2782"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}