Merhaba arkadaşlar sizce bu robots.tx sizceyeterli mi karar veremedim. Yapmam gereken değişiklik var mı ?
Kod:
User-agent: Googlebot
User-agent: googlebot-image
User-agent: googlebot-mobile
User-agent: MSNBot
User-agent: Slurp
User-agent: Teoma
User-agent: Gigabot
User-agent: Robozilla
User-agent: Nutch
User-agent: ia_archiver
User-agent: baiduspider
User-agent: naverbot
User-agent: yeti
User-agent: yahoo-mmcrawler
User-agent: psbot
User-agent: yahoo-blogs/v3.9
User-agent: *
Disallow: /threads/*/reply
User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /hesap/
Disallow: /members/
Disallow: /kullanicilar/
Disallow: /attachments/
Disallow: /goto/
Disallow: /posts/
Disallow: /login/
Disallow: /admin.php
Disallow: /help?*
Disallow: /yardim?*
Disallow: /search/
Disallow: /arama/
Disallow: /misc/
Disallow: /install/
Disallow: /admin.php
Disallow: /yonlendirme?*
Disallow: /yonlendirme*
Disallow: /misc/cookies*
Allow: /
Sitemap: https://site.com.tr/sitemap.xml
