yii2-robots-txt Module for generate robots.txt file by url rules

Yii2 module for render robots.txt

  1. Installation
  2. Configuration example
  3. License

Installation

The preferred way to install this extension is through composer.

Install

Either run

$ php composer.phar require execut/yii2-robots-txt "dev-master"

or add

"execut/yii2-robots-txt": "dev-master"

to the `require` section of your composer.json file.

Configuration example

Add to application config folowing rules:

[
    'components' => [
        'urlManager' => [
            'rules' => [
                ['pattern' => 'robots', 'route' => 'robotsTxt/web/index', 'suffix' => '.txt'],
            ]
        ]
    ],
    'modules' => [
        'robotsTxt' => [
            'class' => 'execut\robotsTxt\Module',
            'components'    => [
                'generator' => [
                    'class' => \execut\robotsTxt\Generator::class,
                    'host' => 'localhost',
                    'sitemap' => 'sitemap.xml',
                    //or generate link through the url rules
                    'sitemap' => [
                        'sitemapModule/sitemapController/sitemapAction',
                    ],
                    'userAgent' => [
                        '*' => [
                            'Disallow' => [
                                'noIndexedHtmlFile.html',
                                [
                                    'notIndexedModule/noIndexedController/noIndexedAction',
                                    'noIndexedActionParam' => 'noIndexedActionParamValue',
                                ]
                            ],
                            'Allow' => [
                                //..
                            ],
                        ],
                        'BingBot' => [
                            'Sitemap' => '/sitemapSpecialForBing.xml',
                            'Disallow' => [
                                //..
                            ],
                            'Allow' => [
                                //..
                            ],
                        ],
                    ],
                ],
            ],
        ],
    ],
];
php

After configuration robots.txt is opened by link /robots.txt

License

yii2-robots-txt is released under the Apache License Version 2.0. See the bundled LICENSE.md for details.

0 0
1 follower
0 downloads
Yii Version: 2.0
License: Apache-2.0
Category: Others
Developed by: eXeCUT
Created on: Jul 13, 2016
Last updated: a year ago