[mindplay]

I have worked with the PHP tokenizer many times over the years, and it occurred to me, I can probably save us a lot of time migrating the codebase to use namespaces - since the current framework is already neatly organized in folders.

So I hacked out this script:

https://gist.github.com/1126748

This will scan through the entire Yii codebase and rewrite the scripts, adding a namespace declaration and use-statements as needed.

For example, here's an example of CRegularExpressionValidator after the treatment:

<?php

namespace Yii\Validators;

use Yii\Validators\CValidator;
use Yii\Base\CException;
use Yii\Yii;

/**
 * CRegularExpressionValidator class file.
 *
 * @author Qiang Xue <qiang.xue@gmail.com>
 * @link http://www.yiiframework.com/
 * @copyright Copyright &copy; 2008-2010 Yii Software LLC
 * @license http://www.yiiframework.com/license/
 */

/**
 * CRegularExpressionValidator validates that the attribute value matches to the specified {@link pattern regular expression}.
 *
 * @author Qiang Xue <qiang.xue@gmail.com>
 * @version $Id: CRegularExpressionValidator.php 1678 2010-01-07 21:02:00Z qiang.xue $
 * @package system.validators
 * @since 1.0
 */
class CRegularExpressionValidator extends CValidator
{
	/**
	 * @var string the regular expression to be matched with
	 */
	public $pattern;
	/**
	 * @var boolean whether the attribute value can be null or empty. Defaults to true,
	 * meaning that if the attribute is empty, it is considered valid.
	 */
	public $allowEmpty=true;

	/**
	 * Validates the attribute of the object.
	 * If there is any error, the error message is added to the object.
	 * @param CModel the object being validated
	 * @param string the attribute being validated
	 */
	protected function validateAttribute($object,$attribute)
	{
		$value=$object->$attribute;
		if($this->allowEmpty && $this->isEmpty($value))
			return;
		if($this->pattern===null)
			throw new CException(Yii::t('yii','The "pattern" property must be specified with a valid regular expression.'));
		if(!preg_match($this->pattern,$value))
		{
			$message=$this->message!==null?$this->message:Yii::t('yii','{attribute} is invalid.');
			$this->addError($object,$attribute,$message);
		}
	}
}

It seems to work pretty well already, although I'm not going to say this is complete yet - and of course it currently just does the preview of the processed individual files, it does not write out the files to disk yet.

What do you think?

[mindplay]

Any interest?

If so, I would try to finish this - if not...

[samdark]

Can be very handy if we'll decide to start with existing codebase.

[qiang]

This is interesting. However, it is probably of little use because we are not migrating, but rewriting for 2.0. There are many changes, including names and coding styles.

Your code actually inspired me with an idea. We have talked about dividing the core into packages so that people can choose to uninstall some to reduce the size. I am generally against this strategy because of extra efforts to be needed by both core team and end users. I think instead of that, we can have a finer way to remove unneeded code. For example, we can have a script like yours, that will analyze the code of a Yii application, find out what Yii classes are not involved during, and then give suggestions for removing them.

Here are the basic steps I am thinking of that this script might need:
1. Determine the included core classes. This may require users to access his application in different ways, trying to cover most usages.
2. Analyze the included core classes, and find out referenced core classes (e.g. Exception, which may not be covered in step 1)
3. Give suggestions of unneeded classes and possibly delete them upon confirmation

Step 1 is the most difficult. It can be coupled with developer testing process (anyway, the developer has to test every code he wrote.)
Step 2 may need some special handling because some core class dependency may not be obvious.

This approach may result in finest and leanest core code base for a deployment.

Suggestions? Objections?

[samdark]

qiang
It solves distribution size problem but involves significant time to do app profiling. The problem with it is when one will need missing classes back for, for example, third-party extension package, he'll have to repeat the whole process.

Packages are different. Extension package will have requirements in its config so installer will be able to add missing parts if needed. From the other side, having tons of packages inside of the core will make third-party packages writing complex because of the need to specify a lot of core dependencies.

[qiang]

Yeah, I understand what packages can do. As you said, the trouble of having too many packages in the core perhaps will overshadow its benefit. And you're right that step 1 of my proposal is not trivial. I was just proposing some wild thinking.

[mindplay]

It is possible, for the most part, to do dependency analysis on PHP code statically, rather than trying to detect dependencies at run-time, by running all the code - that sounds more like code coverage than dependency analysis.

I don't know, I'm not sure I like the idea of simply removing classes from a framework once the app is ready for deployment. Suppose you continue development of the app and you realize you needed some of them?

Starting completely from scratch sounds like a lot of work... why not take from the original codebase what's already there and works well?

My script could be used to rename classes, by the way - and fix references throughout the codebase, etc.

It can't fix the code-style of course, but there are plenty of IDEs and source-code formatting tools that can reformat source-code to fit a particular style.

It sounds like you're going to write a whole different framework? I don't fully understand why. You didn't make any mistakes that are terrible enough to justify scrapping your entire codebase - so what is Yii 2.0 going to be, an entirely different architecture?

[qiang]

No, not throwing away old code. As you said, we are on the right track. There's no need to throw away known successes and turn to something unknown.
The main work will be going through much of the existing code line by line, polishing them and conforming to some new standards. There are some major new development/re-implementation work as well, such as AR. The automatic script may help, but it will take significant amount of efforts to fine tune it, and I doubt if it worths the effort.

[mindplay]

 qiang, on 06 August 2011 - 07:17 PM, said:

There are some major new development/re-implementation work as well, such as AR.

Just curious, what are your reasons for wanting to rewrite AR?

There are numerous complete/stable AR (and other ORM) implementations for PHP in the wild - have you looked at them all, and what are your main reasons for wanting to write your own again?

In my opinion, your time could be much better spent on the framework itself - persistence is without a doubt going to take up much (if not most) of your time... I personally am much more interested in what you will be doing with the framework itself, than yet another AR implementation... But that's just me :-)

[qiang]

Note that 2.0 is not just about adding namespaces to core classes. We want to take this chance to solve some problems that would be possible to solve in a minor version.

There are a few things in 2.0 that needs major work (this doesn't mean the usage/API will change significantly): AR, client script management, base javascript code, module/package.

There's no reason we should embrace a 3rd party AR implementation while our own is already superior and has been used by many more users than any others (note I'm not talking about ORM). IMO, AR is one of the most effective ways for agile Web development, which is one of the main goals of Yii, and that's why we should devote more effort to make it better, rather than drop it. Through our past experience and user feedback, we have spotted several places that we need to improve AR. While the API will remain largely unchanged, the underlying implementation will change significantly. We hope the new AR gives users more flexibility and better performance.

[waitforit]

I am not an accomplished developer like you folks here but I have been reading these threads with great interest and I would definitely like to see certain areas of AR enhanced, particularly relational AR.

The biggest frustration as a new Yii user was trying to implement saving of related records, especially MANY_MANY. There are some GREAT extensions for this but I would really like to see a 'Yii way' of doing it.

[mindplay]

 qiang, on 08 August 2011 - 09:49 AM, said:

Note that 2.0 is not just about adding namespaces to core classes. We want to take this chance to solve some problems that would be possible to solve in a minor version.

Of course - I just figured this might be a helpful boost to get over one of the more tedious hurdles. Manually correcting class-names and fixing references isn't a very good use of anybody's time.

Either way, I'm porting one of my own codebases to use namespaces with this tool :-)

 qiang, on 08 August 2011 - 09:49 AM, said:

There are a few things in 2.0 that needs major work (this doesn't mean the usage/API will change significantly): AR, client script management, base javascript code, module/package.

There's no reason we should embrace a 3rd party AR implementation while our own is already superior and has been used by many more users than any others (note I'm not talking about ORM). IMO, AR is one of the most effective ways for agile Web development, which is one of the main goals of Yii, and that's why we should devote more effort to make it better, rather than drop it. Through our past experience and user feedback, we have spotted several places that we need to improve AR. While the API will remain largely unchanged, the underlying implementation will change significantly. We hope the new AR gives users more flexibility and better performance.

Ah, well that makes sense - I thought you were saying you wanted to start over from scratch; which would be crazy. But rebuilding with a stronger and more optimized architecture based on proven ideas, while maintaining a similar API, that sounds good to me! Thank you for clarifying. :-)

One architectural issue I would highlight then, is the idea of maintaining a single "pseudo-static" instance of the model-type itself - I don't like that, and it has caused me problems more than once. (for example recently, a coworker needed to first count and then select objects using the same criteria - but the criteria, being attached to the static instance, gets cleared out after the count. another example is when you're building a query and need to perform another query involving the same type - because there is only one static instance of each "criteria builder" for every type, this sometimes means that lots of refactoring is needed to make a simple addition to an existing codebase...)

I submit that you're mixing two different areas of responsibility: the "object relational mapper", e.g. the component that is responsible for schema reflection, translating to/from SQL values, building queries, etc. - and the model type itself.

Do you plan on separating those areas of responsibility in the new architecture, or will they remain mixed-up as they are now?