Yii 1.1 vs 1.0, versus Kohana 2/3 performance comparision
Posted 15 November 2009 - 12:39 PM
I'm looking for some benchmark comparing performance of mentioned frameworks... is there any? If I choose Yii (which I'll probably do) which version would be better?
I had experience with Kohana v2 which unfortunately seems to be too slow for us (each evening we have 40 000 users active in 3 minutes) so I had to rewrite parts of the code to "pure" PHP... which I'd like to avoid this time... Therefore I'm looking for the most optimal framework.
Thanks in advance,
Posted 16 November 2009 - 04:16 AM
Kohana vs Yii - More than you ever wanted to read
I have used Kohana v2 at work for the past year and have felt like I have had more problems with it than help. I have used Yii for the last 6 months on personal projects, and have been happy with everything I have done. It should be noted, however, that at work we are stretching Kohana's abilities in a very unique way.
Advantages of Kohana
- The Cascading File System allows easy overwriting of core functionality
- Hooks allow for quick and easy handling of core events
- It has a large active community and lots of contributed content
- Query builder is easy to use and easy to read
Advantages of Yii
- Is much lighter on your servers
- Extensions are self-contained and easy to use
- Application Components provide for easy customization of framework functionality
- Auto loading and file import are extremely quick
- The Active Record and DAO systems are fast and concise
- Uses PHPUnit for unit testings (1.1+)
- Memory footprint is small
- Core code is clear and concise. Easy to read and decipher.
PAGE EXECUTION/SERVER LOAD
Yii is faster in all of my benchmarks, but the difference in time is insignificant on a decent machine. The real bottle neck comes from the database not PHP. Server load with many concurrent connections was greater for Kohana than it was for Yii.
Kohana has a very power feature they call the Cascading File System (CFS), that allows you to easily and quickly override core system functionality without having to change any code currently inheriting from those system files. This feature is extremely useful, but does come with a performance cost of having to scan many directories to find a single file.
Kohana also has hooks (basically event handlers) which allow you to 'hook' into system events for custom functionality. This hooking functionality is extremely useful for internationalization, error logging, etc.
To override Yii core classes, you must create a child class and override specific methods. This is (when you get down to it), the same as what Kohana does but Kohana handles it for you. Because you must override the classes with a new class name, if you decide you need to override core functionality (like the render function) for all controllers you currently have, there is more coding involved with doing so.
However, Yii's application components do allow you to easily and quickly override the system components quickly and easily. Just create a new child class of the system component, then change your config to use the new child class instead. The application components also allow you to add custom components (such as 3rd party extensions) easily to your application and to make them quickly accessible both to PHP and to the programmer.
THIRD PARTY CODE
Kohana has a much larger repository of third party modules than Yii does. However, Kohana has also been around longer and, as such, has a much larger community.
Kohana has a module system that allows you to bring in 3rd party code and seamlessly plug it in to your system with only 1 config file change. It has come in useful to my company several times, but has also caused problems due to poor programming in the modules. Because of the CFS if a module's file has the same name as one of your system files, there will be a "collision" and only the first file found will ever be loaded. Thus either making your code or their code not work properly.
It should also be noted that a Kohana Module is not a self contained Module. Modules simply extend from the base filesystem and essentially become more a method of grouping code into functional units than creating self contained projects. (I could go on about this, but there is no need here. Ask and I can elaborate).
Yii extensions are easy to bring in as either a custom application component or to simply import as use at will. They are able to be (and should always be) completely self contained. This containment allows for you to write your application however you want without fear that you are going to conflict with third party code. It also makes it very easy to create your own extensions for distribution to the community.
Also, because you can simply use a Yii::import() call to pull in a specific file from an extension when you need it, you are able to quickly pull in any file you want.
Yii path aliases also allow for short path names to both preconfigured and custom locations inside of a project.
Kohana currently support the following databases: MsSQL, MySQL, MySQLi, PostgreSQL, and PDOSqlite. Support for each of the database systems is derived from a "database driver", written in PHP, in the core code. This allows the drivers to be easily override (using the CFS) and custom functionality added to them.
All of Kohana's prepared statements are handled by a PHP implementation that does not support named parameters. (I know this is true for v2. I think v3 may be using PDO, but I am not sure).
Kohana has a query builder that is very flexible. It can be used on its own, or coupled with their ORM system to help make data accessing easier. It does not do advanced database functions - like forcing keys - but will build most queries easily and correctly.
Kohana also has a fairly robust ORM system that allows you to quickly and easily create code to do your basic CRUD (literally can be done with one line of PHP). The ORM also has built in functionality for retrieving data from related tables, once the relations are set up in the ORM object's code (a simple array).
The ORM, however, has one major flaw. All of its code is based around the assumption that your Primary Key (PKey) is an auto_incrementing integer named 'id'. You can override the PKey name in the ORM class for a table but this value is almost completely ignore in their relational database system. This, literally, cost my company thousands of dollars in developer time to make it work with PKeys not named 'id'.
The most glaring flaw with Kohana's databasing is the speed. The root cause behind Kohana's speed issue is that it does a result_seek every time the current item is retrieved from the Result object. Iterating through a 2000 item result set takes upwards of 30 seconds. This makes their system all but unusable for anything requiring large amounts of data. (I have written a database driver for MySQL that doesn't have this problem and can be supplied if requested - it is 300% faster than the default driver).
Yii uses PDO at the core of it's database system. As such, all prepared statements are handled by the database itself and are much more optimized for reuse and execution. This offers of significant performance increase for data processing scripts.
Yii doesn't really have a query builder. It uses a Database Criteria (CDbCriteria) to store query information that is then compiled into a query later on. It can be used as a query builder (and is used by the Active Record system for just that), but is not something I would call a query builder. As of 1.0.10 there are many more useful functions that help the DbCriteria be more programmer friendly and usable.
Yii's Active Record (AR) system is just as easy to set up as Kohana's (easier with the scaffolding) and is orders of magnitudes faster. The AR system supports relational databases and is built around the concept of Database Access Objects (DAO) and ties directly to the PDO. And because it is using the DAO, there is not extra overhead for query building and relational calculations.
However, because the DAO system (behind the scenes) puts the entire result set into an array, Yii's built-in functionality is not optimal for use with extremely large data sets.
Kohana has a unit test modules that can easily be added to any current project. Because of the way it is structured it poses very little risk of file collision in the CFS. It is easy to use and has a very good output. It is, however, designed to be run in a browser window and, because of this, is restricted by execution time limits on the server. It has all of your basic Assert functionality.
Yii's unit testing is built around PHPUnit. It is included in the core (of 1.1+) and is very easy to create your own tests. Because it is using PHPUnit, it is meant to be used on the command line (as far as I know) and does not suffer from execution time limits. It is also not reliant on a potentially buggy proprietary system.
Kohana loads a lot of files into memory for every page execution. Because of this, it has a very large memory footprint compared to Yii. It also causes more load on the servers because of the number of files it must load just for configuration purposes. However, for the functionality that is offered, this is expected.
Yii has a very small memory footprint for a framework of its size (and much smaller than Kohana's). Because Yii's core functionality is broken into smaller logical parts, for any given page less files are loaded into memory to do what needs doing. This smaller footprint and logical grouping of parts, has a much smaller load on servers as well.
Posted 16 November 2009 - 05:28 AM
thank you very much for your response!
Actually I knew everything you wrote about Kohana, as I used it extensively for almost a year. In our case, the bottleneck wasn't db-querying related. Even the queries automatically built with ORM, which were using 2-3 tables at once (using mentioned relationships defined in models) were fast enough, and in case they were not, it wansn't a big deal to write it "manually". Also the indexes on database were correctly defined, so that part was OK. We had problems with server load, which was (I believe) caused by Kohana. Under the heavy load, we had lot's of PHP processes using quite a lot of CPU. Once again, it's not because of DB (there was almost no load on server running MySQL, it's responses were *very* fast). We ended up anyway with extensive caching, almost everything was loaded from cache.
After a lot of profiling it came out that the bottleneck was Kohana's core functionality. Like converting everything to utf8, loading configs, initializing controllers etc. That is why your advices and comparision on core functionality load is extremely helpful for me, probably I'm not going to use Kohana this time.
Could you however tell me something more about Yii databases support? I must admit that ORM in Kohana speeded up development a lot. It was used almost *everywhere*, we were writing queries "manually" only when extra performance was needed. As I wrote above, this was ok as for performance (especially since Kohana introduced "with" parameter which caused forcing JOINS)... Is the Yii Relational Active Record as usable as Kohana's ORM?
Thanks in advance!
Posted 16 November 2009 - 12:43 PM
The AR system in Yii is just as fast to use and Kohana ORM. It is, however, much more optimal with how it builds queries. I've been through almost every line of the Kohana ORM and it's bloated beyond what it really needs. The AR system does everything the Kohana ORM system can do (at least in my usage scenarios) and it does it faster.
Because Yii is tied directly to PDO, it also gives you the ability to reuse your query statements (CDbCommand). This is useful because the database server does not have to re-parse the query every time you run it, it just puts the new values into the already compiled statement. This can significantly improve performance when you are doing a lot of individual insert queries, but has an insignificant effect with most regular usage.
Yii is a bit different with how you can use the AR to and conditionals to your queries. Instead of extending the query builder functionality (like Kohana does), it provides you with a large list of find* methods: find, findAll, findAllByAttribute, findAllByPk, findAllBySql, findByAttribute, findByPk, findBySql. As such, instead of chaining on where() calls, you use find*ByAttribute method and pass the conditions in an array. In general, I find this easier to read than the query builder chains.
And, as of this writing, I have discovered that my statement about Yii always putting the entire result set into an array before you can manipulate it was incorrect. There is the CDbDataReader class that is initialized with an instance of a CDbCommand and then allows you to iterate through the result set manually. This DataReader class, however, is not natively used in the AR system. You can only take advantage of it through DAO or if you fetch the DbCriteria from the AR and then use it inside DAO. However, if you are using the AR to handle large result sets you will have other areas of concern (like memory usage and object creation overhead).
All in all, I feel like the Yii AR allows the programmer much more control over what is going on than the Kohana ORM does. It also does everything it does much faster. As long as you remember how to properly use find() - because I often revert to Kohana mode and use them wrong - then I highly recommend the Yii system.
Does that cover what you were wondering about?
Posted 16 November 2009 - 01:15 PM
I know that, just didn't mentioned, thanks
I had the same feeling when finished reading docs related to AR - in fact I missed something like find*BySql in Kohana ... especially since Kohana doesn't allow custom SQL queries for ORM, one is forced to use limited set of functionality provided by mentioned chaining methods...
Thank you very much again, that is definitely much more than I expected to read here!!
Posted 16 November 2009 - 10:22 PM
Yep, my bad. I think I will edit the post to reflect that. Nice catch, thanks. Had so many Kohana and Yii things going through my head and in-laws talking to me in the background. I'm surprised I didn't make more mistakes like that.