Objections on the "Hello World" benchmark

http://www.nabble.co…td19914787.html

http://groups.google…ab14fbcd9201dd#

Well, I think there’s something true in some objections: if a framework is faster with a “Hello World” maybe it could be slower otherwere (ex. db management etc.). However, maybe you want to replicate to some of them  ;)

I've seen you already replied to the last discussion I've linked

Quote

Well, I think there's something true in some objections: if a framework is faster with a "Hello World" maybe it could be slower otherwere (ex. db management etc.).

I can't see the logic behind this conclusion. According to this conclusion, if a framework needs 10 seconds to run "hello world", it may still be faster than other frameworks in more complex scenarios?

Remember that the “hello world” benchmark reveals the RPS upper limit of each framework. Do not confuse it with real world RPS numbers. Theoretically, the RPS upper limit indicates the minimum framework overhead.

Real world RPS numbers are affected by many more factors, and they can hardly be compared in a fair way between frameworks. For example, in Paul’s benchmarks, he tries to display “hello world” using framework’s view rendering logic. However, each framework may have several ways to optimize the view rendering process (e.g. using page or fragment caching). This indeed makes his conclusions not very meaningful unless he explores every possible optimization for every framework.

The RPS upper limit alone is not the whole story. We still need to consider the feature set. Otherwise, why don’t we just use plain HTML or PHP? The fact is that Yii, while having the small overhead, has a much richer feature set than CakePHP and many other frameworks.

So will Yii be slower than others in real world scenarios? I am very confident to say it won't, because I have read through the code of nearly every popular PHP framework. Of course, we still need numbers to support my conclusion.

Quote

Quote

Well, I think there's something true in some objections: if a framework is faster with a "Hello World" maybe it could be slower otherwere (ex. db management etc.).

I can't see the logic behind this conclusion. According to this conclusion, if a framework needs 10 seconds to run "hello world", it may still be faster than other frameworks in more complex scenarios?

Well, you could have an app that uses 2 seconds for Hello World and 50 seconds for a db fetch, and an app that uses 10 seconds for Hello World and 20 seconds for a db fetch. So yes, it could be.

Quote

Remember that the "hello world" benchmark reveals the RPS upper limit of each framework. Do not confuse it with real world RPS numbers. Theoretically, the RPS upper limit indicates the minimum framework overhead.

Real world RPS numbers are affected by many more factors, and they can hardly be compared in a fair way between frameworks. For example, in Paul’s benchmarks, he tries to display “hello world” using framework’s view rendering logic. However, each framework may have several ways to optimize the view rendering process (e.g. using page or fragment caching). This indeed makes his conclusions not very meaningful unless he explores every possible optimization for every framework.

The RPS upper limit alone is not the whole story. We still need to consider the feature set. Otherwise, why don’t we just use plain HTML or PHP? The fact is that Yii, while having the small overhead, has a much richer feature set than CakePHP and many other frameworks.

So will Yii be slower than others in real world scenarios? I am very confident to say it won't, because I have read through the code of nearly every popular PHP framework. Of course, we still need numbers to support my conclusion.

This is reasonable :)

Quote

Quote

Quote

Well, I think there's something true in some objections: if a framework is faster with a "Hello World" maybe it could be slower otherwere (ex. db management etc.).

I can't see the logic behind this conclusion. According to this conclusion, if a framework needs 10 seconds to run "hello world", it may still be faster than other frameworks in more complex scenarios?

Well, you could have an app that uses 2 seconds for Hello World and 50 seconds for a db fetch, and an app that uses 10 seconds for Hello World and 20 seconds for a db fetch. So yes, it could be.

Yeah, that's still possible. But then would you choose such a framework that it takes at least 10 seconds to do anything? Of course 10 seconds sound exaggerating, but for a really large-scale Web application, this upper-limit number is of particular importance.