[pestaa]

Being very careful about performance and speed in general, I'm planning to create a seamlessly integrated benchmarking tool in Yii. I'm looking for experienced contributors, as this project might require significant amount of precise work. With this, I want to reach the following points:

• If you are familiar with New Relic (performance monitor for RoR), you have already seen a very convenient user interface.
  
• I want to show as useful information to administrators as possible to help them identify and remove bottlenecks.
  
• I want it to run in production mode so the data represents a real usage pattern.
  
• RPM (request/minute) is calculated whenever possible.
  
• Controller actions are measured one-by-one in order to recognize the slowest ones.
  
• Capability to exclude admin controllers (not all administrator pageviews).
  
• Put queries into performance order so the most intensive can be identified, thus simplified.

I thought it would be a good idea not to start until Yii 1.1 gets released, so we can keep maintainability costs down. But we can gather ideas here.

For now, I think it should be able to measure and compare controller actions and sql queries, calculate RPMs and suggest fixes for bottlenecks (through numbers of course).

What do you think? Would you like to join?

[samdark]

Running it in production will slow execution down, right?

What will be the difference between this one and xdebug / xhprof?

[pestaa]

It saves a few profiling measurement to db (preferably innodb), so the overhead should be minimal.

The difference is that this extension would show real usage rates and can take this information into account.

[samdark]

Will it work automatically like xdebug profiler or require some code to mark start and end points for benchmarking?

Will it use Yii hooks (preinit() etc.) and PHP magic __call() to mark measurment points?

[jonah]

Good idea! Maybe add the ability for developers to set marks, where the time between marks are measured. Like here:

http://codeigniter.c.../benchmark.html

[qiang]

@jonah: you can use Yii::beginProfile and endProfile, and then use Yii::getLogger() to retrieve profiling results. See CLogger API for more details.

[qiang]

In case you are not aware, you can set CDbConnection::enableProfiling=true to enable profiling of DB queries. This is very useful in identifying bottlenecks in SQLs.

[pestaa]

samdark,

It will integrate as seamlessly as possible, and will do some basic measurement depending on configuration. (Developer may decide not to profile queries, because he's sure they are optimized enough.)

Attaching to application events and providing extended classes would be a few tools we can use.

jonah,

Marks are relatively difficult to deal with, because the extension would not know which two marks should be compared. Saving all of them is quite easy if we only store milliseconds that passed since onBeginRequest (or something early event).

As of now, I'm thinking about giving each request a unique id, and storing all relevant data in mysql. PK should be [requestID, markID] I suppose.

[pestaa]

Qiang,

Yes, but built-in profiling only shows queries in the current request, and cannot compare slowest queries in application scope for the last 7 days or 1 month or whatever range you prefer.

Edit
The extension should use existing profiling tools if they fit its needs.

This post has been edited by pestaa: 27 August 2009 - 05:13 PM

[qiang]

@pestaa: I don't mean the built-in profiling can achieve your goal. It is mainly used during development phase. IMO, your goal should be achieved using an external tool rather than an internal Yii extension. Of course, Yii may provide some support for such kind of tools. But first, the tool needs to have some clear definition.

[jonah]

pestaa, on 27 August 2009 - 05:06 PM, said:

Marks are relatively difficult to deal with, because the extension would not know which two marks should be compared. Saving all of them is quite easy if we only store milliseconds that passed since onBeginRequest (or something early event).

As of now, I'm thinking about giving each request a unique id, and storing all relevant data in mysql. PK should be [requestID, markID] I suppose.

From the codeigniter page I linked to, you can see how they solved the problem of pairing up marks to be compared. There are other options of course though..

[pestaa]

qiang,

Every good benchmark tool is customized for the application to generate the most useful data. Since all Yii applications follow the same schema (onBeginRequest, CController, onEndRequest), we should create an extension to have this profiling tool better integrated into the system.

By the way, aren't the definitions I mentioned in the first post clear enough for now? I opened this topic to see what we can come up with.

jonah,

The fact is that this tool should not know which marks are going to be compared. I like the idea to freely compare customized points of application flow though.

I thought that each mark should be saved as a row, so the end-user can compare the time spent between them in all requests where both mark appear. This way devs are not forced to write comparisons in the source.

[jonah]

Ah, I see. That would probably be best if you can implant a nice UI for it. Some people will want to benchmark only small pieces of code too, like for instance maybe a foreach loop, just to try to optimize it. In that case you would hardly want to compare the instance of time of the foreach loop to any other instance of time, like for instance the time between the foreach loop and the start of the request is pretty much meaningless.

[pestaa]

Yes, of course, that would be senseless, I was just thinking loudly how the best implementation would look like. Because it is easier to store (and deal with) a few hundred milliseconds, rather than a couple of zillion milliseconds passed since 1st Jan 1970.

Will you be interested to join as soon as 1.1 gets released? As time allows you to do so, of course.

[jonah]

Sure, yeah I'm be interested in helping where I can. I'll be waiting for the 1.1 release