Yii Framework Forum: Benchmark Extension proposal - Yii Framework Forum

Jump to content

Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

Benchmark Extension proposal Looking for contributors Rate Topic: ***** 1 Votes

#1 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 27 August 2009 - 05:00 AM

Being very careful about performance and speed in general, I'm planning to create a seamlessly integrated benchmarking tool in Yii. I'm looking for experienced contributors, as this project might require significant amount of precise work. With this, I want to reach the following points:

  • If you are familiar with New Relic (performance monitor for RoR), you have already seen a very convenient user interface.
  • I want to show as useful information to administrators as possible to help them identify and remove bottlenecks.
  • I want it to run in production mode so the data represents a real usage pattern.
  • RPM (request/minute) is calculated whenever possible.
  • Controller actions are measured one-by-one in order to recognize the slowest ones.
  • Capability to exclude admin controllers (not all administrator pageviews).
  • Put queries into performance order so the most intensive can be identified, thus simplified.


I thought it would be a good idea not to start until Yii 1.1 gets released, so we can keep maintainability costs down. But we can gather ideas here.

For now, I think it should be able to measure and compare controller actions and sql queries, calculate RPMs and suggest fixes for bottlenecks (through numbers of course).

What do you think? Would you like to join?
0

#2 User is offline   samdark 

  • Having fun
  • Yii
  • Group: Yii Dev Team
  • Posts: 3,346
  • Joined: 17-January 09
  • Location:Russia

Posted 27 August 2009 - 05:22 AM

Running it in production will slow execution down, right?

What will be the difference between this one and xdebug / xhprof?
Yii 1.1 Application Development Cookbook

Enjoying Yii? Star us at github: 1.1 and 2.0.
0

#3 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 27 August 2009 - 06:24 AM

It saves a few profiling measurement to db (preferably innodb), so the overhead should be minimal.

The difference is that this extension would show real usage rates and can take this information into account.
0

#4 User is offline   samdark 

  • Having fun
  • Yii
  • Group: Yii Dev Team
  • Posts: 3,346
  • Joined: 17-January 09
  • Location:Russia

Posted 27 August 2009 - 04:12 PM

Will it work automatically like xdebug profiler or require some code to mark start and end points for benchmarking?

Will it use Yii hooks (preinit() etc.) and PHP magic __call() to mark measurment points?
Yii 1.1 Application Development Cookbook

Enjoying Yii? Star us at github: 1.1 and 2.0.
0

#5 User is offline   jonah 

  • Advanced Member
  • Yii
  • Group: Yii Dev Team
  • Posts: 733
  • Joined: 27-November 08
  • Location:California (US)

Posted 27 August 2009 - 04:21 PM

Good idea! Maybe add the ability for developers to set marks, where the time between marks are measured. Like here:

http://codeigniter.c.../benchmark.html
http://php-thoughts.cubedwater.com - my bloggings about Yii
0

#6 User is offline   qiang 

  • Yii Project Lead
  • Yii
  • Group: Yii Dev Team
  • Posts: 5,875
  • Joined: 04-October 08
  • Location:DC, USA

Posted 27 August 2009 - 04:39 PM

@jonah: you can use Yii::beginProfile and endProfile, and then use Yii::getLogger() to retrieve profiling results. See CLogger API for more details.
0

#7 User is offline   qiang 

  • Yii Project Lead
  • Yii
  • Group: Yii Dev Team
  • Posts: 5,875
  • Joined: 04-October 08
  • Location:DC, USA

Posted 27 August 2009 - 04:41 PM

In case you are not aware, you can set CDbConnection::enableProfiling=true to enable profiling of DB queries. This is very useful in identifying bottlenecks in SQLs.
0

#8 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 27 August 2009 - 05:06 PM

samdark,

It will integrate as seamlessly as possible, and will do some basic measurement depending on configuration. (Developer may decide not to profile queries, because he's sure they are optimized enough.)

Attaching to application events and providing extended classes would be a few tools we can use.

jonah,

Marks are relatively difficult to deal with, because the extension would not know which two marks should be compared. Saving all of them is quite easy if we only store milliseconds that passed since onBeginRequest (or something early event).

As of now, I'm thinking about giving each request a unique id, and storing all relevant data in mysql. PK should be [requestID, markID] I suppose.
0

#9 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 27 August 2009 - 05:09 PM

Qiang,

Yes, but built-in profiling only shows queries in the current request, and cannot compare slowest queries in application scope for the last 7 days or 1 month or whatever range you prefer.

Edit
The extension should use existing profiling tools if they fit its needs.

This post has been edited by pestaa: 27 August 2009 - 05:13 PM

0

#10 User is offline   qiang 

  • Yii Project Lead
  • Yii
  • Group: Yii Dev Team
  • Posts: 5,875
  • Joined: 04-October 08
  • Location:DC, USA

Posted 27 August 2009 - 06:40 PM

@pestaa: I don't mean the built-in profiling can achieve your goal. It is mainly used during development phase. IMO, your goal should be achieved using an external tool rather than an internal Yii extension. Of course, Yii may provide some support for such kind of tools. But first, the tool needs to have some clear definition.
0

#11 User is offline   jonah 

  • Advanced Member
  • Yii
  • Group: Yii Dev Team
  • Posts: 733
  • Joined: 27-November 08
  • Location:California (US)

Posted 27 August 2009 - 07:22 PM

View Postpestaa, on 27 August 2009 - 05:06 PM, said:

Marks are relatively difficult to deal with, because the extension would not know which two marks should be compared. Saving all of them is quite easy if we only store milliseconds that passed since onBeginRequest (or something early event).

As of now, I'm thinking about giving each request a unique id, and storing all relevant data in mysql. PK should be [requestID, markID] I suppose.


From the codeigniter page I linked to, you can see how they solved the problem of pairing up marks to be compared. There are other options of course though..
http://php-thoughts.cubedwater.com - my bloggings about Yii
0

#12 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 28 August 2009 - 01:38 AM

qiang,

Every good benchmark tool is customized for the application to generate the most useful data. Since all Yii applications follow the same schema (onBeginRequest, CController, onEndRequest), we should create an extension to have this profiling tool better integrated into the system.

By the way, aren't the definitions I mentioned in the first post clear enough for now? I opened this topic to see what we can come up with.

jonah,

The fact is that this tool should not know which marks are going to be compared. I like the idea to freely compare customized points of application flow though.

I thought that each mark should be saved as a row, so the end-user can compare the time spent between them in all requests where both mark appear. This way devs are not forced to write comparisons in the source.
0

#13 User is offline   jonah 

  • Advanced Member
  • Yii
  • Group: Yii Dev Team
  • Posts: 733
  • Joined: 27-November 08
  • Location:California (US)

Posted 28 August 2009 - 12:32 PM

Ah, I see. That would probably be best if you can implant a nice UI for it. Some people will want to benchmark only small pieces of code too, like for instance maybe a foreach loop, just to try to optimize it. In that case you would hardly want to compare the instance of time of the foreach loop to any other instance of time, like for instance the time between the foreach loop and the start of the request is pretty much meaningless.
http://php-thoughts.cubedwater.com - my bloggings about Yii
0

#14 User is offline   pestaa 

  • past Yii dev member
  • PipPipPipPip
  • Yii
  • Group: Members
  • Posts: 705
  • Joined: 07-May 09
  • Location:Hungary

Posted 28 August 2009 - 01:13 PM

Yes, of course, that would be senseless, I was just thinking loudly how the best implementation would look like. Because it is easier to store (and deal with) a few hundred milliseconds, rather than a couple of zillion milliseconds passed since 1st Jan 1970. :)

Will you be interested to join as soon as 1.1 gets released? As time allows you to do so, of course.
0

#15 User is offline   jonah 

  • Advanced Member
  • Yii
  • Group: Yii Dev Team
  • Posts: 733
  • Joined: 27-November 08
  • Location:California (US)

Posted 28 August 2009 - 02:29 PM

Sure, yeah I'm be interested in helping where I can. I'll be waiting for the 1.1 release
http://php-thoughts.cubedwater.com - my bloggings about Yii
0

Share this topic:


Page 1 of 1
  • You cannot start a new topic
  • You cannot reply to this topic

1 User(s) are reading this topic
0 members, 1 guests, 0 anonymous users