Data Caching Advice Needed

Hi

I’m working on my first yii application which will be an exam test type site so users will be presented with a number of questions pulled from the database. The questions will be random but once they’re pulled there’s no need to hit the db again.

I’m just thinking of the best way to handle this so I’ve been reading the wiki and documentation about caching in yii. I want to be able to use the same data set from page to page. I’m pretty sure data caching is the way to go but as I’ve never done it before I’m just polling the community for ideas and suggestions. What would be the best way to approach this type of thing and are there other options I haven’t considered?

I was also thinking about the idea of caching on amazon cloudfront for extra speed. Any thoughts on that?

Looking forward to your replies.

Thanks

Steve

Wow amazon! You expecting some really big data?

I’m new to yii but essentially to cache out to a file and the do a read of that file.

Have your model/controller pull the query based on life of the cache file then load the questions into an array. Then keep track of which questions have been used from the array when you loop through or pick up from the last key/value set with an array counter.

How big is your data set? 1000’s of questions if sonyake thenrisk of having a solid connection to amazon. If not, even a simple php cavhe setup should do you good.

I see the biggest issue in the looping of the array compared to accessing your cache of the db query.

I would also skip the time out of the cache for a manual update from the admin. Make adjustments to question set and rewrite the cache. This will also speed up the rejuvenation of the cache files but means you have to have an active mechnaism to write and update from aomething like an admin panel.

Sorry for not providing the proper Yii functions for doing this :(

Thanks for the response SapporoGuy.

Yeah amazon might be overkill but I was just thinking through some ideas and trying to think of ways to optimize it.

I think each persons test will only consist of about 50-100 questions with each having 4 options so it’s not too big. Would be good if I could cache the whole thing client side. Maybe ajax or something…just a thought. Haven’t had much experience with ajax though so I’d have to research it.

Thanks again SapporoGuy

Regards

Steve

PS: Just thought of another issue i’ll need to resolve. Storing their answers so I can calculate their score at the end. Maybe the ‘hangman’ demo will have some clues. I’ll check it out…

I think you should go back to what you want to build an what your requirements are.

Is this an exam thy works like a survey – then pull in each question as a group and then save each grouo of answets based in test sections. Which might fit well with a scorm package.

If you’re in my daily enviornment and have compiters and students next to each other and cheating is a definite and clear - Yes. You’ll need to consider randomiquestions and a way to associate everything back together.

Either way, you still have pages / sections so writting the answers on a per page basis is probably not that big of performance hit. By doing it this way yiu’ll atill have the option of having a running tally score or the possibility to go back and redo a section.

I’m with SapporoGuy on this. I once worked on a survey system that had very similar requirements. You might be in for some complex queries, but none of them are terribly expensive if you’re keeping an eye on your indices. The only technique that might be helfpul to you is query caching. But be careful with what you are caching ;)

Oh, and for the pulling of random questions: Don’t forget you can ORDER BY RAND() LIMIT n in MySQL. Together with INSERT INTO … SELECT FROM … you’ll be ready to go in an instant :)

Hi Da:Sourcerer

Thanks for your input.

I checked out your mysql link but I’m not sure that would be appropriate. It says specifically in the documentation that they can’t be used for temporary data which is what I’d want.

I’m just looking for the most efficient way to store the data from a query and then store the users responses so that at the end I can produce a report for them with questions, answers and score.

I’ll keep thinking about it and see what I can come up with.

Thanks again!

Steve

PS: Getting the questions by RAND is no problem. I was planning on doing that. It’s more the most efficient way to page through the questions and temporarily store the responses.

Sometimes the better mouse trap is simplicity.

If your not soing automatic grading for multiple choice and are doing a report type system I’d agreebwith rand

For pulling questions or even doing it on the array level which might be less intensive since you could pull from cache rather than the db.

Here’s an interesting thought.

Use the jquery auto-save function that I’ve seen vanilla forums use.

Another crazy thought is slapping everything into the user’s session variable … Have no clue how "expensive that would be though … :(

I thought about putting the answers in the user session but like you i’m not sure how expensive that would be. I figure it would just be something like a series of id’s for the selected answers.

I’ll look at the jquery auto-save function too. Never heard of it before :D

Steve

Hmm, another idea.

If uour questions are numbered then RND’d you could sort the array before you do the insert! Finally got that out my system after all these years! Sweet :)

:) I’ve been wanting to work on a LMS if i ever get the freetime!

I’m not sure but i can see the expense as being a Ram memory hog if you got oodles of people take the test at the same time. Speed will be sweet though!

I won’t be inserting anything in the database. I won’t be keeping a record of questions asked and answered but only the final score. I want to display a report at the end of the test and they can save it as a pdf or something but I don’t see any value in saving the actual questions and answers.

Thanks

Steve

:mellow:

What will you do if someone objects the final score? At least storing the q/a-pairing in immutable form might be valuable.

Hi - These will just be practice/test exams so there’s nothing for them to object or disagree about.

Steve

ok, here’s one thing that you could actually do to improve system performance: When I built that survey system, we had grouped questions, too. I just loaded them all once an used jQuery to display one group at a time. I think you could do the same. However, your system might not be safe from manipulations.

I’m thinking that just doing data caching will be better. I wouldn’t really have much of a clue about using jquery to be honest. Tried looking for examples but couldn’t find much.

Data caching and storing answer ids in a session variable seems to be the best way to go at the moment.

Thanks for all the helpful suggestions guys.

Steve