yiimongodbsuite

Full-featured ActiveRecord like support for MongoDB in Yii
85 followers

This extension is an almost complete, ActiveRecord like support for MongoDB in Yii It originally started as a fork of MongoRecord extension written by tyohan, to fix some major bugs, and add full featured suite for MongoDB developers.

IMPORTANT Info: I've developed this extension as a hobby project. Now I had stopped using Yii in my hobby and work projects. I'm no longer maintaining this code. My codebase is available for easy fork/download on a github. Feel free to continue my work, YMDS already has active community, you can find support mostly on google groups (described bellow).

Flattr this

PLEASE refer to the new FULL-Documentation page

There is also an google groups for topics related to YMDS, everyone are welcome, to post threads, questions, and support requests there.

Work-around for using the OR operator with this extension provided in comments

This is the 1.4 preview release 1

New in 1.4 PR1

The YMDS feature list has grown up recently, some of new ones are:

  • MongoDB session handler, for in db session storage
  • Ability to set use cursors flag with criteria object
  • Support for using OR operator with criteria object
  • Many bugfixes

This is a Preview Release of YMDS 1.4 it is not ready for stable usage I publish it because I need help with testing and code stabilization, anyone are welcome to download, test and submit bug fixes.

Added in this release:

  • Few annoying bugs fixed
  • Documentation corrects
  • Added EMongoPartialDocument class, that supports full-featured partial loading of documents from DB
  • Added fixtures manager, that can replace the Yii default one, and work with Mongo model
  • Ability to do massive partial updates of multiple document

The Key Feature List:

Features covered from standard Yii ActiveRecord implementation

  • Support of using Class::model()->find / findAll / count / countByAttributes and other Yii ActiveRecord syntax
  • Named scopes, along with default scope and parameterized scopes, just like in AR
  • Ready to go out-of-box EFFICIENT DataProvider, witch use native php db driver sort, limit and offset features for returning results!
  • Model classes and embedded documents inherit from CModel, so you can use every class witch can handle of CModel (ie: Gii form generator)
  • Relations support idea/concept/example
  • Support for generating CRUD for EMongoDocument models, with Gii!
  • Support for generating mongo document models from existing SQL tables!
  • Very easy to use criteria object, you don't have to create complex MongoDB query arrays!

MongoDB related feature list

  • Support of schema-less documents with Yii standard rules and validation features
  • Embedded/Nested document support (Embedded documents have their own Model classes with their own rules and other methods)
  • (almost, limited only by MongoDB 4MB limit of single document) endless document embedding/nesting
  • Lazy loading/creating of embedded documents
  • Ability to set FSync and/or Safe flag of DB write operations on different scopes, globally, on model level, and on single model object
  • Ability to use efficient MongoDB Cursors instead of raw arrays of results, returned by the findAll* methods
  • MongoDB GridFS feature support, thanks to work of: Jose Martinez and Philippe Gaultier
  • Support for using any other than _id field as a Primary Key, for a collection
  • Support to have different models in single collection!
  • automated efficient index definition for collections, per model
  • Support "Soft" documents, documents that do not have fixed list of attributes
  • Ability to do Extreme Efficent document partial updates, that make use of MongoDB $set operator/feature
  • Support for PHP Mongo driver versions below 1.0.5
  • Support for partial loading of documents from DB

Limitations:

  • The main limitations are only those present in MongoDB itself, like the 4mb data transfer limit. But That's not a big deal either.
  • In it's current incarnation, This extension does NOT work with the "$or" criteria operator. When we get it working we will remove this line and add an example.

IMPORTANT: The version on GitHub is more up to date as fixes are pushed to the project. This may or may not get updated on a regular basis

Requirements

  • Yii 1.1.5 is required
  • MongoDB latest stable is recommended. Untested with older versions.

Setup

In your protected/config/main.php config file. Comment out (or delete) the current 'db' array for your database in the components section, and add the following to the file:

'import' => array(
      ...
      'ext.YiiMongoDbSuite.*',
    ),
 
    'components' => array(
      ...
      'mongodb' => array(
        'class'            => 'EMongoDB',
        'connectionString' => 'mongodb://localhost',
        'dbName'           => 'myDatabaseName',
        'fsyncFlag'        => true,
        'safeFlag'         => true,
        'useCursor'        => false
      ),
    ),
  • ConnectionString: 'localhost' should be changed to the ip or hostname of your host being connected to. For example if connecting to a server it might be 'connectionString' => 'mongodb://username@xxx.xx.xx.xx' where xx.xx.xx.xx is the ip (or hostname) of your webserver or host.
  • dbName: is the name you want the collections to be stored in. The database name.
  • fsyncFlag if is set to true, this makes mongodb make sure all writes to the database are safely stored to disk. (as default, true)
  • safeFlag if is set to true, mongodb will wait to retrieve status of all write operations, and check if everything went OK. (as default, true)
  • useCursors if is set to true, extension will return EMongoCursor instead of raw pre-populated arrays, form findAll* methods, (default to false, for backwards compatibility)

That's all you have to do for setup. You can use it very much like the active record. Example:

$client = new Client();
    $client->first_name='something';
    $client->save();
    $clients = Client::model()->findAll();

Basic usage

Just define following model:

class User extends EMongoDocument
    {
      public $login;
      public $name;
      public $pass;
 
      // This has to be defined in every model, this is same as with standard Yii ActiveRecord
      public static function model($className=__CLASS__)
      {
        return parent::model($className);
      }
 
      // This method is required!
      public function getCollectionName()
      {
        return 'users';
      }
 
      public function rules()
      {
        return array(
          array('login, pass', 'required'),
          array('login, pass', 'length', 'max' => 20),
          array('name', 'length', 'max' => 255),
        );
      }
 
      public function attributeLabels()
      {
        return array(
          'login'  => 'User Login',
          'name'   => 'Full name',
          'pass'   => 'Password',
        );
      }
    }

And thats it! Now start using this User model class like standard Yii AR model

Embedded documents

NOTE: For performance reasons embedded documents should extend from EMongoEmbeddedDocument instead of EMongoDocument

EMongoEmbeddedDocument is almost identical as EMongoDocument, in fact EMongoDocument extends from EMongoEmbeddedDocument and adds to it DB connection related stuff.

NOTE: Embedded documents should not have a static model() method!

So if you have a User.php model, and an UserAddress.php model which is the embedded document. Lest assume we have following embedded document:

class UserAddress extends EMongoEmbeddedDocument
    {
      public $city;
      public $street;
      public $house;
      public $apartment;
      public $zip;
 
      public function rules()
      {
        return array(
          array('city, street, house', 'length', 'max'=>255),
          array('house, apartment, zip', 'length', 'max'=>10),
        );
      }
 
      public function attributeLabels()
      {
        return array(
          'zip'=>'Postal Code',
        );
      }
    }

Now we can add this method to our User model from previous section:

class User extends EMongoDocument {
      ...
 
      public function embeddedDocuments()
      {
        return array(
          // property name => embedded document class name
          'address'=>'UserAddress'
        );
      }
 
      ...
    }

And using it is as easy as Pie!

$client = new Client;
    $client->address->city='New York';
    $client->save();

it will automatically call validation for model and all embedded documents! You even can nest embedded documents in embedded documents, just define embeddedDocuments() method with array of another embedded documents IMPORTANT: This mechanism uses recurrency, and will not handle with circular nesting, you have to use this feature with care :P

Arrays

You easily can store arrays in DB!

Simple arrays

  • just define a property for an array, and store an array in it.

Arrays of embedded documents

  • there is no way (that i know) where i can easily provide mechanism for this, you have to write Your own
  • This is how I accomplish it for now:
// add a property for your array of embedded documents
    public $addresses;
 
      // add EmbeddedArraysBehavior
    public function behaviors()
    {
      return array(
        array(
          'class'=>'ext.YiiMongoDbSuite.extra.EEmbeddedArraysBehavior',
          'arrayPropertyName'=>'addresses', // name of property
          'arrayDocClassName'=>'ClientAddress' // class name of documents in array
        ),
      );
    }

So for the user, if you want them to be able to save multiple addresses, you can do this:

$c = new Client;
    $c->addresses[0] = new ClientAddress;
    $c->addresses[0]->city='NY';
    $c->save(); // behavior will handle validation of array too

or

$c = Client::model()->find();
    foreach($c->addresses as $addr)
    {
        echo $addr->city;
    }

Querying

This is one of the things that makes this extension great. It's very easy to query for the objects you want.

// simple find first. just like normal AR.
    $object = ModelClass::model()->find()

Now suppose you want to only retrieve users, that have a status of 1 (active). There is an object just for that, making queries easy.

$c = new EMongoCriteria;
    $c->status('==', 1);
    $users = ModelClass::model->findAll($c);

and now $users will be an array of all users with the status key in their document set to 1. This is a good way to list only active users. What's that? You only want to show the 10 most recent activated users? Thats easy too.

$c = new EMongoCriteria;
    $c->active('==', 1)->limit(10);
 
    $users = ModelClass::model->findAll($c);

It's that easy. In place of the 'equals' key, you can use any of the following operators


    - 'greater'   | >
    - 'greaterEq' | >=
    - 'less'      | <
    - 'lessEq'    | <=
    - 'notEq'     | !=, <>
    - 'in'        | 
    - 'notIn'     | 
    - 'all'       | 
    - 'size'      | 
    - 'exists'    | 
    - 'type'      | // BSON type see mongodb docs for this
    - 'notExists' | 
    - 'mod'       | %
    - 'equals'    | ==
    - 'where'     | // JavaScript operator


*NOTICE: the $or operator in newer versions of mongodb does NOT work with this extension yet. We will add it to the list above when it is fixed.Newer versions of mongo db will work, just not the $or operator. For examples and use for how to use these operators effectively, use the MongoDB Operators Documentation here.

Here are a few more examples for using criteria:

// first you must create a new criteria object
    $criteria = new EMongoCriteria;
 
    // find the single user with the personal_number == 12345
    $criteria->personal_number('==', 12345);
    // OR like this:
    $criteria->personal_number = 12345; 
 
    $user = User::model->find($criteria);
 
    // find all users in New York. This will search in the embedded document of UserAddress
    $criteria->address->city('==', 'New York');
    // Or
    $criteria->address->city = 'New York';
    $users = User::model()->findAll($criteria);
 
    // Ok now try this. Only active users, only show at most 10 users, and sort by first name, descending, and offset by 20 (pagination):
    // note the sort syntax. it must have an array value and use the => syntax.
    $criteria->status('==', 1)->limit(10)->sort(array('firstName' => EMongoCriteria::SORT_DESC))->offset(20);
    $users = User::model()->findAll($criteria);
 
    // A more advanced case. All users with a personal_number evenly divisible by 10, sorted by first name ascending, limit 10 users, offset by 25 users (pagination), and remove any address fields from the returned result.
    $criteria->personal_number('%', array(10, 0)) // modulo => personal_number % 10 == 0
             ->sort(array('firstName' => EMongoCriteria::SORT_ASC))
             ->limit(10)
             ->offset(25);
    $users = User::model()->findAll($criteria);
 
    // You can even use the where operator with javascript like so:
    $criteria->fieldName('where', ' expression in javascript ie: this.field > this.field2');
    // but remember that this kind of query is a bit slower than normal finds.

Regexp / SQL LIKE replacemt

You can use native PHP Mongo driver class MongoRegex, to query:

// Create criteria
    $criteria = new EMongoCriteria;
    // Find all records witch have first name starring on a, b and c, case insensitive search
    $criteria->first_name = new MongoRegex('/[abc].*/i');
    $clients = Client::model()->findAll($criteria);
    // see phpdoc for MongoRegex class for more examples

for reference on how to use query array see: http://www.php.net/manual/en/mongocollection.find.php

Creating criteria object from an array:

// Example criteria
    $array = array(
        'conditions'=>array(
            // field name => operator definition
            'FieldName1'=>array('greaterEq' => 10), // Or 'FieldName1'=>array('>=', 10)
            'FieldName2'=>array('in' => array(1, 2, 3)),
            'FieldName3'=>array('exists'),
        ),
        'limit'=>10,
        'offset'=>25,
        'sort'=>array('fieldName1'=>EMongoCriteria::SORT_ASC, 'fieldName4'=>EMongoCriteria::SORT_DESC),
    );
    $criteria = new EMongoCriteria($array);
    // or
    $clients = ClientModel::model()->findAll($array);

Known bugs

  • Remember, this is not complete yet. So at this stage, it can have some ;]
  • If you find any please let me know
  • As said before, it does not work with the OR operators

Resources

Contribution needed!

  • I'm not English native speaker, need someone who can correct/rewrite/write my documentation and/or PHPDoc's in code
  • Any help would be great :)
  • Contact me: darek.krk on a gmail dot com or via PM

Big thanks goes to:

  • tyohan: for first inspirations and idea of extension
  • luckysmack: for big help with testing and documentation
  • Jose Martinez and Philippe Gaultier, for implementing and sharing GridFS support
  • Nagy Attila Gábor, for big help with new functionality and testing

Total 20 comments

#7500 report it
elsonwu at 2012/03/26 12:17pm
#7490: bug in unique validator

@intel352
thank you for your reply, that is right.

#7490 report it
intel352 at 2012/03/26 03:00am
Re: bug in unique validator

Re: #7485 by @elsonwu

The performance that would occur in your fix is not ideal, as your solution would issue a count and a find for every unique validation, and a find request is already less performant than a count request.

I suggest you instead use the patch I provide here: https://github.com/intel352/YiiMongoDbSuite/commit/65f7ae77c9b4364ba22a94c322555b33b6a16222

#7485 report it
elsonwu at 2012/03/26 02:32am
I found a bug when using EMongoUniqueValidator in update.

When I update a record, such as user and I set the field email unique. I update the username and the email not modified, then it will show error. Here is my bugfix for it.

class EMongoUniqueValidator extends CValidator
{
    public $allowEmpty=true;
    public function validateAttribute($object, $attribute)
    {
        $value = $object->{$attribute};
        if($this->allowEmpty && ($value === null || $value === ''))
            return;
 
        $criteria = new EMongoCriteria;
        $criteria->{$attribute} = $value;
        $count = $object->model()->count($criteria);
 
        //if($count !== 0)
        //    $this->addError(
        //    $object,
        //    $attribute,
        //    Yii::t('yii', '{attribute} is not unique in DB.')
        //);
        //$count = $object->model()->count($criteria);
        $obj = $object->model()->find($criteria);
        if( ! empty($obj))
        {
            if($object->getIsNewRecord() || ($obj->id != $object->id))
                 $this->addError($object, $attribute, Yii::t('yii', '{attribute} is not unique in DB.'));
        }
    }
}
#6955 report it
mintao at 2012/02/15 12:35pm
Forked? Yes - at github

https://github.com/mintao/YiiMongoDbSuite

#6949 report it
yiimann at 2012/02/15 12:02pm
fork

anyone forked this extension ?

#5586 report it
ahirmayur at 2011/10/22 04:01am
CActiveDataProvider Compatibality

Can this work together with CActiveDataProvider? -> Yes it does for sure; I am using MongoDB with MySQL.

Is this question make sense? -> Not really since you are using Yii where compatibility and interoperability matters.

#5582 report it
nettrinity at 2011/10/21 03:32pm
CActiveDataProvider Compatibality

Can this work together with CActiveDataProvider?(asked from a beginner, because I just learned how to use data provider today:) ) It seems it's not mentioned in the Manual.

Is this question make sense?

#5556 report it
Blizz at 2011/10/20 06:21am
Definitely production ready :)

It definitely is :) Besides the fact that caching was added here and there (see my previous comment) and a new version of deleteByPk to mimic the default behavior (Yii's CActiveRecord implementation of it does not call before/afterDelete) it is being used as is and working very good.

#5553 report it
joblo at 2011/10/20 04:21am
good enough for production environment
#5550 report it
nettrinity at 2011/10/19 05:42pm
Is it OK to use this extension in production environment?

Please let me if you have found it good enough to use in a production environment.

Thanks!

#5507 report it
Blizz at 2011/10/17 11:17am
Possible performance issue

Great extension!

I hope you allow me to give you a small piece of advice :)

Determining meta data for databases is usually very slow. It's the same for mongo. It's the reason Yii caches it by default.

Depending on your server a call MongoCollection::getIndexInfo() can take 500ms or more to complete. Since you are doing it in your init() function on every page, this will cause serious issues when you are running under higher load (trust me, we experienced it :) ). Best thing is to just use the Yii::app()->cache to temporary cache the actual data so that you can just load it out of the cache for most pageviews.

We just did:

$name = dbName . collectionName;
$indexes = Yii::app()->cache->get($name);
if (!is_array($indexes))
{   
   <determine indexes the regular way>
   Yii::app()->cache->set($name, $indexes, 1800);
}
...

This gives you a huge performance boost for almost no work.

#5392 report it
maris at 2011/10/09 09:04am
Dynamic attribute defination.

By defining attributes by properties, you lost one of the biggest privilegies of MongoDB, that can have non regular structure. What if add new method, that will add custom attributes in protected property $_attributes like in CActiveRecord..

On save code might look like this.

$user = new User;
$user->name = 'John';
$user->surname = 'Backer';
 
$user->addCustomAttribute('age');
$user->age = 15;
 
OR
 
$user->setCustomAttribute('age', 15);
 
$user->save();

And on find:

$user = User::model()->findByPk(....);
echo $user->age;

Hopefully my advice will help you, MG.

#5334 report it
canni at 2011/10/05 01:11pm
Info

Please read the info on top of this page, this project is free for take-over.

#5304 report it
ahirmayur at 2011/10/03 04:00am
YiiMongoDBSuite Nested Embedded documents

How to make Nested Embedded Documents???

http://pastebin.com/raw.php?i=wmBydGSR

#5057 report it
bear at 2011/09/11 12:06am
why lost $criteria parameter in findByAttributes and findAllByAttributes

In EMongoDocument Class , the findByAttributes and findAllByAttributes method has only one parameter, but in the comment, also I think is needed, should add the $criteria parameter. And these two methods also lost trace log code. See the original code:

/**
     * Finds document with the specified attributes.
     *
     * See {@link find()} for detailed explanation about $condition.
     * @param mixed $pk primary key value(s). Use array for multiple primary keys. For composite key, each key value must be an array (column name=>column value).
     * @param array|EMongoCriteria $condition query criteria.
     * @return the document found. An null is returned if none is found.
     * @since v1.0
     */
    public function findByAttributes(array $attributes)
    {
        $criteria = new EMongoCriteria();
        foreach($attributes as $name=>$value)
        {
            $criteria->$name('==', $value);
        }
 
        return $this->find($criteria);
    }

may should be like this:

public function findByAttributes(array $attributes, $criteria=null)
    {
        Yii::trace(get_class($this).'.findByAttributes()','ext.MongoDb.EMongoDocument');
        $criteria = new EMongoCriteria($criteria);
        foreach($attributes as $name=>$value)
        {
            $criteria->$name('==', $value);
        }
 
        return $this->find($criteria);
    }
#5049 report it
yiqing95 at 2011/09/10 12:59am
some questions

hi canni : thanks for your great extension!

want to know if i can use both RDBMS and the Mongodb in one system , the relation i just use method declaration but the yii 's way(which return some arrays ) , ie: in User AR class

public function addresses()
{
    return Address::model()->findAllByAttributes(array('attribute_with_client_id'=>$this->primaryKey()));
}

the Address is MongoDocument . this is an one to many relation example ,other relations implement the same way . i need some advice .

and the softdocument is very useful , little advice: when a function accept an array which all elements is string ,i will use this way :

if (is_string($params) || is_array($params)) {
              //you may be trim every array item  
            $params = is_string($params) ? explode(',', $params) : $params;
        } else{
 
      // throw some  exception ,tell the caller parameter is invalid
   }

now i can use such way to call the function: $model = new MixedModel();

$model->initSoftAttributes(array('field2', 'field3')); ///or $model->initSoftAttributes('field4,field5');

#4579 report it
Sammaye at 2011/07/21 05:30pm
Ok I see

I see now you use reflection of public attributes.

#4577 report it
Sammaye at 2011/07/21 01:56pm
How do you define between soft and hard attributes

Hi,

I have been browsing your documentation but you seem to treat class variables all the same. There is no need to pre-define a schema. This is a problem since I have classes in my Yii project which use non-db variables.

I was wondering how do you fill the metaData array that Yii uses in SQL if mongo has no schema?

#4515 report it
huggie at 2011/07/15 10:23am
Work-around for using the OR operator

Work-around for using the OR operator is not found in the comments anymore (it got deleted?) I have been using this regex solution which works for me (when working with strings):

board:/ai|uai/ avg 0.000650s

board:/^(uai|ai)$/ avg 0.000780s

#4408 report it
bear at 2011/07/06 10:03am
CUniqueValidator

Great! I found it can't use CUniqueValidator rule so far now.

Leave a comment

Please to leave your comment.

Create extension