Yii 1.1: directmongosuite

A collection of components for the mongoDB
34 followers

Why another 'suite' for the mongoDB if there exists the extension yiimongodbsuite?

Two reasons:

  • yiimongodbsuite is great and saves a lot of work when you need models and UI for CRUD operations. But the sideeffect is sometimes a poor performance (because all the overhead of AR behind) and you are not really free to make use of the schema-less property of mongoDB. So in my project I always worked with direct mongoDB operations too and implemented some little helpers to speed up working with the mongoDB.

  • Beside the yiimongodbsuite there exists a few extensions (cache, session ...) for the mongoDB. But you have to download and install all these utils separately and - that's the problem - you have to configure the server/database extra for each of these extensions.

So this extension comes with following extensions in one package, modified (and renamed). All components (by default) use the same connection component.

Integrated components

Extensions by aoyagikouhei

My published extensions

and some new helper components

  • EDMSBehavior
  • EDMSConnection
  • EDMSDataprovider
  • EDMSQuery
  • EDMSLogAction
  • EDMSSequence (since v0.2)

Requirements

  • tested with Yii 1.1.8, but should work with 1.1.6+ too
  • PHP mongoDB driver installed

Installation

Download the zip-file and extract it into protected/extensions. So you have all the components in /protected/extensions/directmongosuite/components/... The version is set to the low number 0.1 because maybe in the future there will be integrated more components.

The minimal configuration with the default settings in config/main.php

Import all components by default:

// autoloading model and component classes
    'import'=>array(
        'application.models.*',
        'application.components.*',
        'ext.directmongosuite.components.*',
    ),

Add the application behavior component EDMSBehavior:

'behaviors' => array(
                      'edms' => array(
                        'class'=>'EDMSBehavior',
 
                                                 // 'connectionId' = 'mongodb' //if you work with yiimongodbsuite 
 
                        //see the application component 'EDMSConnection' below
                        // 'connectionId' = 'edms' //default;
                        //'debug'=>true //for extended logging
                      )
            ),

Add the other application components under 'components'

// application components
    'components'=>array(
 
        //configure the mongodb connection
        //set the values for server and options analog to the constructor 
        //Mongo::__construct from the PHP manual
        'edms' => array(
            'class'            => 'EDMSConnection',
            'dbName'           => 'testdb',
                        //'server'           => 'mongodb://localhost:27017' //default
                        //'options'  => array(.....); 
        ),
 
        //manage the httpsession in the collection 'edms_httpsession'
        'session'=>array(
                        'class'=>'EDMSHttpSession',
                        //set this explizit if you want to switch servers/databases
                        //See below: Switching between servers and databases                        
                        //'connectionId'=>'edms',
                        //'dbName'=>'testdb',
                    ),
 
        //manage the cache in the collection 'edms_cache'
        'cache' => array(
            'class'=>'EDMSCache',    
            //set to false after first use of the cache to increase performance
            'ensureIndex' => true,
 
            //Maybe set connectionId and dbName too: see Switching between servers and databases 
        ),
 
        //log into the collection 'edms_log'
        'log'=>array(
            'class'=>'CLogRouter',
            'routes'=>array(
                array(
                    'class'=>'EDMSLogRoute',
                      'levels'=>'trace, info, error, warning, edms', //add the level edms
                      //Maybe set connectionId and dbName too: see Switching between servers and databases 
                    ),
            ),
        ),
 
        //uses the collection 'edms_authmanager' for the authmanager
        'authManager'=>array(
            'class'=>'EDMSAuthManager',
        ),

Comment the components you don't want to use.

I don't want to publish all configurable options here. Please take a look at the comments in the header of the source files or the public properties of the components.

Note for yiimongodbsuite users

If you have the yiimongodbsuite installed and configured, you can set the 'connectionId' => 'mongodb' in the behavior configuration 'EDMSBehavior'. Then you don't have to add the application connection class 'edms' => array('class'=> 'EDMSConnection', ...) In this case the behavior uses the 'EMongoDb' class from the yiimongodbsuite for the connection.

Basic usage

Use the application behaviors 'edms...' to work directly with the PHP mongoDB core classes:

//create the PHP class 'Mongo' with the configured connection (see id 'edms' in config/main.php)
$mongo = Yii::app()->edmsMongo();
 
//create the PHP class 'MongoDB' 
$mongoDb = Yii::app()->edmsMongoDB();
 
//create the PHP class 'MongoCollection' with the collectionName 'members'
$collection = Yii::app()->edmsMongoCollection('members');

You can work with these class instances as you would do with the PHP core classes. The only difference is, that you use a preconfigured connection (in config/main.php)

For example:

Yii::app()->edmsMongoDB()->createCollection('test');
$collections = Yii::app()->edmsMongoDB()->listCollections();
 
foreach ($collections as $mongoCollection)
{
  echo $mongoCollection->getName() . '<br/>';
  var_dump($mongoCollection->getIndexInfo());
}

Switching between servers and databases

The application behavior methods from above can use another 'connectionId' or 'dbName' than the one preconfigured in the 'behavior' section of the config/main.php.

You can add more connection components in main.php like 'edms'. There you can set another server or default database.

'components'=>array(
               //another mongodb connection
        'mongoServer2' => array(
            'class'            => 'EDMSConnection',
            'dbName'           => 'defaultdb',
                        'server'           => 'mongodb://192.168.100.10:27017' 
        ),
//use another connectionId configured in config/main.php
$mongoDb = Yii::app()->edmsMongoDB('mongoServer2');
 
//use the local server (see 'emds'), but not the default configured database 'testdb'
$collection = Yii::app()->edmsMongoCollection('members','memberDb');
 
//use database 'memberDb', but on the server 'mongoServer2'
$collection = Yii::app()->edmsMongoCollection('members','memberDb','mongoServer2');

Used like above you switch between servers/databases for one call. If you want to change the server for all following calls, you can do like below:

Yii::app()->edmsSetConnectionId('mongoServer2');
 
//from here you work on the 'mongoServer2' with the configured db 'defaultdb'
Yii::app()->edmsMongoDB()->createCollection('test');
$collection = Yii::app()->edmsMongoCollection('test');
$collection = Yii::app()->edmsMongoCollection('groups');
....

Important to know:

If you switch the connection with 'edmsSetConnectionId' this will change the settings for the logging, cache and session components too! To ensure that these components always work with the same connectionId/dbName you have to set these properties in the config/main.php too.

Logging

If you log into the mongoDB (installed EDMSLogRoute in config/main.php) you can use the EDMSLogAction in a controller of your choice to list, filter an search within the log entries.

Add the EDMSLogAction to the 'actions' method of the controller.

/**
     * Declares class-based actions.
     */
    public function actions()
    {
        return array(
            'showlog'=>array(
                'class'=>'EDMSLogAction',
            ),
        );
    }

Now you can call the view via the url 'controllerid/showlog'.

Database queries: find ...

When working with PHP and the mongoDB a usual find will look like this:

$criteria = array('age'=>array('$gt'=>30));
$select = array('firstname','lastname','age');
$sort = array('age'=>-1); //age desc
$limit = 100; //the first 100
 
$connection = new Mongo();
$db = $connection->testdb;
$collection = $db->members;
 
$cursor = $collection->find($criteria,$select);
$cursor->sort($sort); 
$cursor->limit($limit); 
 
$result = array();
if (!empty($cursor))
 foreach($cursor as $id=>$value)
   $result[] = $value;
 
$this->render('list',array('data'=>$result));
....

Now you have an array with the results of your query,

With the help of the component EDMSQuery you get the result array with a few lines:

$criteria = array('age'=>array('$gt'=>30));
$select = array('firstname','lastname','age');
$sort = array('age'=>-1); //age desc
$limit = 100; //the first 100
 
$result = EDMSQuery::instance('members')->findArray($criteria,$select,$sort,$limit);
 
$this->render('list',array('data'=>$result));

Note:

EDMSQuery::instance operates on the preconfigured server/database. If you want access another one you have to use the constructor:

$query = new EDMSQuery('members','membersDb');
 
//or the membersDb at the 'mongoServer2'
$query = new EDMSQuery('members','membersDb','mongoServer2');

Please take a look at the code/comments in EDMSQuery.php regarding the usage of the other methods. I don't want to explain it here, because as a directmongodb user you should know howto work with the mongodb.

find operations:

  • findCursor (returns the MongoCursor after the find operation)
  • findOne, findGroupBy, findCountBy, findDistinct

update operations:

  • atomicUpdate (update parts of a record), update, upsert

operations for nested arrays:

  • addToSet, removeFromSet

and more ...

Dataprovider

The directmongosuite comes with the EDMSDataprovider. So you can render the 'find' results of a mongoDB query into standard Yii components: CListview ...

The constructor needs the MongoCursor after a find operation and supports 'sort' and 'pagination'.

$cursor = Yii::app()->edmsMongoCollection('members')->find($criteria,$select);
//or
$cursor = EDMSQuery::instance('members')->findCursor($criteria,$select);
 
$dataProvider = new EDMSDataProvider($cursor,
            array(
                     'sort'=>array('create_time'=>-1,  //desc
                     'pagination'=>array(
                          'pageSize'=>20,
                        ),
                     ));
 
var_dump($dataProvider->getData());

The dataprovider above returns the rows as arrays, like the CArrayDataProvider. But if you need/want to get an array of standardobject or even models as data you can set the third constructor param '$objectClassName' or better use the EDMSQuery:

//the same as above: $config is the configarray for the dataprovider with sort, pagination ...
$dataProvider = EDMSQuery::instance('members')->getArrayDataProvider($criteria,$select,$config);
 
//the data as array of 'stdClass' objects
$dataProvider = EDMSQuery::instance('members')->getObjectDataProvider($criteria,$select,$config);
 
//the data as array of models (instances of the class 'ContactForm')
$dataProvider = EDMSQuery::instance('members')->getModelDataProvider('ContactForm',$criteria,$select,$config);

As 'models' you can use CFormModel or the models from the yiimongodbsuite. The dataprovider uses 'setAttributes' if this method exists, otherwise assigns the values to the public properties of the class.

But I would recomment to use the getArrayDataProvider because of the best performance with no overhead.

So you can do something like this if you want to save the input form the Yii default ContactForm in the mongoDB:

public function actionContact()
{
    $model=new ContactForm; //add the _id property!!
    if(isset($_POST['ContactForm']))
    {
        $model->attributes=$_POST['ContactForm'];
        if($model->validate())
        {
            EDMSQuery::instance('contacts')->insert($model->attributes);
 
            Yii::app()->user->setFlash('contact','Thank you for contacting us. We will respond to you as soon as possible.');
            $this->refresh();
        }
    }
 
    $this->render('contact',array('model'=>$model));
 
}
 
 
public function actionListContacts()
{
    $dataProvider = EDMSQuery::instance('contacts')->getModelDataProvider('ContactForm');
    $this->render('list',array('dataProvider'=>$dataProvider));
}

In you list-view you can now use the CListView component.

Note: If you need to operate (delete, update...) with a single contact item, you have to add the public property '_id' to the ContactForm model.

EDMSSequence

Use the static methods of this component if you want to generate autoincrement integer values. It simulates the sequence/autoincrement features from other databases (oracle, mysql, firebird ...)

//returns the incremented value by 1 of the sequence 'default'
 // 1 on the first usage
 echo EDMSSequence::nextVal();
 echo EDMSSequence::nextVal();
 
  //the next value from the sequence 'mysequence'
  echo EDMSSequence::nextVal('mysequence'); 
 
  //the next value from the sequence 'mysequence' incremented by 100
  echo EDMSSequence::nextVal('mysequence',100);    
 
  //more methods
  //called with no param sequenceName, always the sequence 'default' will be used
  echo EDMSSequence::currentVal(); //return the current value
 
  EDMSSequence::setVal(1000); //set the current value to 1000
 
  EDMSSequence::remove(); //delete the sequence
 
  //Call this once if you have a lot of different sequences to handle
  EDMSSequence::ensureIndex(); //create the 'sequence' index

EDMSQuery::callAfter...

Since v0.2.5 there exists the methods of the EDMSQuery

  • setCallAfterMongoDbCreate
  • setCallAfterCollectionCreate
  • setCallAfterCollectionFind
  • setCallAfterCursorCreate

These functions allow to call methods of the created objects before find, findArray ... is executed.

For example you can 'hook' into the DSMQuery::findArray() method to call object methods after creating Mongodb, collection, cursor.

Usage:

$result = EDSMQuery::instance('mycollection')
 ->setCallAfterCollectionCreate(array('ensureIndex'=>array('x'=>1)))
 ->setCallAfterCursorCreate(array('slaveOkay'=>true,'timeout'=100))
 ->findArray(...)

This is the same a when you do

$collection->ensureIndex(array('x'=>1));
$cursor = $collection->find($criteria, $select); 
$cursor->slaveOk(true);
$cursor->timeout(100);
foreach($cursor...)

But the difference is that you can do this in combination with findArray... So the find operations of EDMSQuery are more flexible now.

For example:

EDMSQuery::instance($collectionName)
->setCallAfterCursorCreate(array('limit'=>1,'sort'=>array('weight'=>1,'title'=>-1)))
->findArray($criteria,$select);

is the same as the short call

EDMSQuery::instance($collectionName)
->findArray($criteria,$select,array('weight'=>1,'title'=>-1),1);

But there are a lot of other functions you maybe want to use before a find operation: see the PHP-MongoDB-Manual

Note: If you use object methods that return results you can call EDMSQuery::getCallResults(type,method) after the operation.

Take a look at the code for details.

Summary

This extension cannot replace the yiimongodbsuite if you want to work with AR/models/dbcriteria ... in yii-style. In my projects I work with both mongodb extensions.

With the directmongosuite you have to build your queries with criteria and options with the syntax direct for the mongoDB. So please look at the manuals there about the howto: select, insert, update, delete ... and other operations.

This extension is only a little bridge between the mongoDB and Yii.

I know there could be a lot of more helper methods in the EDMSQuery. Maybe someone can publish useful methods in the forum.

Please use this topic for comments and disussion:

directmongosuite forum topic

Resources

Changelog

  • v0.2.6: changed file EDMSDataProvider.php Bugfix by ricca509: Sorting in CListview. Many thanks.
  • v0.2.5: changed files EDMSQuery, EDMSBehavior Added/Changed from developers input here or in the forum

    • kamilko: added 'skip' paramter to find methods of EDMSQuery (see comment below)
    • rall0r: changed atomicUpdate to allow multiple modifiers (see this forum topic)
    • Added public property 'setSlaveOkay' to EMDSBehavior to allow configure in config/main.php
    • New methods callAfter ... in EDMSQuery (see above)
  • v0.2: added EDMSSequence, other components unchanged

  • v0.1.3: changed EDMSLogViewer.php, views/view.php, views/_view.php
    • Logviewer didn't show the date. Added configurable property $dateTimeFormat with 'y-m-d H:i:s.' as default.
  • v0.1.2: changed EDMSQuery, EDMSDataProvider
    • bugfix: findArray always returned empty array
    • new: findCountBy
    • changed: better performance for EDMSQuery::getModelDataProvider if it's not a CModel instance
  • v0.1.1 Bugfix: Exception on clear log in EDMSLogViewer
  • v0.1 Initial release

Total 15 comments

#15567 report it
Rajcsányi Zoltán at 2013/11/21 11:57am
re: Problem with UTF-8 and sessions

I tested this problem, but it worked. Could you give me another example to test?

if (isset($_SESSION['Peña Nieto']){
  echo 'Peña Nieto:'.$_SESSION['Peña Nieto']; 
}
$_SESSION['Peña Nieto']='Peña Nieto';
#15566 report it
MetaYii at 2013/11/21 11:35am
Problem with UTF-8 and sessions

Sessions are corrupted when UTF-8 encoded strings are stored with this extension. For instance, if the string "Peña Nieto" is stored in session, the session is lost. This seems weird to me, but I guess the values should be JSON-encoded.

#15210 report it
Joblo at 2013/10/18 02:50am
directmongosuite / mongoyii

For me it's the best combination, because both use the original query syntax/criteria of the mongodb ('$or', '$in' ...). This is the reason I have migrated my projects from mongodbsuite to mongoyii.

  • directmongosuite will use the db-connection configuration from mongoyii in the next release (finished but working a while for testing).

  • I use: EDMSlog with the LogViewer included, EDMSCache, EMDSSequence is important for me.

  • I use mongoyii only for the UI on CRUD-operations, but directmonosuite for reading, displaying lists... EDMSQuery::getArrayDataProvider() - it's small footprint and better performance. In most cases I don't have to create a model for each record (with all overhead methods: behaviors, ...), but it support CListView, CGridView ... like the CArrayDataProvider from Yii. If I need models I use EDMSQuery::getModelDataProvider().

  • I use EDMSQuery::insert for internal journaling, importing/eporting data ... where no userinput is behind and I don't need the overhead of a CModel with the behaviors/validators... The next release will include a EDMSDocument (a PHP stdClass) for best performance without all the CComponent/CModel overhead but with methods like insert(), update(), save(), search(), getAttributes() ...

#15209 report it
Rajcsányi Zoltán at 2013/10/18 01:41am
Using directmongosuit and mongoyii extensions in a webapp

What do you think about using directmongosuite and MongoYii extensions in a same webapp? Each extensions have log handler, but I need directmongsuit's Session and cache mechanisms too.

#15195 report it
Joblo at 2013/10/16 06:58am
safe option

But take aware, the 'safe' options is Deprecated. See the note in the manual: "Please use the WriteConcern w option." If you update mongodb/driver you maybe you will have problems in the future.

This extension will soon be updated to use the mongodb PHP driver 1.3.x (MongoClient instead of Mongo).

#15193 report it
M0ka at 2013/10/16 06:22am
Thanks :)

I have solved it by this

try {
            if (EDMSQuery::instance($this->collectionName())->insert($insert, array('safe' => true))){
                $this->attributes = $insert;
                return true;
            }
        } catch (MongoCursorException $ex) {
            if ($ex->getCode() == 11000 || $ex->getCode() == 11001){
                $this->attributes = $_POST['Users'];
                $this->addError('email', Yii::t('default', 'emailDuplicated'));
                return false;
            }
        }
#15188 report it
Joblo at 2013/10/16 04:26am
insert

Take a look at the manual MongoCollection::insert().

You have to submit the option 'w' (=WriteConcerns) to get more information on insert. If assigned, the return value will be an array(). You have to check for keys 'ok', 'err', 'errmsg'...

$resultArray=EDMSQuery::instance($this->collectionName())->insert($insert,array('w'=>1))
#15181 report it
M0ka at 2013/10/15 01:02pm
Amazing but...

I have a collection which have a unique index at the email attribute, whenever I enter the same email, the method

EDMSQuery::instance($this->collectionName())->insert($insert)

always return true, how to get the duplication error?

#11357 report it
catataw at 2013/01/08 11:57am
gridfs
class EDMSGridFS {

  protected static $_instance;
  private $_gridFs;
  private $_connectionId;
  private $_dbName;

 public function __construct($dbName = null, $connectionId = null) {
    $this->_dbName = $dbName;
    $this->_connectionId = $connectionId;
  }


public function getDb() {
    if (!isset($this->_db))
      $this->_db = Yii::app()->edmsMongoDb($this->_dbName, $this->_connectionId);

       return $this->_db;
  }

  public function storeFile($fileName,$extra = array()){
    $grid = $this->getDb()->getGridFS();
    return $grid->storeFile($fileName,$extra);
  }

  public function storeBytes($bytes,$extra = array()){
    $grid = $this->getDb()->getGridFS();
    return $grid->storeBytes($bytes,$extra);
  }

  public function removeFile($id){
    $grid = $this->getDb()->getGridFS();
    return $grid->delete($id);
  }

  public function getFile($id){
    $grid = $this->getDb()->getGridFS();
    return $grid->get(new MongoId($id));
  }
}
#9012 report it
dreammaker at 2012/07/12 02:28pm
The wrong code

EMDSCache has a typo that casues a logical error.

There is the next method

protected function gc()
    {
        //delete expired entries
        $criteria = array(
            'expired' => array('$gt' => 0),
            'expired' => array('$lt' => time()),
        );
 
        $this->getCollection()->remove($criteria);
    }

but we have not the 'expired' property. In the other places the field is called 'expire'. That mismatch causes not deleting the expired records.

The right code is

protected function gc()
    {
        //delete expired entries
        $criteria = array(
            'expire' => array('$gt' => 0),
            'expire' => array('$lt' => time()),
        );
 
        $this->getCollection()->remove($criteria);
    }
#8658 report it
aoyagikouhei at 2012/06/17 07:33pm
EMongoDbHttpSession modified.

EMongoDbHttpSession ver1.2 had a bug timeout. The timeout property used CHttpSession. But ver1.2 use mongodb timeout. I modified 'timeout' to 'mongoTimeout'.

You might want to fix EDMSHttpSession.

#7076 report it
Pafnucy at 2012/02/22 02:12pm
Skip

I find out that for performance reasons sometimes it is good to use skip() func from MongoDB.
So I did some modification for: EDMSQuery->findCursor() as well as for EDMSQuery->findArray() and EDMSQuery->findObject().

public function findCursor($criteria=array(),$select=array(),$sort=array(),$limit=null,$skip=null)
{
    //always add the '_id'
    if (!empty($select) && !in_array('_id',$select))
        array_push($select,'_id');
 
    $cursor = $this->getCollection()->find($criteria, $select)->skip($skip);
 
    if (!empty($limit) && is_numeric($limit))
        $cursor->limit($limit);
 
    if (!empty($sort) && is_array($sort))
        $cursor->sort($sort);
 
    return $cursor;
}
public function findArray($criteria=array(),$select=array(),$sort=array(),$limit=null,$skip=null)
{
    $result = array();
 
    $cursor = $this->findCursor($criteria,$select,$sort,$limit,$skip);
 
    if (!empty($cursor) && $cursor->count())
        foreach ($cursor as $id => $value)
           $result[] = $value;
 
    return $result;
}
public function findObjects($criteria=array(),$select=array(),$sort=array(),$limit=null,$skip=null)
{
    $result = array();
 
    $cursor = $this->findCursor($criteria,$select,$sort,$limit,$skip);
 
    if (!empty($cursor) && $cursor->count())
        foreach ($cursor as $id => $value)
            $result[] = (object)$value;
 
    return $result;
}
#6479 report it
thaddeusmt at 2012/01/12 05:21pm
Error using the yiimongodbsuite connectionId config option

I fixed it by adding this property to the EDMSConnection class:

public $connectionId;
#5694 report it
yiqing95 at 2011/11/02 09:19am
yet another amazing mogodb suite ! thanks for sharing this

joblo : good job! i download it and read the source code , i must to say it 's great . one can use both mongosuite together , this for querying and the old one for Create/Update/Delete operations (the create and update actions may use the generated form by mogodbsuite and you don't have to deal with the annoying validation manually ,the form logic will be same as yii's CActiveForm which will be generated by the yiimogodbsuite's gii tools, when using the "model" to operate the mongo you get the benefits from yii's CModel class , there are some hooks you can use : like before/after XXX )

#5689 report it
aoyagikouhei at 2011/11/01 09:55pm
OK!

I will support directmongosuite with mongodblogroute and mongodbhttpsession.

Leave a comment

Please to leave your comment.

Create extension
  • Yii Version: 1.1
  • License: New BSD License
  • Developed by: Joblo
  • Category: Database
  • Votes: +16
  • Downloaded: 1,670 times
  • Created on: Nov 1, 2011
  • Last updated: May 25, 2012
  • Tags: mongodb