Definitive Upload Extension

Ok, this is a “what do you think” topic of an idea I’d like to see in yii

My problem, as the rest of you is I need a good upload module (integration with any project to have multiupload features would be great), that can be plugged to any other entity I have in my model with as little effort as it can be done…

My current idea is:

  • write a in-widget File model to map uploads in the database

  • mount an upload as a Widget with actions associated (that will be plugged in the controller): upload/delete/etc

  • write a behaviour that would attach a preValidate and postSave listeners to save the uploaded files

  • views to support custom theming (fileList, etc)

The aim of the thing is removing cohesion of the widget with the main application, to use you’d only need: adding widget actions to the controller, attaching the behaviour to the model of the entity, and using the widget in the view

So… how do you see?

Have you checked the extensions repository ?

There is already at least a couple of widgets there like FancyUpload and plUpload

nz

yes, that’s true, but what I’m talking about is a more “model centric” approach, all those components base their configuration in the user… they are just interfaces to the javascript component itself, so in my actions I’ll have to process the file uploads (or have an action specifically for that)

what I’m proposing is that the user have as few config as he can… with my approach it’d be declaring a behaviour in their models, appending the actions the widget offers to the main controller, use the widget and the model will support file uploads

really I’m thinking for this in the drupal way with nodes with enhanced functionality appended with no config in the user side

i’ll try to provide a prototype so you can see what I’m talking about

Greetings!

Hwangar

I have implanted file handling stuff many times so let me show the path I tend to take:




<?php

class modelname extends CActiveRecord

{

	public $file;


	/**

	* Gets the path or url of the folder that the files are stored in.

	* 

	* @param bool whether we want the path or url

	*/

	public function getFolder($internal=false) {

		$a = 'FOLDER_OF_FILES/'; // must be located two levels above the location of this model

		if ($internal===true)

			return dirname(dirname(dirname(__FILE__))).'/'.$a; //see?

		else

			return Yii::app()->baseUrl.'/'.$a;

	}

	

	/**

	 * @return array validation rules for model attributes.

	 */

	public function rules() {

		return array(

			array('file', 'file'),

			array('file', 'required'),

		);

	}

	

	/**

	* Gets the path or url to the file

	* 

	* @param bool whether we want the path or url

	*/

	public function getUrl($internal=false) {

		return $this->getFolder($internal).$this->fileName;

	}

	

	

	/**

	* The file name on the server of the file

	*/

	public function getFileName() {

		return $this->id.'.'.$this->extension;

	}

	

	protected function afterSave() {

		if ($this->isNewRecord) {

			if (!file_exists($this->getFolder(true)))

				mkdir($this->getFolder(true));

			$this->file->saveAs($this->getUrl(true));

		}

		return parent::afterSave();

	}

	

	protected function afterDelete() {

		unlink($this->getUrl(true)); //delete the file


		parent::afterDelete();

	}

	


}

Hey, thanks Jonah!! that’d handle the “1 file” requirement, but what approach do you suggest for the more generic one model with as many files as you want? (user could restrict to a number)

Yesterday I took the challenge of writing a few lines that I’ll try to publish in a near future but here are an advance:

I told yesterday I wanted to take out of the "extension user" as much code as I can, so that all can be more "automatic", I created an extension with:

  • a File model, that autocreates and manages file table

  • a generic abstract UploadWidget following Strategy pattern, to be extended by concrete strategies (I started by SimpleUpload using a simple filefield, and I’ll follow with SwfUpload that I implemented a couple of months ago with very good result)

  • a generic UploadBehaviour, extended by the concrete implementations

With this, in my "user code", providing I created one custom model with yiishell, I had to:

  • configurate (just by comfort) the config file main.php with:



'ext.upload.*',

'ext.upload.simple.*',



  • add to my custom model:



    public function relations() {

        return array(

            'files'=>array(self::HAS_MANY, 'File', 'EXid', 'condition'=>"entity='".get_class($this)."'"),

        );

    }


    public function behaviors() {

        return array(

                'UploadBehaviour' => array('class' => 'application.extensions.upload.simple.SimpleUploadBehaviour',)

        );

    }



  • add to the view form of the model:



	<div class="row">

        <?php $this->widget("SimpleUploadWidget", array(

            'files' => $model->files,

        )); ?>

	</div>



  • I’d have to append to customController the actions provided by the widget (for advanced usage to show filePanel, deletion of files, etc.

… and for the basic example, it’s working, in your code you just have to worry about displaying files in the way you want using $myCustomModel->files (I have to think about but maybe the extension should provide with a few views to make this even easier…)

I’d like to make it work as a proof of concept to write more extensions with this approach, as I told I use to write modules for drupal, and the next I have in mind is a categorization extension

Thank you guys… I’ll keep you informed

Hwangar

Other thing I’m thinking about, maybe Jonah could show an approach…

Dealing with more advanced file uploads (ajax ones), you can see that "file uploading" happens out of the "flow" of the saving of the model, that is, the sequence should be:

  • show new/edit form

  • user uploads a file -> ajax POST

  • user uploads a file -> ajax POST

  • user uploads a file -> ajax POST

  • user saves the form, form is saved and files should be linked to the model

I have implemented this in the past in the following way:

ajax file uploading goes to a temp dir, storing them in the session until the save occurs, where files move to a repository and are saved in database, the drawback of this approach is the temp dir will have orphan files of “evil” users that upload things and later don’t save the things in the end

I’m open to receive ideas about this…

Thank you!

Hwangar

@Hwangar:

Your approach with tmp dir and session sounds good to me, i’d use the same mechanism. To clean up the orphaned files your implementation needs a garbage collection logic (just like PHP’s sessions). You could use a method that checks for files, older than e.g. a day and remove them. In your component you could have a configurable probability for executing this gc method. So like every 100th request will fire it up and clean the orphaned tmp dirs.

…or use a cron job to clean up thee tmp dir

You plan looks pretty good to me. As I read it I am reminded that it would be nice if Yii had a way to define relations via a behavior. I’m going to think of possible solutions for that.

I’ve tried implanting cross-browser multi-file uploads in that past and I got pretty fed up with it. I ended up just using the HTML5 multi-upload feature which is supported in both the latest version of firefox and chrome (and told the other users to upgrade). It doesn’t have a nice progress bar or anything though. If you can get this extension working I think it will be very useful

If I was to make another attempt at multi-file uploads for all browsers I would use this personally:

http://www.plupload.com

I just spent a day with plupload and could never get it to post the files as a multipart form. Which was rather essential for me since I needed that information on the server side.

Right now I am working with http://www.uploadify.com/. There are 2 extensions written for that currently, I am using the euploadify version and modding it as I go. Some key features I am adding is automatic session and form state submission on uploads and also wrapping the file upload response as a CAction containing the object CUploadedFile. So this widget will be embeddable in other widgets.

nz

@jonah:

Also first thought about a cron job. But that creates another dependency for that component that’s not really necessary. Components should always be as self contained as possible.

I’ve never needed multiple file uploads via AJAX. But if i did, i’d probably have used YUI uploader. YUI js is known for being very robust. So i expect a rock solid implementation.

Another thing i’d like to see in such a component is (optinal) utilization of APC’s RFC1867 feature for an upload progress bar. Already used that for a non-ajax upload form. Pretty easy to implement but nevertheless impressive ;).

Ok, it’s too early to publish an extension but as a proof of concept I uploaded a project at google code in

https://code.google.com/p/yii-upload/

At the wiki I put the integration code to manage nearly all cases you can have dealing with models with uploads associated, I plan to support for each desired Model:

  • As many files as developer wants allowing limiting

  • Easy implementation without LOOKING INTO the widget code, when I get a component I want it to simply work, whatever it does, preferibly I don’t want to know HOW it works, this is for the case… oh, no! it’s not working! or… I want something adapted

  • Simple: just add behaviour to your model, declare relation to File model (provided), add support actions to your controller, and… USE IT, component even will create files table and directories it needs (Configuration by Exception)

  • I plan, for more advanced examples to add custom views for files (that you can theme and custom support actions (ajax upload, ajax deletion, etc) used by that views or by the developer

This is a 2 hours code, that is, I provide only with a normal "SimpleUpload" working, and I plan to try plupload to support ajax multiple files uploading for any Model and adapting my code of SwfUpload in the future

So… have a look and don’t be very bad with me… this is a -0.5 version :P

Greetings

Hwangar

Ok, it’s too early to publish an extension but as a proof of concept I uploaded a project at google code in

https://code.google.com/p/yii-upload/

At the wiki I put the integration code to manage nearly all cases you can have dealing with models with uploads associated, I plan to support for each desired Model:

  • As many files as developer wants allowing limiting

  • Easy implementation without LOOKING INTO the widget code, when I get a component I want it to simply work, whatever it does, preferibly I don’t want to know HOW it works, this is for the case… oh, no! it’s not working! or… I want something adapted

  • Simple: just add behaviour to your model, declare relation to File model (provided), add support actions to your controller, and… USE IT, component even will create files table and directories it needs (Configuration by Exception)

  • I plan, for more advanced examples to add custom views for files (that you can theme and custom support actions (ajax upload, ajax deletion, etc) used by that views or by the developer

This is a 2 hours code, that is, I provide only with a normal "SimpleUpload" working, and I plan to try plupload to support ajax multiple files uploading for any Model and adapting my code of SwfUpload in the future

So… have a look and don’t be very bad with me… this is a -0.5 version :P

Greetings

Hwangar

Haven’t had time to play with it but you can automate the relation addition to the model with this:

http://code.google.com/p/yii/issues/detail?id=943

See the attached file.

Impressive Jonah… great info for this kind of extensions

I’ll try in the “simple” upload right now, let’s see…

Currently I’m watching at plupload to write some lines with it

By the way, the point I see weaker until now in this “databased zero-conf” extensions is the action attaching, I added actions to the Widget and I require the user to append those actions to the controller with a certain prefix (right now a constant in the superclass)… but anyway I need the user provides the url of a certain action to the widget, that is it’s not so automatic I’d wish

Example, in “simple upload” the upload is “traditional”, on form submit, but just for playing I added a ajax link for delete file calling to a DeleteUploadAction provided by the widget… if you want to use the view “fileList” for showing a list of files I need you to provide me with the url of the action because I don’t have means to know WHERE exactly url is mapped

maybe modifying the controllerMap at CWebApplication mapping custom actions? I’m lost at this point

Thx, Hwangar

I looked on code.google.com/p/yii-upload/ but the last committed code is from April: closed prj?

I’m trying to use it but it seems unfinished… well, let’s say I can’t make it work properly… I’m not yet that big experts in yii…


"SimpleUploadWidget"."files" is not defined

bye

I think by now the mainstream (or soon to be) method is to use the built-in browser multi-upload functionality. Chrome and Firefox now have it, and I would think the new, or next version of IE has/will have it. Just tell the other users to upgrade, or let them upload one at a time. No JS needed. Just use the multiple="" attribute in the file input

That wuold be nice if IE6 will disappear and people keep their browser updated… but that’s not true now… so u have either to keep sites compatibles with older browser or to breach in your customers’ brain… don’t u think?

16% seems high to me… I think it is closer to these numbers: http://www.w3schools.com/browsers/browsers_explorer.asp

Also, Google Analytics tells me my websites get less than .1%…