Yii 1.1: X-Sendfile - serve large static files efficiently from web applications

21 followers

Basics

Normally when we want users to download a file, that file is put in a folder under the web application root and the web server does the rest.

Most of the time this is not enough, because the user needs to be authenticated or the file name should be searched in the database.

In that case a script like this would be used:

//.. authenticate and authorize, redirect/exit if failed
authenticate(); 
//.. get the file to be downloaded, redirect/exit if it does not exist
$file = determine_file();
//.. get the content of the requested file 
$content=file_get_contents($file);
//.. send appropriate headers
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file) . '"');
header("Content-Length: ". filesize($file));
echo $content;

For this in Yii we have CHttpRequest->sendFile().

Is something wrong with this?

This means that our script has to read the file from the disk, which goes through the output buffer, is flushed to the web server and processed before sent to the client.

In case of big files it will consume a lot of memory, and if the file is bigger than memory_limit, it could break the script or exceed script max execution time.

When loading entire files into memory, you then have to unload them, and your thread is busy for that process.

Caching is a real pain too.

Not to mention implementing download resuming.

Ideal solution

Ideally our script would process user authentication and/or search the database for the file name and then instruct the web server to send the file to the user. This way the script would be more responsive as it could continue processing as soon as it instructs the web server to process the request.

Meet the X-Sendfile

X-Sendfile is a feature that allows a web application to redirect the request for a file to the web server that in turn processes the request, this way eliminating the need to perform tasks like reading the file and sending it to the user.

X-Sendfile can improve your web application performance, especially when working with very big files, as the web server will load the file you specified and send it to the user.

But what is X-Sendfile?

X-Sendfile is a special header option that tells the web server to ignore the content of the response and replace it by the file that is specified in the X-Sendfile header.

When the web server encounters the presence of such header it will discard all output and send the file specified by that header using web server internals including all optimizations like caching-headers and download resuming.

Is there something I should know before using X-Sendfile?

This option is disabled by default as it's still not a standard feature.

If this option is disabled by the web server, when this method is called a download configuration dialog will open but the downloaded file will have 0 bytes.

This option allows to download files that are not under web folders, and even files that are otherwise protected (deny from all) like .htaccess.

Different web servers has implemented this option using different directive:

sendfile directive    - web server application using it
---------------------   -------------------------------
X-Sendfile            - Apache, Lighttpd v1.5, Cherokee
X-LIGHTTPD-send-file  - Lighttpd v1.4
X-Accel-Redirect      - Nginx, Cherokee

The disadvantage to using X-SendFile is that you lose control over the transfer mechanism. What if you want to perform some tasks after the client has received the file? For example: to allow a user to download a file only once. With X-Sendfile this could not be done because the script continues to run as soon as it sends the file to the web server for processing and so the script could not know if the download was successfull.

How to use this from Yii?

From Yii version 1.1.6 there is CHttpRequest->xSendFile() that allows to send files using the X-Sendfile directive.

CHttpRequest->xSendFile($fullName, $options)

Where $fullName is the file name with full path and $options are the additional options like saveName, mimeType, xHeader and terminate.

saveName: file name shown to the user. Useful when creating temporary files with cryptic names, to avoid collisions, but still serving the user a nicely named file.

mimeType: mime type of the file for proper file handling, if not set it will be guessed automatically based on the file name.

xHeader: by default this method uses the directive X-Sendfile, this option can be used to set different directive if needed by the web server.

terminate: by default the script will terminate execution after sending the X-Sendfile header, setting this option to false this can be prevented.

Example

Yii::app()->request->xSendFile('/home/user/Pictures/picture1.jpg',array(
   'saveName'=>'RequestedFile.jpg',
   'mimeType'=>'image/jpeg',
   'terminate'=>false,
));

The image located on the server at '/home/user/Pictures/picture1.jpg' will be downloaded as 'RequestedFile.jpg'. The users downloading the file does not know where the requested file is located on the server.

Additional documentations

Apache
Lighttpd
Nginx
Cherokee

Total 5 comments

#11168 report it
BraindeadBZH at 2012/12/21 08:43am
Re: Bug with Rewrite engine.

@jiaming

I also had some trouble with the rewrite engine which I use to hide index.php. I found the solution by adding a XSendFilePath directive pointing to the root folder where the files served by XSendFile are.

#8180 report it
Maurizio Domba Cerin at 2012/05/15 02:45am
Re: Bug with rewrite engine.

@jzhong5

I just found this article, see if it helps you - http://stackoverflow.com/questions/8866549/confilict-between-rewriteengine-and-xsendfile

If not, it would be better to ask on the forum for a better discussion.

#8174 report it
jiaming at 2012/05/14 07:02pm
Bug with Rewrite engine.

It looks like XsendFile has a HUGE BUG: it is not working with Rewriteengine.

When you rewrite your url and get rid of index.php Xsendfile will keep giving you index.php not FOUND...

#6373 report it
brecht at 2012/01/04 08:51am
Apache XSendFile module has a bug

The current XSendFile module for apache has a bug. A fix is downloadable here:

http://www.calazan.com/apache-2-2-partial-results-are-valid-but-processing-is-incomplete-unable-to-stat-file-x-sendfile/

Perhaps it's not a bug, but fixes the 0 byte filesize on download.

#5054 report it
ManInTheBox at 2011/09/10 11:46am
Enabling XSendFile

If you stuck with 0 bytes from downloaded file, like mentioned in this article, you haven't enabled xsendfile_module. For apache users in your configuration file (eg. httpd.conf) add this two lines and restart web server:

# enable xsendfile
XSendFile On

# enable sending files from parent dirs
XSendFileAllowAbove On

More detailed explanation is here: XSendFile explained

Maybe it would be helpful for someone.

Leave a comment

Please to leave your comment.

Write new article