Symphony.

A while back there was a disussion about the must-revalidate, nocache headers sent by JIT. I can’t find the old topic, so I thought I’d start a new one.

Do you have any idea where the headers in jit are set? I thought they were set at the image.php file, but they aren’t. The constant flickering of the images when clicking though the sites I make is really annoying, and it’s costing alot of bandwidth..

Any help would be appreciated!

I guess in /extensions/jit_image_manipulation/lib/class.image.php:184:

public static function renderOutputHeaders($output, $dest=NULL)

Thanks!

JIT and browser caching

I played around with this the other day but couldn’t get the headers to be sent correctly. It’s on my to-do list to find a fix. In fact I was reading the relevant chapters of High Performance Web Sites only this morning to understand the issue more fully. To get caching working at its most basic level we need to:

• prevent the Cache-Control header being sent
• send a Last-Modified header with the file’s last modified date rather than now()

If this can be achieved then the browser should use its cached version. The browser will still make an HTTP request for the image, but since we return the same Last-Modified date hopefully it’ll be interpreted as a 304 Not Modified (in which case the browser will use its cached version) and the server won’t send the entire image back.

OK, I think I’ve nailed this. Removing the no-cache isn’t enough. Each time the image is requested the browser will first make a conditional GET request and changes its behaviour depending on the response:

If a 200 is returned then it will download the new asset entirely.

If a 304 Not Modified is returned then it stops there, doesn’t download the asset, and instead serves the asset from the cache.

The Last-Modified time of the browser’s cached asset is sent in the request. By comparing this to the last modified date of the original file on the server, we can choose to send back a standard 200 (new file) or 304 (file hasn’t changed).

By modifying the renderOutputHeaders method to also receive the file path (so it can calculate the last modified date) I’ve successfully got images caching:

public static function renderOutputHeaders($output, $dest=NULL, $image_path=NULL){

    // if image_path is passed to this function we can check last modified dates
    if ($image_path) {
        $last_modified_time = filemtime($image_path);
        header('Last-Modified: ' .  gmdate("D, d M Y H:i:s", $last_modified_time) . ' GMT');

        if (@strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) == $last_modified_time) {
            header('HTTP/1.1 304 Not Modified');
            exit();
        }               
    }

    /// return the correct MIME type
    header('Content-Type: ' . image_type_to_mime_type($output));

    // remove no-cache headers
    header('Cache-Control: ');
    header('Pragma: ');

If you replace the file on the server with a new file of the same name, browsers will receive that new asset, since the last modified date will be newer.

The other thread starts discussing changing Expires headers, which is an extra step you can take, but one that I wouldn’t recommend unless you really understand what you’re doing. Sending an Expires header date far in the future prevents the browser making its initial condition GET request to see if the file on the server has changed or not. These requests can all add up, but aren’t major.

It’s important to get the JIT extension to correctly serve 304 Not Modified since this is now Apache will serve “static” images by default. JIT should act exactly the same.

Looks great! But where exactly do you pass the image path to the method?

In image.php around line 119 where $image_path is defined, add this directly underneath:

$original_image_path = $image_path;

And around line 185 change the call to renderOutputHeaders to:

Image::renderOutputHeaders($meta->type, NULL, $original_image_path);

This does not affect loading external images since these are not cached on disk by Symphony at all, but that’s a topic for another day entirely.

Thanks Nick, this is great.

I made about the same changes a few minutes after I posted thanks, but I forgot to upload them, sorry!

Have you changed the display method in the image class aswell?

Questions from the less-techy in the room:

(1) Should we start implementing this in our websites down? By this I mean is this stable enough that using it wouldn’t break anything?

(2) Assuming the first answer is yes, where should we be putting the code? Or, alternatively, is an update to the JIT extension immanent?

Thanks Nick.

I think this could be improved a bit more to return as soon as possible 304.

I am afraid all we can do here is re-implement Apaches header behaviour so I tried to solve it the way I’ve done it a few years back. I’ve forked JIT and made a couple of changes to it:

Cache directory structure is “more verbose”. This means no MD5 hashes but nested directories.

/manifest/cache/2/80/80/5/images/IMG_3005.JPG

instead of

/manifest/a3cca2b2aa1e3b5b3b5aad99a8529074.JPG

This enables us to create rewrite rules that can bypass PHP completely for cached images. So only the first request for a JITted image would go through PHP, all successing requests are handled by Apache (and its caching behaviour).

According to Firebug the results were quite dramatic on my local development machine: 11ms for a cached image in JIT 1.08 vs. 2ms for a cached image in JIT 1.09-phoque.

This is by no means a final product, this is merely some sort of proof of concept and a request for comments. What do you think, could this be a step in the right direction?

If you want to try it for yourself I suggest you take a look at the “Updating” section in the README.markdown file.

A neat solution! But with the problem that if the source image changes (but the file name stays the same) then it will never re-create the thumbnail. An alternative to this method would be leaving JIT as-is but having it send a far-future Expires header. This way, the browser doesn’t even do a conditional request to get the 304, it will serve the image from the local cache regardless. They both have the same problem that if the image changes, the old cached version is still used.

According to Firebug the results were quite dramatic on my local development machine: 11ms for a cached image in JIT 1.08 vs. 2ms for a cached image in JIT 1.09-phoque.

What result do you get for the modified 304 code I’ve been trialling?

I think running the request through PHP is still acceptable if we can move the last-modified check higher up the processing chain to return the 304 header.

A neat solution! But with the problem that if the source image changes (but the file name stays the same) then it will never re-create the thumbnail.

Maybe the EntryPostEdit delegate could be used to purge all files with that filename once an entry has been edited.

Of course I’ve also been thinking about a garbage collector for both old files and empty directories. Probably somewhere in one of the generic admin page delegates (and a way to prevent it from executing every single time the delegate fires).

I’ve reduced the amount of changes to get 304 headers being sent from JIT to just a few lines.

http://github.com/nickdunn/jit_image_manipulation/commit/390c34ce0cee5dd6d52d8bfa9b6e710514d45fa0

If you want to try it on your own sites, this should be enough to enable browser caching.

Has any of this been integrated into the official extension at all? I’m at a development stage where I’m preparing to output the images of a portfolio using JIT into my templates, and I’m always concious of caching and loading times.

I really want to have cached images but don’t have the knowledge to fully understand what is being discussed here and which changes should attempt, I really hope it is officially added for ease of me using this extension how I want…

Yes, the 304 header stuff is in v1.09 which is the latest on Github. I imagine this will be the version bundled with Symphony 2.0.8 when it is released, but you should be able to use this with earlier versions of Symphony.

Yeah, I’m playing with 2.0.8RC1 still. Need to update to RC2 asap though and play with that…

Thanks Nick!

I normally try to optimize my site according to e.g. the Yahoo Performance Rules etc.

I just discovered the fact that JIT-generated images have no Expires Header set (which is discussed before). I also noticed that Etags are set.

I have a ‘far-future’ Expires Header set on all my (standard) images and would like the header set on JIT-generated images also. I understand the issues with caching CMS content but, in my case, I don’t think the images are changed very often.

Also, if I understand the Yahoo Etag Rule correctly, Etags are not always helpful: Yahoo actually advises you to turn them off in some (most?) cases.

Apart from this, I noticed my (cached!) JIT-generated image (with Etag) actually returns a 200 OK response, not a 304 Not Modified which seems to kind of defeat the purpose of the Etag header.

Conclusion: my JIT-generated images seem to cache correctly locally (e.g. ‘working offline’ still shows the image) but do not return a Far-Future Expires header. They do return an Etag header, which does not return a ‘403’ but instead a ‘200’…

Would it be correct to assume the Etag functionality in my JIT does not work? Would it be usefull to be able to add Far-Future Expires headers (even though the images are cached locally)?

Thanks.

If I’m pulling in remote sources in an image-heavy photoblog-style site (e.g. Tumblr/Flickr feeds), would it be considered bad practice to resize all images on the fly using JIT?

It seems like a good simple solution but I’m wondering if performance/bandwidth will be an issue and whether it would be better to somehow resize and save the images locally via CRON, then feed them back into a data source?