Support memcached objects

[monkeyamore@…]

It would be nice to see support for memcached objects ( http://www.danga.com/memcached/) and seems like a great fit. On .save() of an object, it updates the in-memory cache and the database, when a request for an object comes in, it first checks the memcache. The cache can be located on one machine or have different object types spread across multiple servers (but the same object not cached multiple times), servicing multiple web servers and avoid the need for replication and dramatically reduce the strain on the db server. Query Caches (with mysql) are nice but go invalid frequently and for each replicated server, the query cache is also replicated (poor utilization of server resources).

From the listed site: "Danga Interactive developed memcached to enhance the speed of LiveJournal.com, a site which was already doing 20 million+ dynamic page views per day for 1 million users with a bunch of webservers and a bunch of database servers. memcached dropped the database load to almost nothing, yielding faster page load times for users, better resource utilization, and faster access to the databases on a memcache miss."

[david]

I really think we should have caching support in 2.0. I could imagine interchangeable caching adapters one could write to interface with any caching system desired. So this shouldn't be memcached-only, IMO. We could have a basic "PropelCacheAdapter" you can extend, the class name used in the factory would be either set prior to initialization, or stored in the runtime conf. Then you could write a memcached driver, or a good old, vanilla filesystem cache.

[monkeyamore@…]

I agree with that. Making a generic interface would be better with the ability to have different drivers plugged in, potentially different classes could use different caching methods, I'd imagine. The interface to memcached is quite simple and includes 4 basic functions: get, put, replace, delete. The functions also support options for automatic expiration of the cache - which some drivers would not support.

[francois]

I also think the "PropelCacheAdapter" is a great idea !

I would add optional business rules for each class of object :

• min & max size
• expiration (already mentionned)
• modular purge algorithm with statistics (number of hits, time spent in cache), easy to do with a strategy pattern so we can have a "PropelCachePurgeStrategy". The simplest is to delete the older, but that's not always what you want. The number of hits can be an important parameter...
• statistics on cache usage (% of successful hits), with details per class.

May be web front-end to access those parameters, and display graphics, etc ...

It may seems overkill as feature, but tuning business applications regarding objects lifecycle can become very important on large scale business projects.

[teo]

Replying to david:

I really think we should have caching support in 2.0. I could imagine interchangeable caching adapters one could write to interface with any caching system desired. So this shouldn't be memcached-only, IMO. We could have a basic "PropelCacheAdapter" you can extend, the class name used in the factory would be either set prior to initialization, or stored in the runtime conf. Then you could write a memcached driver, or a good old, vanilla filesystem cache.

Hi. Is there any memcached support planned for propel in any version? I'm just about to make a big deployment and I'm interested on that.

Thanks, teo@…

[Oscar]

I also would badly like a memcache support in Propel ...

[Emil Ivanov]

Supporting many caching backends via adapters will be very useful. Zend_Cache can be used (for example) or other, already developed solution with compatible licence. Reinventing the wheel usually isn't a good idea.

Regards

[derek@…]

I have just implemented caching by creating a custom PeerBuilder object. It was pretty simple, and works great. A simple tutorial on how to do this might relieve some of the pressure to deliver built-in caching.

The only problem is, I can't figure out how to invalidate a cached object on save :-|

[jarno]

Replying to Emil Ivanov:

Supporting many caching backends via adapters will be very useful. Zend_Cache can be used (for example) or other, already developed solution with compatible licence. Reinventing the wheel usually isn't a good idea.

Regards

I too think it'd be a good idea to just create a generic interface for this, and then ship with adapters to some good existing implementations (Zend_Cache, ezcCache, etc). Adding a direct dependency to any external library for this would probably not be such a good idea, though.

+1 for this.

[jarno]

How would this work outside the context of just storing individual objects by their primary key? Since unless you only display individual rows for which you know the primary key beforehand, you'll have to go to the DB first anyway.

How badly out of scope would storing complete result sets of already-hydrated objects go? This would obviously add a lot of issues, and being able to reflect updates on the cached objects would probably be downright infeasible.