kateray.net

Scaling 1: Caching with Heroku

Caching is a bitch. I know, there are a million blog posts (and even a railscast) out there that say it’s just a few super-fast-easy steps. But for those of us who’ve learned Rails without having a background in databases, those super easy steps sometimes skip a lot we’re assumed to know.
 
Get Memcached working on Heroku
My story about memcached is a little embarrassing. I’m willing to divulge it here because it’s a good screening process for this blog: If, after reading this, you conclude that I am an idiot (and that you, by comparison, are not), then I’m probably not writing for you anyway.

So memcached is just a gigantic hash where you can store chunks of data associated with unique keys. There aren’t too many commands you need to learn

#Store some data and associate it with a key
$ Rails.cache.write(‘name_of_your_key’, some_data_to_store)
#Retrieve that data by looking up its key
$ Rails.cache.read(‘name_of_your_key’)
#Delete the data
$ Rails.cache.delete(‘name_of_your_key’)
#Check if there’s already anything stored with this key; if so, retrieve it, if not, store it and retrieve it
$ Rails.cache.fetch(‘name_of_your_key’) { some_data_to_store }

Simple enough. And Heroku gives us a memcached add-on with 5 MB of space for free, along with some easy installation instructions that I’ll include here so that you don’t even have to bother clicking that link

1) Add ‘memcached-northscale’ to your gem manifest (.gems) or gemfile
2) In config/environment.rb somewhere in the ‘Rails::Initializer.run do |config|’ block add the line config.gem ‘memcached-northscale’, :lib => ‘memcached
3) Add the line require memcached in the same place, config/environment.rb, or in config/environments/production.rb if you only want to worry about memcached in production mode (you’ll get Seg faults when you try to run on localhost if you don’t have memcached set up locally)
4) Add the line config.cache_store = :mem_cache_store, Memcached::Rails.new in your config/environments/production.rb
5) In config/environments/production.rb, make sure config.action_controller.perform_caching = true and config.action_view.cache_template_loading = true
6) Install the memcached add-on on Heroku, either through the website or by running $ heroku addons:add memcache in your terminal

Okay, so it should be set up to work. Here’s the embarrassing part

7) To check it out, open up your heroku console ($ heroku console) and type mc = Memcached.new. Now, all Heroku will tell you is that you’ll see some ‘…truncated output…’ In fact, what you’ll see is

=> …@servers=[<Memcached::Rails::Server: mc3.ec2.northscale.net:11211 [8] (CONNECTED)>], @default_ttl=604800, @options={:ketama_weighted=>true, :rcv_timeout=>0.25, :exception_retry_limit=>5, :verify_key=>true, :support_cas=>false, :timeout=>0.25, :server_failure_limit=>2, :exceptions_to_retry=>[Memcached::ServerIsMarkedDead, Memcached::ATimeoutOccurred, Memcached::ConnectionBindFailure, Memcached::ConnectionFailure, Memcached::ConnectionSocketCreateFailure, Memcached::Failure, Memcached::MemoryAllocationFailure, Memcached::ReadFailure, Memcached::ServerError, Memcached::SystemError, Memcached::UnknownReadFailure, Memcached::WriteFailure], :distribution=>:consistent_ketama, :buffer_requests=>false, :show_backtraces=>false, :use_udp=>false, :sort_hosts=>false, :tcp_nodelay=>false, :poll_timeout=>0.25, :prefix_delimiter=>”“, :binary_protocol=>true, :credentials=>[“…”, “…”], :no_block=>false, :cache_lookups=>true, :connect_timeout=>4, :default_ttl=>604800, :hash=>:fnv1_32, :retry_timeout=>30, :hash_with_prefix_key=>true, :default_weight=>8, :auto_eject_hosts=>true}, @not_stored=#<Memcached::NotStored: Memcached::NotStored>, @struct=#<Rlibmemcached::MemcachedSt:0x2ae2f3b1b470>, @not_found=#<Memcached::NotFound: Memcached::NotFound>, @logger=nil>

Perhaps you’ll think this is an error, potentially you might post it as a problem on StackOverflow, maybe after awhile you’ll even submit a support request to Heroku and have a long email conversation with a weirdly attractive Heroku employee who finally shuts you up completely by telling you, days later, “That’s not an error - that is the return value showing you that it’s working.”

Still reading? So once you’ve recovered your composure, you’re going to want to actually do something with memcached.

Figure out what to cache
There are a few uses of caching that are pretty standard across Rails apps. You can cache pages that are mostly static/look generally the same for all your users (skip to the next section if you just want to do that), you can cache fragments of a page that are frequently repeated, and you’re probably using some expensive database calls like ‘User.all’ that would be better to run once, cache the results of, and then update only when, for example, a new user is created. That would be something like this

#app/models/user.rb
def self.all_cached
Rails.cache.fetch(‘User.all’) { all }
end

def self.delete_cached
Rails.cache.delete(‘User.all’)
end

#anywhere in your application you were calling ‘User.all’, change to User.all_cached

#app/controllers/users_controller.rb
def create
User.delete_cached

However, a better way to know where caching will make the most sense for your app is to use some sort of performance analytics tool. Heroku pointed me to New Relic, which so far has been awesome. The support staff is extremely helpful (they actually called me on the phone after I signed up to see if I had any questions. That was a little weird, especially because the employee who called me for some reason has the same name as my dad, but I interpreted it as a nice gesture). Plus the dashboard is pretty



That’s actually kind of important to me because spending all this time on caching is extremely unsatisfying next to feature-building or even bug-fixing. All the hours of work you put into optimizing SQL queries or whatever just don’t change your site that much - pages load slightly faster. If the joy you get comes from building something, but not so much from your clever interactions with the database, this can get pretty frustrating. Something like New Relic at least lets you see some visual indication of your work.

New Relic has a freebie version that’s pretty easy to set up and use.

1) Add ‘newrelic_rpm’ to your gem manifest (.gems) or gemfile
2) In config/environment.rb somewhere in the ‘Rails::Initializer.run do |config|’ block add the line config.gem ‘newrelic_rpm’
3) Download the newrelic.yml file they provide and save it in your config folder
4) Restart your servers with the command ‘heroku restart’
5) Wait a little bit, and your opaque database transactions will start showing up in the New Relic dashboard as beautiful rainbow colors

Caching mostly-static pages
Page caching is the fastest type of caching, because it basically just stores the html that’s generated the first time a page is requested and then spits it back at whoever requests the page again, without having to talk to the database. If anything on that page needs to get updated, you can tell the cache to expire after a specified amount of time or whenever a particular action on your site is executed. Perfect! Except that the expiration part doesn’t work on Heroku. (Read this post about why not, or just take my word for it.)

So you have to use action caching instead. That’s almost as easy. Say we’re caching the index action

#app/controllers/root_controller.rb
X caches_action :index

Then you have to specify when you want the cache to expire. Since the only time my front page might be updated is when somebody backs a question, I stuck a line into my Backs Controller

#app/controllers/backs_controller.rb
…
def create
expire_action :controller => ‘root’, :action => :index
…
end

Everything was going great, until

#someone tries to login from the front page
ActionController::InvalidAuthenticityToken

What the hell? So I learned about this thing called an authenticity token, which is a randomly generated number that’s stored in a hidden field in your forms, to prevent people from submitting forms from outside your site (if that doesn’t make any sense, read this StackOverflow explanation). Basically, the authenticity token was being cached along with everything else on the page. At first, I just disabled this protection:

#config/environment.rb
X config.action_controller.allow_forgery_protection = false

But that made me feel so vulnerable I couldn’t fall asleep all night. The real fix turned out to be simple:

#app/controllers/rootcontroller.rb
caches_action :index, :layout => false

Assuming the page you’re working on uses a file from your app/views/layouts folder (mine didn’t, I had to change that), this will ensure that the code from that layout file isn’t cached. That means you can still have things that are specific to a user (like their name at the top and a link to their settings) or in my case, that the login form/authenticity token would be regenerated every time the page was requested.

Actually the problem wasn’t quite fixed



But that was because it was rendering an earlier cache of the page, as well as regenerating the layout code. All I had to do was clear out my cache:

#in my heroku console
ActionController::Base.cache_store.clear

That’s all, everything was fast and wonderful after that.

The Internet teaches you Ruby-on-Rails

I went to a Y Combinator Q&A last night and started getting frustrated when the audience almost completely shifted the talk to the problem of finding technical co-founders and how extremely difficult that is. Yes, it is difficult. More difficult, I think, than learning some of that tech yourself, which it sounded like most of these intelligent people weren’t trying to do at all.  I can’t say I’ve bootstrapped a CS education (my little brother is completely dismissive of my allegiance to Rails), but I knew essentially nothing about programming last spring and have since learned enough to handle most of the tech for a live, social application. To me that’s proof that basic-level programming does not require any special type of mind, but just regular hard work. 

At any rate, the talk prompted me to publish this post about learning to code (mostly) from the Internet.  This is meant to be useful, not profound - I started it yesterday based on an email of links I sent to my roommate, who is just working on his first Ruby project.

Start with a book - I went on a family vacation last June during which I had almost no Internet access, but I brought along the SitePoint Simply Rails 2 book with me. Most of the time I was there I was impatient to get back to actually building my app, but in retrospect that week of slower, guided learning was much more valuable. Rails does a lot of magic for you, and up until that point the way I coded was mostly to hack away at something until it worked. Learning just a little bit of background on how the magic works saves you time overall. 

If you don’t feel like paying for a book, I’d also recommend Michael Hartl’s Rails Tutorial, which takes you through making a Rails project along with version control on Github, deploying to Heroku, and testing.

Railscasts - If you ever bother to look carefully enough through Kommons.com, you might discover that almost all of the coolest things it does were at some point the subject of one of Ryan Bates’ 200+ Railscasts. They’re free (unlike Peepcode), usually under 10 minutes, and Ryan Bates has a lovely, soothing voice. I guess he’s a minor celebrity in the Rails dev community; the comments on his screencasts tend to be very positive, usually something along the lines of “OMG I love you Ryan!!!!!!!!!!!!!!”

StackOverflow - StackOverflow is extraordinary, I don’t really know how anyone finds anything on it, but you can post the most esoteric or stupid question and within the hour you will probably have an answer. They are kind of mean if your questions are especially stupid, but they give you an answer anyway.

Google - Occasionally, StackOverflow will fail you, or you’ll be so completely stumped by an error that you don’t know how you would pose a question, in which case my technique has been to cut and paste the error message you’re getting straight into Google. You’ll probably end up on the RailsForum (which I’ve never actually searched through myself, but come to via Google) and on a lot of Google discussion groups, but there are so many people trying to do what you’re doing that it’s likely someone has already written about your exact problem.

Docs - All the harder stuff is much easier to learn once you’re actually building, because you can instantly, intimately feel the reward of a deeper understanding. All the frustration you’ve built up from staring at error messages that don’t mean anything to you means that everything new you learn feels like a grand, personal discovery and opens up your eyes to the possibilities of what you can actually do with Ruby. So…what this means is diving into the Ruby docs and the Rails API, and looking up things like Enumerables, which you’ve probably been using for awhile without really getting, or ActiveRecord.

Github - If you’ve gotten to the point that you’re generally comfortable with Ruby syntax and the Rails MVC architecture, an interesting way to learn more is by reading through open-source projects on Github that have something to do with something you’re trying to do. You get to see the way a different person writes code and might see some patterns you want to repeat (not to mention lifting pieces of code straight into your own project - these programmers are pretty okay with that). Github search sucks a little bit, but you’ll probably be running into Github projects more than you want to during your countless Google searches.

I didn’t mention programmer-friends, since I was sticking to resources on the Internet, but working with someone IRL who can explain to you what s/he is doing is invaluable (shoutout to @jcn, @talsafran, @danielbachhuber, and the younger Ray, Lucas). I’ve also had some success changing my Gchat status to whatever tech issue I’m having, and having someone who’s online come to my aid.