Four Years of Ruby Development Notes
Part of my series of notes from the Professional Ruby Conference. See them all here.
This talk is from Ezra Zygmuntowicz from Engine Yard.
He’s going to go over his history working and deploying with Rails.
Started with WEBrick, which rocked for development but obviously not for production. He first tried out Apache with FastCGI. Didn’t work that well either. Had to write scripts to kill zombies. Lighttpd + FastCGI was bad. Lighttpd + SCGI was bad. Another Apache combination that sucked too.
Then…MONGREL! Pure text, can hit individual mongrels, great platform for sure and still in use. Lighttpd’s proxy module failed. Lightspeed was next and it was nice, but their license sucks. Mongrel + Nginx was next! This combination works really well and is battle-tested. Ebb is actually faster than Mongrel since it’s single threaded and event-driven. Mongrel though is still the best general purpose server.
Passenger has done an awesome job, but it’s geared mostly towards development and shared hosting/small VPS. Nginx + Mongrel or Thin for high volume deployments. Passenger might be there someday, but not yet. Ruby Enterprise Edition helps with memory, but Apache is a hog so it doesn’t matter.
Rack: the great equalizer! Boils down any web server down to an object with a ‘call’ method on it. Makes it a lot easier to write handlers that are server agnostic. Rack is fast becoming a standard and it future-proofs your code/app.
Nanite: builds scalable backends for Ruby apps. Nanite ‘agents’ report back to a broker, which then fan out reports to various Nanite ‘mappers’ on the front end. It’s built around RabbitMQ, which is written in erlang. This means it’s fast, clusterable, and very scalable. Apps just need to appear fast, not actually be fast. This system is crazy useful for building apps on scalable cloud sites like EC2. Basically, it makes it easy to build scalable backends.
Engine Yard As a Service! Going to open it up for other hosting platforms to use their infrastructure with. Other services are very low level, but they want to make it easy and approachable. They’re going to attempt to apply the convention over configuration that Rails provides to deployment. Instead of logging into your server to make changes, you log into their application and describe what your app runs on. It’s the genetic information of your app and allows for greater portability. Allows you to build an entire server from scratch and deploy your app with one click. This is a much different way of thinking about deployment! No manual changes, all done through site’s interface. Ridiculous.
Hi, couldn’t help but notice:
“Passenger has done an awesome job, but it’s geared mostly towards development and shared hosting/small VPS. Nginx + Mongrel or Thin for high volume deployments. Passenger might be there someday, but not yet. Ruby Enterprise Edition helps with memory, but Apache is a hog so it doesn’t matter.”
I’m wondering where this claim is coming from, seeing as we’ve never designed Passenger to be “for just these purposes” (although it seems to do great in the mentioned scenarios as well).
In fact, it was designed and built from the ground to serve rails applications in as many conceivable scenarios as possible, including heavy traffic sites.
As a matter of fact, a lot of sites are currently serving hundreds of millions of requests through passenger on a daily basis and definitely not in a small VPS/shared hosting scenario. Some of them are mentioned in the following urls and include MTV, The New York Times, Yammer/Geni etc… and you can find more information on that here: http://groups.google.com/group/phusion-passenger/browse_thread/thread/521dd6281331dd9e/d5b4b1a863fff852
and
http://groups.google.com/group/phusion-passenger/t/14ac77db5810c54b
We’re actually working on case studies right now with some of the mentioned sites to highlight how Passenger has helped their situation. And if that can’t convince you, maybe our buddy Zed Shaw (author of Mongrel) can
http://www.zedshaw.com/blog/index.html
November 18th, 2008 at 9:17 pmNinh, I can guarantee what Ezra said about Passenger definitely isn’t as inflammatory as my notes have made out to be. He definitely said Passenger was a good thing, and I use it in production for all of the sites I run!
I’ve poked him on twitter about your response and I’ll attempt to hunt him down here at the conference.
November 18th, 2008 at 9:25 pmI never said passenger was designed only for small hosting. I just said that in my own testing that is what I would use it for. All my systems are 64bit, REE segfaults like mad on 64bit linux so passenger doesn’t offer me any memory savings, in fact its the opposite as apache uses a ton of resources compared to nginx.
Don’t get me wrong I;d love to use passenger and I think you guys are doing wonderful work. I just have nightmares about apache and so far passenger doesn’t give me any advantages over my tride and true, battle tested stack.
I expect this to change in the future when I can get over my fear of apache
So thumbs up to the phusion guys, you are doing great work. But there is still room for nginx + mongrel|thin for a while to come.
November 18th, 2008 at 9:38 pmNick:
Blegh, sensationalist blogger
Ezra:
Interesting. Unfortunately, we’ve been unable to reproduce the segfauls on 64bit ubuntu linux on an Athlon64.
We’ve disabled tcmalloc for 64 bit actually since it’s known not to work on those platforms, and it should use the standard memory allocator instead on those platforms.
Could you send us more information on the problems you’ve been experiencing and the setup you’re working with? Especially the architecture and CPU are important to know so we could at the very least take a look at it.
November 18th, 2008 at 9:52 pm@Ezra: I’m finding your logic a little difficult to follow. From what I read:
1) Apache uses ‘a ton’ more resources than Nginx
2) REE doesn’t work on 64-bit systems == no memory savings
3) Therefore Passenger isn’t good enough for use in large deployments (or at EY at least) because it uses more resources.
If the concern is resource usage, then why run on a 64bit platform in the first place? It doesn’t make any sense to me. I’ve ran identical mongrel+nginx setups on 32 and 64 bit platforms, and the memory usage by ruby was nearly *double* on the 64bit system as opposed to the 32bit system (see more here: http://www.ruby-forum.com/topic/137794).
I’ve also got Nginx and Apache running, and while Apache eats 10 times the amount of memory as compared to Nginx, we’re talking about 2MB difference (280Kb vs 2.2MB), which is a trivial difference compared to the difference of running a sizable rails/merb app on a 64-bit OS.
November 19th, 2008 at 11:01 pm[...] 7. Four Years of Ruby Development, from Ezra Zygmuntowicz [...]
November 21st, 2008 at 12:11 pm[...] Four Years of Ruby Development Notes This talk is from Ezra Zygmuntowicz from Engine Yard. He’s going to go over his history working and deploying with Rails. Interesting notes on upcoming Engine Yard offerings. Comments on Passenger. [...]
November 24th, 2008 at 4:19 am[...] reading some responses on Ezra’s talk on Rails deployment, and in particular, after reading this summary by an attendee of this talk, the impression seemed to be made that Passenger was only, if not more [...]
December 6th, 2008 at 11:43 pmNice post. Thank you for the info. Keep it up.
December 7th, 2008 at 7:10 pm