As mentioned in a previous post about url_for, sometimes Rails magic needs to be de-emphasized or eliminated to attain the performance you need.

Rails provides excellent statistics to help you see how your application is humming along.

While you can go into the depth with various tools, like wrapping Benchmark block around code to see how it performs, the hints about where to look are in the logs.

A sample action will generate the following information in the logs:

Completed in 1.77621 (0 reqs/sec) | Rendering: 0.10928 (6%) | DB: 0.50002 (28%)

The discrepancy between the completed, rendering and db times lets know that you have overhead hidden away in an action.

Unless you are doing serious processing, like generating a PDF or accessing a web service, there is an obvious place to start.

The objects you are retrieving with ActiveRecord.

Having had experience with Apple’s WebObjects, I was familiar with the overhead involved in retrieving and instantiating objects.

Instantiating is the key place to look first.

In my case, the performance issue only revealed itself when a customer using the page ended up with thousands of records, not tens or even hundreds.

This action was using a typical AR finder method, with conditions to affect the results based on user input.

In development mode I was able to view the query and craft a custom finder that relied on find_by_sql.

On top of that, I honed down the columns retrieved to the bare minimum.

By default, ActiveRecord retrieves all columns, which can add significantly to the overhead during instantiation.

You can go lower, by retrieving the values using the bare bones database connection.

In my case, that wasn’t needed - yet.

The last month I spent optimizing and improving performance as one of my projects grew in use.

One of the page had to display hundreds and even thousands of links.

Taking pages from others discoveries, I learned that linkto can be dangerous in large doses. Well the real culprit in my case was urlfor.

Replacing it with a simple string concatenation method boosted performance 3-4 fold.

Its not the fault of Rails, free features and Rails magic always costs something.

In most cases it is an acceptable, reasonable and enjoyable cost.

This is also the pleasure of optimizing late, not early - or only when needed.

I have been relying on MySQL’s Full-Text search index for sometime.

Recently a user ran into a black hole in that feature.

He wanted to find records containing a simple phrase, “thank you”.

But, the search resulted in no records found.

You see, “thank” and “you” are stop words in the default full-text index. Meaning they are discarded or given no relevance in the search results.

After some digging, I discovered that others have run into this same limitation and opted to remove the default stop word list.

Quick fix? Not quite.

Removing the stop word list, forces you to rebuild the index. Which on my modestly sized database meant 30 minutes of downtime on that table.

Fair enough. I performed that over the weekend.

Fast forward a couple days. Guess what happens now?

Other users are getting less than stellar search results - of course. Seemed a fair trade off at the time, between the two extremes.

I could keep tuning the stop words and narrowing the gap between the two extremes, but tuning mysql full-text index takes far too long because of the table locking required to rebuild the index.

So I am moving on to external search indexers, like Sphinx. Which can rebuild the entire index in a few minutes without requiring the downtime for the users.

Macurls.com is a fun little application I wrote while learning Rails. It is a TinyURL.com clone I use on my uber-short domains (f8p.com. g5u.com, lubz.com, pv4.net).

I kept it around because I like seeing the stats for URL usage, and playing with new deployment and coding techniques occasionally.

This weekend I rewrote the application using the new kid on the block, Merb.

Honestly, not a lot of rewriting had to be done. Merb supports ActiveRecord, uses Ruby and deploys using Mongrell.

What did I get out of it?

Merb uses less memory, is faster and helps me follow my mantra - doing more with less.

Worth checking out.

This morning I finished configuring my SliceHost VPS. It went fairly well, being slightly Linux challenged, the Slicehost wiki, forums and support team helped me get everything done in about 4 hours.

Every slice they offer comes barebones, meaning how the various distributions ship. Slicehost simply adds the needed xen tools and default SSH configuration.

The only hick-up I encountered, was that Ubuntu 7.10 (Gutsy) had courier mail enabled by default.

Having chosen a 512 MB slice, with the plan of running Nginx, Mongrel and MySQL, I didn’t want the overhead of the mail server.

It was easy enough to remove the startup scripts, once I familiarized myself with the proper init.d commands.

With Hosted Solutions and Amazon EC2, I relied on their ACL/Firewalls to keep the riffraff out. ON my slice I had to setup iptables. Which was amazingly simple, specially compared to Leopard’s retarded firewall.

The optional backup service lets you take snapshots and have period backups of your slice. If anything goes awry you can restore at anytime through their slice management application. A great value for only $10 more a month!

Slicehost billing model lets you to add and remove slices as needed, only paying for what you use. This is similar to how Amazon EC2 works.

While they aren’t in the same game, on demand computing, SliceHost appears to solve my business challenge of scaling over time, without purchasing tons of hardware and rack space.

Over the next month I will see how performance compares as I move several projects.

Very nice for $48 a month!

Two big thumbs up so far.

Today I snapped up my slice of slicehost.com and so far I am loving it.

By the weekend I can turn down my EC2 instances and move some projects from the Xserves into their new Ubuntu home.

In December I am going to test running my largest project over there, with the hope it can move completely to a VPS solution in 2008.

Hmm. Wonder how long before OpenSolaris will be ready for the VPS market?

For those interested, I am running Ubuntu 7 (Gutsy) running Nginx, Ruby, Rails, Merb, MySQL and a liberal sprinkling of assorted gems.

Email is handled by my own carefully crafted powerhouse servers still.

DNS management is handled through Godaddy and EveryDNS.net.

As for DNS queries, again I highly recommend using OpenDNS. The ability to clear their caches manually after updates is a huge time saver.

”Like Ruby on Rails, Merb is an MVC framework. Unlike Rails, Merb is ORM-agnostic, JavaScript library agnostic, and template language agnostic, preferring plugins that add in support for a particular feature rather than trying to produce a monolithic library with everything in the core. In fact, this is a guiding principle of the project, which has led to third-party support for the ActiveRecord, DataMapper, and Sequel ORMs.”

(Via Merb | Looking for a better framework?.)

Ruby on Rails is where I dwell, having used it for a couple year now, but this new framework looks interesting.

This weekend I am going clone MacURLs.com in it and see how it performs.

My current data center provider, Hosted Solution, announced they are discontinuing their router-based ACL/Firewall service.

This has been in place for years and frankly provided a lot of security.

Since they were blocking at the outside router, instead of on the internal network things have been very safe and quiet since the day my servers were installed.

Now they are asking us to either implement comparable firewalls on every server, or pay them to install a firewall appliance for $199 or more each month, which will consume one of our paid rack spaces.

If it wasn’t such short notice, I would seriously consider moving the servers elsewhere.

I wonder how this will affect our bandwidth costs, given that malicious traffic will now make it inside the network, past the router they measure our usage on?

My attempt to use Media Temple’s Grid service went down in flames.

They were polite, but for technical reasons the Rails container didn’t let me setup their testapp, as documented in the knowledge base.

Now I am back in search for a good VPS provider.

Despite having a rack of XServes and Mac Minis at my partner company, I really want to explore the VPS world.

Burstable scalability is increasingly important in the blog-driven world. Where a single Digg can bring your application crashing down in flames.

For the past year I have been running several sites over on Amazon EC2, with good success. Only once did the server soft reboot and everything came back up perfectly.

The major drawback of Amazon EC2 is server instances have no permanent storage. If it goes down hard, you lose everything on the virtual disk.

It is fairly simple to rig up semi-permanent storage or backups by linking up to an S3 bucket, but performance and costs take a hit.

If anyone has any recommendation, please drop them in the comments.

I placed an order with Slicehost.com, but it is 2 weeks until I can get in there.

Tonight I am moving some of my projects to MediaTemplate.net’s Grid Server.

I tried it when it launched, but it was not fully backed at the time.

After waiting for Amazon’s EC2 to get some form of management for the virtual file system, I finally had to look elsewhere.

I will post my experience later, so far though their support team has been awesome!

Update #1:

8 hours after signing up my Ruby on Rails container is not working.

A server error is holding it hostage and I am in support limbo waiting for an “admin” to fix it, since they don’t support Ruby on Rails development.

They only dove in when I explained that I am just trying to get the testapp running as they outline in their knowledge base.

Update #2

Its the morning and still no Rails container at Media Temple. I disabled it and going to re-enable to see if it might come to life, but now they are trying to charge me $25 again to get what I had before.

Bon Voyage Update

This morning I called Media Temple, only to be informed that troubleshooting their Ruby on Rails Container was a courtesy and completely unsupported. This despite them confirming last night that “their” system was not working as it should.

Nothing is more confidence rattling in a hosting service than being told what you are paying for is unsupported and they are helping you as a courtesy.

I asked for and expect a full refund, we will see what they do.

With more information moving from Email to RSS, these feeds can now consume as much time as an overflowing Inbox.

After embracing the excellent Inbox Zero techniques Merlin Mann has helped spread, I realized that my RSS feeds had become the new bottleneck and a major time commitment each day.

Funny how removing one obstacle, can reveal a new one?

So, I fired up NetNewsWire and started looking for patterns that made sense.

The original organization, was a mix of folders that had unrelated meaning. Such as Mac news, Cool, Technology, etc.

Useless for any real purpose.

I decided to sort the feeds into groups that helped me prioritize my reading.

I chose not to apply action-based labels to the feeds since it is likely that the feeds will not lead to any actions for me to perform.

The feeds now look like this:

Essential are feeds that I should to keep an eye on.

Profitable are feeds that lead directly to business opportunities.

Casual are feeds that I enjoy from time to time, but are easily marked as read and ignored when I don’t have time.

Internal are feeds that run within my business and are not public.

Technical are for security and technical information that I migrated from email lists to RSS feeds.

Cleaning up my feeds has reduced the time it takes to digest the news from a couple hours to about 20-30 minutes.

Putting the feeds into prioritized groups helped, but there is another level of management you can automate.

NetNewsWire lets you set custom refresh settings for each group or feed.

The Casual group is set to only check every 24 hours and never do it when the feeds are manually refreshed.

This customization reduces distractions during a busy day when the feeds refresh and you want to check for important updates considerably.

Moving to this system has allowed me to capture more value from RSS feeds in far less time than ever before.

”What follows are instructions for building and installing MySQL 5 on Mac OS X. These instructions should work perfectly on both Tiger and Leopard.”

(Via Hivelogic - The Narrative - Installing MySQL on Mac OS X.)

Very interesting!

”Heroku: An Online Rails Development and App Hosting Environment”

(Via Heroku: An Online Rails Development and App Hosting Environment.)

This article says what I have failed to put into words in the past.

”There is really nothing religious about our use of open source. We use it because it’s better on the scales of merit that we care about. For infrastructure software, such as web servers, databases, server operating systems, programming languages, and web frameworks, the scales of merit lend themselves incredibly well to open-source development. Thus, we use it and are passionate about it.

For desktop operating systems? Not so much. There are just too many disciplines involved that programmers are not naturally good at and don’t have sufficient levels of taste to prepare masterfully. And programmers constitute the vast majority of builders in the open source communit”

(Via Ask 37signals: Why OS X and not Linux? - (37signals).)

I use Mac OS X on the desktop and open source for deployment for the exact same reasons.

With every new major release of Linux and Solaris, I take a look to see if either is coming close to the desktop experience I now enjoy.

Call me a fanboy or part of the cult if you like, but it really is true that I get more done using the platform I have than any other.

I need to post a list of the tools I can not live without, that I have not found true equals for on Linux or Solaris.

”The Sparkle framework just nails the process of providing an automatic update mechanism for Cocoa applications. I’m really impressed with how smoothly it works and the idea of using RSS as a notification mechanism, in the form of an appcast, is perfect. In fact, it works so smoothly that the part of me that automates everything wanted to take it one step further and make pushing a new version of a Cocoa application I’m working on as easy as pushing a button. Sort of like using Capistrano with Rails, but from within Xcode. After all, once Sparkle is wired into an application, doing the rest of the work to build archives, create feeds, and upload things seems like a real drag. So, last night I sat down and hacked away on it and have come up with a first cut solution.”

(Via James Duncan Davidson – Sparklemation.)