Computers, Open-source, Ruby, Software

Upgrading to Rails 4

Recently, I started a new job and the first big assignment was to upgrade their software stack to a more recent version. They are running Rails 3.2 and wanted to upgrade as far forward as they can. With Rails 3.2 support gone for all but severe bug fixes, and Rails 5 due any month now, this is something they wisely didn’t want to put off.

Its a smaller company, and they have been open to a lot of my feedback and suggestions. I was basically given the reins and told to do what needed to be done to get us upgraded.

So the first task was some research, and I stumbled upon the official Rails upgrade guide pretty quickly. It nicely outlines the breaking changes. Fortunately, the big change was to strong parameters, but this can be deferred by including protected_attributes and kicking this can down the road. We will be logging what controller actions receive which parameters, instead of raising so we will have some time to collect some data before we switch over in one painful release.

The guides stressed that the test suite is critical during the upgrade. I was fortunate enough to have a project with adequate testing coverage. It wasn’t the 80% sweet spot, but it was certainly valuable at ~40%. However, the suite had fallen into disuse, so the first task was to get them back to green.

Once the test suite was green, it became a matter of KEEPING it green. Luck smiled a second time and they had an old CI server that had fallen into disuse. It was powered by CruiseControl.rb and it was little fuss to get it back up and running again. The migrations could no longer be played from the projects inception to the current time.

This is where luck stopped smiling upon me. The project did not track db/schema.rb and the migrations were not playable. The only way to get an instance of the database was to download the schema from production. Not the best practice, so I went about tracking the schema, and getting adoption of this new practice. Further complicating the schema approach was the decision to move all older migrations into subfolders in db/migrate by year (e.g. 2011, 2012, etc). This was done I found out because Textmate doesn’t like big directories. The issue is that db:schema:load isn’t recursive in its retrieval of migration versions. It took me a bit to understand what was happening, and how it was happening. After a failed monkey patch to the migrator logic in ActiveRecord, I decided to just move the migrations back into db/migrate and eliminate the subdirectories. Sorry Textmate!

Now the database could be rapidly provisioned, and I got a seed working with a minimal set of data. Back in CI I reconfigured the build script to use db:schema:load instead of db:migrate and with the green test suite, we got builds working again.

We used a utility called CC Menu to show the build status in the notification bar in OS X: http://ccmenu.org/

To make the builds even more visible, I discovered an integration with Slack to report the build status in our chat. https://github.com/epinault/cruisecontrolrb-to-slack . I made my own fork and added some famous movie quotes for successes and failures since I found the default messages lacking: https://github.com/bsimpson/cruisecontrolrb-to-slack . I didn’t think our female developers would appreciate the “you’re a stud!” message.

Back to the Rails 4 upgrade. The tests are passing in master, so I made a feature branch that will be long lived called “rails-upgrade”. I merge master in daily. The “rails-upgrade” branch will serve as an integration point for other features branches that will merge into it. The plan is to keep any upgrade related changes out of master until its time to deploy. That means separate branches, separate CI builds, and separate staging servers for manual QA.

One lesson I’ve learned is that a deprecation warning may not always be just informational. In particular, Rails 4 requires all scopes to be callable (lambdas, or procs). This was breaking the way that associations with scopes would be built: users.roles.admin.find_or_create! would previously find an associated admin record, or create it. However, in Rails 4, it fails creation because the role’s reference to user is nil. I’m not sure why, but its reproducable, and changing the admin scope on Role to a callable restores this reference back to user.

Ideally, I’d have wanted to get the test suite green before tackling deprecation warnings because I want to change as little as possible before I get back to a known good status. However, not fixing this deprecation warning was actually causing tests to fail.

Now we are down to a handful of failings tests on Rails 4. Most deal with the ActiveRecord syntax changes. Hopeful I can get these knocked out quickly. Then its on to manual QA.

In summary – get your test suite green. Keep it green. Do the upgrade and get it back to green. Then make any changes to remove deprecation warnings, keeping the suite green. The test suite is your guide during an upgrade like this.

Advertisements
Open-source, Ruby, Software, Thoughts, Uncategorized

Implementing a Configuration

Have you ever seen a configuration in Ruby that yields to a block where properties are set on the yielded object? Rails does this with its environments files:

Your::Application.configure do
  config.cache_classes                     = true
  config.consider_all_requests_local       = false
  config.action_controller.perform_caching = true
  ...
end

Devise is another great example of this pattern:

Devise.setup do |config|
  config.secret_key = '38c8e4958385982971f'
  config.mailer_sender = "noreply@email.com"
  config.mailer = "AuthenticationMailer"
  ...
end

This is an established way to pass in configuration options in the Ruby world. How does this work? What are the alternatives?

Under the Hood

Lets start with our own implementation from scratch. We know that we are calling a method (Application.configure, or Devise.setup) and that method yields to a block:

module Example
  class << self
    def configure
      yield Config
    end
  end
end

You can see that this will yield the class Config when we call Example.configure and pass in block. Config will need to define our properties:

module Example
  class Config
    class << self
      attr_accessor :api_url, :consumer_key, :consumer_secret
    end
  end
end

We can now call our configure method and pass in a block:

Example.configure do |config|
  config.api_url = "http://api.com/v1/"
end

If we try to set a property that does not exist in our Example::Config class, then we get a helpful NoMethodError:

Example.configure do |config|
  config.not_a_real_config_setting = "http://api.com/v1/"
end

# => NoMethodError: undefined method `not_a_real_config_setting=' for Example::Config:Class

This helps to define your configuration interface. Options are explicitly declared. They cannot be mistyped, or dynamically added without an explicit runtime error.

In practice, The definitions of Example, and Example::Class would live in a gem. The Example.configure invokation would live within the code that wants to configure the properties of that gem with things such as credentials, URLs, etc. This seperation of concerns makes sense. Gems should not know anything about the business logic of your application. It should be instantiated with configuration options passed in from the project. However the project should only be aware of the public interface of the gem. We are agreeing upon where these pieces of information are moving from one domain to another. The project doesn’t know where this information will be used, and the gem doesn’t know what the information is until we pass it. So far so good!

Using this Configuration

Now that we’ve passed our information inside the configuration block, we can reference these class level (static) properties in our gem:

module Example
  class Request
    def self.get(path)
      request = Net::HTTP.get(URI.parse(ExampleConfig.api_url + path))
      request['Authorization'] = auth_header(:get, ExampleConfig.api_url)
    end

    private

    def self.auth_header(request_type, uri)
      SimpleOAuth::Header.new(request_type, uri.to_s, {},
                              {consumer_key: Config.consumer_key,
                              consumer_secret: Config.consumer_secret}).to_s
    end
  end
end

This will do a simple GET request passing in SimpleOAuth headers. Inside the get method we call Config.api_url to know where the API lives. This was set by us earlier using the Config object. SimpleOAuth headers are supplied by again calling the Config. You would invoke it like so:

Example.configure do |config|
  config.api_url = "http://api.com/v1/"
  consumer_key = "1895-1192-1234"
  consumer_secret = '76asdfh3heasd8f6akj3hea0a9s76df'
end

Example::Request.get('/products') # => {products: [product1, product2, etc]...}"
Example::Request.get('/users') # => "{users: [user1, user2, etc]...}"

Example::Config becomes the holding location for your configuration information. And by making the properties static, you don’t have to worry about passing around a reference to the instance.

Alternatives

If the yielding to a block is a little too clever for you, you can always instantiate a class and pass in the configuration as part of the constructor:

class Example
  class << self
    attr_accessor :api_url, :consumer_key, :consumer_secret

    def config(api_url:, consumer_key:, consumer_secret:)
      self.api_url = api_url
      self.consumer_key = consumer_key
      self.consumer_secret = consumer_secret
    end
  end
end

This can be instantiated like so:

Example.config(
  api_url: "http://api.com/v1/"
  consumer_key: "1895-1192-1234"
  consumer_secret: '76asdfh3heasd8f6akj3hea0a9s76df'
)

Example.api_url # => "http://api.com/v1/"

This feels less encapsulated to me. Instead of having an interface for our configuration settings, we are just settings properties directly onto a class.

What are your thoughts? What advantages does the block style configure offer over the alternative above?

For more information on the Ruby gem configuration pattern see this excellent blog post:

Computers, Ruby, Software

Lets Talk Strategy

Have you ever used a promotional code on an e-commerce website? How does the promotional code know what items are eligible? Is it a fixed amount off, or a percentage? Does it include shipping and handling? Was there a minimum purchase amount you had to meet to be eligible? These are all design considerations that needed to be taken into account, and the strategy pattern was a great way to represent this in my codebase.

My new requirement was to incorporate a promotional code that took 10% off the shopping cart total (excluding certain items of course – why can requirements never be simple?!) The system already had an existing promotional code system, but it was rudimentary, and only worked for fixed amounts off the total (e.g. $5 off your total).

The strategy pattern is defined as having these qualities:

  • A strategy can be thought of as an algorithm that is conditionally applied at runtime based on conditions.
  • The strategy encapsulates the execution of that algorithm.
  • Finally a strategy is an interface for the algorithm allowing them to be interchangeable.

Implementing the Strategy Pattern

Lets look at the existing promotional code logic before I implement the logic to take a percentage discount:

# app/models/shopping_cart.rb
...
has_many :items

# Returns the most expensive item in the shopping cart
def costliest_item
  items.map { |a,b| b.price <=> a.price}.first
end

def apply_promotional_code!(promotional_code)
  costliest_item.apply_promotional_code! promotional_code
  shopping_cart.total -= promotional_code.amount
end

For added context, a promotional code has a code we can lookup, and an amount property that tells us how much that promotional code is valued at.

We have a ShoppingCart which has many Items associated with it. When we call apply_promotional_code! and pass in a promotional code (e.g. “HOTSUMMER”), it returns the costliest item in the shopping cart (based on price) and associates the promotional code against it.

Why lookup the costliest item at all? Because we want to show an itemized receipt of each item total in the shopping cart. We then subtract this promotional code amount from the shopping basket total, which is the amount that will be charged to the customer’s credit card.

Adding a percentage based promotional code

If we were to change this to a percentage based discount (e.g. 10%) instead of a fixed amount (e.g. $5) we would need to change our calculation in the apply_promotional_code! method to something more complex:

def apply_promotional_code!(promotional_code)
  costliest_item.apply_promotional_code! promotional_code
  if promotional_code.amount.present?
    shopping_cart.total -= promotional_code.amount
  else
    shopping_cart.total -= shopping_cart.total * (promotional_code.percentage / 100.0)
  end
end

We have also changed the number of items this can be applied against (shown in our itemized receipt). If the promotional code is 10% off of the total, then it is 10% off of each item in the shopping cart. This means we will have to revisit our costliest_item method to make it return an array of zero or more items, instead of a single item:

# Returns an array of eligible items for the promotional code
def eligible_items
  if promotional_code.amount.present?
    Array(items.map { |a,b| b.price <=> a.price}.first)
  else
    items
  end
end

def apply_promotional_code!(promotional_code)
  eligible_items.each {|item| item.apply_promotional_code! promotional_code }
  if promotional_code.amount.present?
    shopping_cart.total -= promotional_code.amount
  else
    shopping_cart.total -= shopping_cart.total * (promotional_code.percentage / 100.0)
  end
end

With the addition of a percentage based promotional code alongside the fixed amount promotional code, we have introduced more complexity. apply_promotional_code! now has to be aware of how to apply both a fixed, and a percentage based amount. Further, the eligible_items (formerly costliest_item) now needs to be aware of the type of promotional code. Our promotional logic is spilling over into areas that need not be concerned with promotional logic.

Encapsulating via a strategy

Instead of concerning apply_promotional_code! with the knowledge of all possible types of promotional code types, lets decompose this into two separate strategies. The first will be our original fixed amount against the costliest item in the shopping cart. The second will be our new percentage based discount off the entire shopping cart:

# app/models/promotional_codes/stategies/fixed_amount_against_costliest_item.rb
class FixedAmountAgainstCostliestItem
  attr_accessible :shopping_cart, :promotional_code

  def initialize(items, promotional_code)
    self.shopping_cart = shopping_cart
    self.promotional_code = promotional_code
  end

  def discount
    promotional_code.amount
  end

  def eligible_items
    shopping_cart.eligible_items
  end
end

class PercentageAgainstMultipleItems
  attr_accessible :shopping_cart, :promotional_code

  def initialize(items, promotional_code)
    self.shopping_cart = shopping_cart
    self.promotional_code = promotional_code
  end

  def discount
    shopping_cart.total * (promotional_code.percentage / 100.0)
  end

  def eligible_items
    shopping_cart.eligible_items
  end
end

A few things to note: Firstly there are now two classes, and each is responsible for a single type of promotional code application. Secondly, there is similarity in these two classes. This is a good sign that we want to refactor and make a common super class that these inherit from. Similarity means there is a good chance we can accomplish interchangeability. Lastly, we have proxied the logic of what items are eligible through our two new strategies. As a next pass we will move the eligible_items logic to reside with the strategy. This keeps our promotional code logic in one place:

class BasePromotionalStrategy
  attr_accessible :shopping_cart, :promotional_code

  def initialize(items, promotional_code)
    self.shopping_cart = shopping_cart
    self.promotional_code = promotional_code
  end

  def discount
    raise NotImplementedError
  end

  def eligible_items
    raise NotImplementedError
  end
end

class FixedAmountAgainstCostliestItem < BasePromotionalStrategy
  def discount
    promotional_code.amount
  end

  def eligible_items
    Array(items.map { |a,b| b.price <=> a.price}.select { |item| item.price >= promotional_code.amount }.first)
  end
end

class PercentageAgainstMultipleItems < BasePromotionalStrategy
  def discount
    promotional_code.amount
  end

  def eligible_items
    items
  end
end

Now that we have extracted the common elements into a BasePromotionalStrategyclass, both FixedAmountAgainstCostliestItem and PercentageAgainstMultipleItemscan extend it. With the boilerplate constructor logic out of the way, we can define our interface with method placeholders. As an example of what eligible_items might guard against, we will check that the item cost is greater than or equal to the promotional_code.amount to ensure our shopping cart has a positive balance. (So much for early retirement by buying thousands of paperclips one at a time with a $5 off coupon!). Percentage based promotional codes have a sliding value so there is no minimum spend we need to guard against. You can see that we just return all items as eligible_items.

Regardless of whether we instantiate FixedAmountAgainstCostliestItem or aPercentageAgainstMultipleItems, we can call discount to calculate the amount taken off the shopping cart, and we can call eligible_items to return a list of items that the itemized list should show the promotional discount applying to. In the case of theFixedAmountAgainstCostliestItem it will always be an empty array, or an array with one item, but we consistently return an array type regardless of the number of entities it contains. Consistent return types are an interface must!

Calling our new strategies

Now that we have implemented (and written tests – right?) for our strategy, we can revisit our caller to take advantage of this new logic:

# app/models/promotional_code.rb
class PromotionalCode
  ...
  def strategy
    if amount.present?
      PromotionalCode::Strategies::FixedAmountAgainstCostliestItem
    else
      PromotionalCode::Strategies::PercentageAgainstMultipleItems
    end
  end
end

# app/models/shopping_cart.rb
...
has_many :items

def apply_promotional_code!(promotional_code)
  strategy = promotional_code.strategy.new(self, promotional_code)
  strategy.eligible_items.each { |item| item.apply_promotional_code!(promotional_code) }
  shopping_cart.total -= strategy.discount
end

The PromotionalCode is responsible for knowing which strategy will be executed when it is applied to a shopping cart. This strategy method returns a class reference. This reference is then instantiated passing in the necessary arguments of the shopping_basket, and the promotional_code. This is everything the shopping cart needs to know about executing a promotional code strategy. The ShoppingCart does not know which strategy to instantiate, but simply which arguments to pass to the return result of strategy.

Further, ShoppingBasket now knows nothing about what items are eligible for a PromotionalCode. That logic again resides within the strategy definition. Our promotional code domain is nicely encapsulated. This adheres to the open/closed principle of good object oriented design. Adding a new type of promotional code (e.g. buy one get one free, or a promotional code with a minimum spend) would be as easy as adding a new strategy, and telling the PromotionalCode#strategy when to apply it.

NullStrategy

You might have noticed that we use an if...else as opposed to an if...elsif (for simplicity sake). This means that the percentage based promotional code strategy is returned more often than it should be. Ideally, we’d want to return an inert strategy that confirms to our interface but doesn’t actually give any discounts to our shopping cart. This is where a Null Object pattern could be used in conjunction with a strategy:


# app/models/promotional_codes/stategies/null_strategy.rb
class NullStrategy < BasePromotionalStrategy
  def discount
    0
  end

  def eligible_items
    []
  end
end

# app/models/promotional_code.rb
class PromotionalCode
  ...
  def strategy
    if amount.present?
      PromotionalCode::Strategies::FixedAmountAgainstCostliestItem
    elsif percentage.present?
      PromotionalCode::Strategies::PercentageAgainstMultipleItems
    else
      PromotionalCode::Strategies::NullStrategy
    end
  end
end

One additional benefit of this strategy pattern is the ease of testing. PromotionalCode#strategy could easily be stubbed to return this NullStrategy and still remain consistent with the interface we’ve established in our other strategies. Now in testing, we can simply stub the NullStrategy and continue to chain our interface method calls:

 PromotionalCode.any_instance.stubs(strategy: PromotionalCode::Strategies::NullStrategy) 
 $> PromotionalCode.new.strategy # => PromotionalCode::Strategies::NullStrategy

 $> PromotionalCode.new.discount # => 0
 $> PromotionalCode.new.eligible_items # => []
 ...
 $> shopping_basket.total -= promotional_code.discount

Conclusion

Strategies can be a great way to split different algorithms apart. An indicator that you could benefit from a formalized strategy pattern is when you find conditionals in your methods that change the way something is calculated based on context. In our case, we had a conditional (if...else) that checked whether amount was set on a PromotionalCode. Another indicator was the need to encapsulate the concern of a PromotionalCode. Its neither a concern of ShoppingBasket nor Item. It needed its own home. By implementing a strategy class, we give this logic an obvious place to live.

A strategy can be a great tool when you have a requirement that lives alongside an existing requirement that changes how a calculation is done at runtime.

Additional Information on strategies can be found at https://sourcemaking.com/design_patterns/strategy

Computers, Open-source, Ruby, Software

Don’t Use “#” In the Paperclip Gem

I’ve learned a whole lot more about ImageMagick commands last week than I ever really wanted to know. The problem was that our uploaded images were having content cropped off the top, bottom, and sides.  Like many folks in the Rails world, we pass our attachments through Paperclip to handle all of the nitty gritty resizing operations.

I was interested in understanding how we could prevent our content from being cropped off when I came across an interesting idiom in the geometry settings:

has_croppable_attachment :image,
styles: {
:'630x315' => { geometry: "630x315#", format: :jpg },
}

Well take a look at that. There is a “#” symbol suffixed to my image geometry. I went to ImageMagick to lookup what this flag meant. Spoiler alert: It doesn’t exist there. After some digging around, I discovered that this idiom is provided by the Paperclip gem, and translates to the following convert command:

convert '/path/to/source.jpg' -resize "630x" -crop "630x315+0+0" '/path/to/output.jpg'

You can see the resize + crop combination of commands being built by Paperclip according to their documentation:

Paperclip also adds the “#” option (e.g. “50×50#”), which will resize the image to fit maximally inside the dimensions and then crop the rest off (weighted at the center).

Well, that is no good! If an image is not a 2:1 aspect ratio as per my dimensions 630×315 (or whatever aspect ratio you have) YOU WILL LOSE CONTENT! Time to rethink this…

The dimensions are 306x306
The dimensions are 306×306

 

The image was scaled until it was large enough to cover a 630x315 canvas, then the tops and bottoms cropped off
The image was resized until it was large enough to cover a 630×315 canvas, then the top and bottom was cropped off

Instead of resizing maximally (so that an image is a minimum of 630 width AND a minimum of 315 height) lets resize minimally (so that an image is a maximum of 630 width OR 315 height). The aspect ratio is preserved in both scenarios.

We want to resize while preserving aspect ratio , but we also need to make our canvas 630×315. The canvas dimensions are referred to as the extent command. When we do this, we will likely have space on the top and bottom, or sides we need to fill to have exactly these dimensions. What you fill this background with can be a color (in my case white). We also likely want to center the minimally resized image on this canvas. You can pass these convert options into Paperclip like so:

convert_options: {
:'630x315' => " -background white -gravity center -extent 630x315",
}

The resulting command will look something like this:

convert '/path/to/source.jpg' -resize "630x315" -background white -gravity center -extent 630x315 '/path/to/output.jpg'

Notice that our lossy crop flag has been replaced with a nicer extent flag. We can see the results this has on a similarly sized image:

We have an image smaller than the target 630x315
We have an image smaller than the target 630×315

 

We now have a 630x315 image with the sides filled in with a white background to preserve the dimensions
We now have a 630×315 image with the sides filled in with a white background to preserve the dimensions. Note the sides of the image are white.

As a final note, this works for both images larger and smaller than the target outcome dimensions. Looking at the ImageMagick documentation for flags can be helpful, but daunting as the real power lies in chaining multiple flags for a desired effect.

With a little effort I was able to get what is (in my opinion) a better image resize with just a few custom flags.

Computers, Linux, Open-source, Ruby, Software, Web

Migrating from Bamboo to Cedar

Heroku recently sent me a nice email:

You have one or more applications deployed to Heroku’s Bamboo stack. Bamboo went live almost four years ago, and the time has come for us to retire it.

You may have received a previous message about Bamboo deprecation. That message may have had an erroneous app list, but this email has the correct list of Bamboo apps that you own or are collaborated on. We’re sorry for any confusion.

On June 16th 2015, one year from now, we will shut down the Bamboo stack. The following app/apps you own currently use the Bamboo stack and must be migrated:

This is on the heels of an email about ending legacy routing support. It seems they have been quite busy over at Heroku, but I can’t complain too seriously about free app hosting.

Upgrading a legacy Rails application to the Cedar stack did require a few changes. I’ll document some stumbling blocks for posterity.

Foreman

The first big change described in the general purpose upgrading article from Heroku: https://devcenter.heroku.com/articles/cedar-migration was the use of foreman to manage your web services. I luckily have a simple app, and did not need to worry about resque, mailers, etc. This made my Procfile rather straightforward:

web: bundle exec unicorn -p $PORT -E $RACK_ENV -c config/unicorn.rb

Absent from the foreman crash course is any information about the corresponding .env file. I did need to add a few environmental variables to keep on working in development as usual:


RACK_ENV=development
PORT=3000

Once the Procfile was committed (end the .env) file added to .gitignore I then added Unicorn to the gemfile. I’m not sure if Unicorn is strictly necessary over webrick in development, however this is the example in the Cedar upgrade guide, so I wanted to run as close to production as practical to prevent any surprises.

After installing Unicorn, I then needed to touch config/unicorn.rb since it did not exist. Against, its in the example, but I’m not sure if its strictly necessary especially given that its just an empty file for me. To start your Rails application, you now issue foreman instead of of the older rails s

Devise incompatability

Not directly related to the Cedar changes, but a common gem, so worth mentioning. Devise 2.x has removed migration helpers. I stupidly didn’t lock my version of Devise in my Gemfile so I was confused why this was failing for me. I found the last 1.x version of Devise by running this command:

gem list devise --remote --all, and the specifying ‘1.5.3’ as the second argument in my Gemfile on the devise entry.

PostgreSQL

Heroku requires Postgres in production and I was previously using sqlite in my development environment. Again, to mirror production I wanted to use Postgres my development environment so that I could be as close to production setup as practical. I took a quick trip down memory lane to setup and configure PostgreSQL on my Linux development machine: https://mrfrosti.com/2011/11/postgresql-for-ruby-on-rails-on-ubuntu/ . An interesting observation is that in Ubuntu 14.04 LTS, PostgreSQL runs on port 5433, and not the default 5432. This can be verified with netstat -nlp | grep 5432. If no entries come back, PostgreSQL is running on a non-default port. Try to grep for 5433, or other numbers until you find the process.

Backing up your database

Before I made any server changes, I wanted to have an up to date copy of the production databse on my machine to prepare for the worst. This can be done by using the pgbackup commands:


heroku pgbackups:capture //create a backup
heroku pgbacksup:url //make a public URL for that backup

Then I clicked the public URL and downloaded the file – safe and sound to my computer. By the way, you can import this production database into your local instance of PostgreSQL using the following command:


pg_restore --verbose --clean --no-acl --no-owner -d development /path/to/database

Pre-provisioning your database

A quick note on database availability. I got into a chicken and the egg scenario where the app wouldn’t deploy without the database, and the database seemingly couldn’t be created without the app first being deployed. Heroku has an article on pre provisioning and I found it a necessary prerequisite to deploying to my newly created test Cedar stack: https://devcenter.heroku.com/articles/pre-provision-database

To pre-provision your database, run the following:

heroku addons:add heroku-postgresql

You can even import the database from production, etc as part of the heroku utility. I used the public database URL I created above to populate my new cedar stack database:


heroku pgbackups:restore

Migrating to Cedar

Once I had tested on a new app, and in a feature branch I had confidence everything was working as expected. I was ready to make my changes to the production app by migrating it to Cedar. To do this, the command is:


heroku stack:migrate cedar

I either have an old version of the Heroku gem, or this is a discrepency in the gem and the non-gem packaging, but the docs misidentify this command as: heroku set:stack cedar which was not a valid command for me. The migrate command above appears to be analagous.

Once I merged my cedar feature branch back into master I was ready to push to master. And FAIL. It turns out that I needed to precompile my assets, which had a dependency on the database existing. I tried to pre-provision as I had done on my cedar branch, however the results were the same after running this command.

A quick search yielded https://devcenter.heroku.com/articles/rails-asset-pipeline#troubleshooting the advise to add the following line in the config/application.rb file:

config.assets.initialize_on_precompile = false

Summary

I’ve learned quite a bit about Heroku in this upgrade experience. Their changes force me to use the latest software which is nice in some ways. When everything is running on my website, I don’t often worry about upgrading until I get an email like the one above.

The downside of course is that this upgrade process is a pain in the ass, and is error prone, and affects production websites that are running smoothly. If it isn’t broken, you don’t want to fix it. Except this time you have to in order to have it continue to function after June, 2015.

Best of luck to other people upgrading. Just be patient, and test everything in a new app if you have any doubts.

Computers, Open-source, Software, Thoughts

AngularJS File Uploads with HTML5 FileAPI

AngularJS has an interesting gap in functionality that can make working with file uploads difficult. You might expect attaching a file to an <input type=”file”> to trigger the ng-change event, however this does not happen. There are a number of Stackoverflow questions on the subject, with a popular answer being to use a native onclick attribute and call into Angular’s internals (e.g. onchange=”angular.element(this).scope().fileNameChaged()”)

This solution feels brittle, and relies on some unsupported Angular interactions from the template. To work around this issue, Github user danialfarid has provided the awesome angular-file-upload library to simplify this process by extending Angular’s attributes to include ng-file-select. This is a cleaner implementation. This library also includes an injectable $upload object and its documentation shows how this abstracts the file upload process in the controller. This abstraction (if used) sends the uploaded file to the server immediately, and without the contents of the rest of the form. I wanted to submit this file change with the traditional all-at-once approach that HTML forms take. This way, the user can abandon form changes by neglecting to press the submit button, and keep the original file attachment unmodified.

In order to achieve this, I’ve created a solution that uses the HTML5 FileAPI to base64 encode the contents of the file, and attach it to the form. Instead of reinventing the ng-file-select event, I opted to use the angular-file-upload library described above. However instead of using the injected $upload functionality referenced in its README, we will serialize the attachment with a base64 encoded string.

To begin, create an AngularJS module for your application, and include the angularFileUpload dependency:

window.MyApp = angular.module('MyApp',
  [
    'angularFileUpload'
  ]
)

Next, we will create our AngularJS template and include our HTML input tags:

<div ng-controller="MyCtrl">
  <form ng-submit="save()">
    <input type="file" ng-file-select="onFileSelect($files)" />
    <input type="submit" />
  </form>
</div>

Now we can create our AngularJS controller, and define the onFileSelect function referenced in the the ng-file-select attribute:

class exports.MyCtrl
  @$inject: ['$scope', '$http']

  constructor: (@scope, @$http) ->
    @scope.onFileSelect = @onFileSelect

  onFileSelect: ($files) =>
    angular.forEach $files, (file) =>
      reader = new FileReader()
      reader.onload = (e) =>
        @scope.attachment = e.target.result
      reader.readAsDataURL file

  save: =>
    @$http(
      method: 'POST',
      url: "/path/to/handler",
      data:
        $.param(
          attachment: @scope.attachment
        )
      headers:
        'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'
        'Accept': 'text/javascript'
    )

Our controller is now in place. When the input’s attachment changes, onFileSelect is called which iterates through the collection of files (if multiple) and creates a FileReader instance for each one. The reader then has functionality attached to its onload event in the way of assigning the result to an attribute in our @scope object. The call to readAsDataURL starts reading the file and creates a data: URL representing the file’s data as a base64 encoded string.

Once the form is submitted, the save function is called from the value of ng-submit on our form tag. This performs a standard AngularJS XHR action, and includes the attachment assignment in the params. I have adjusted the Content-Type in the headers to communicate to the server that the content contains URL encoded data. If we had other form fields, we could serialize and append them to the params collection to send to the server alongside the attachment in the same place in the code.

Image Attachments

For added feedback to the user on image attachments, the img tag’s src attribute can accept a base64 encoded string as a value. Since we have this value from our FileReader object, we can update the view instantly with the file without doing any server side processing. To achieve this, we can add an image tag to our HTML file:

<div ng-controller="MyCtrl">
  <form ng-submit="save()">
    <img ng-src="{{attachment}}" />
    <input type="file" ng-file-select="onFileSelect($files)" />
    <input type="submit" />
  </form>
</div>

Next, we can make a few modifications to our onFileSelect function:

onFileSelect: ($files) =>
  angular.forEach $files, (file) =>
    if file.type in ["image/jpeg", "image/png", "image/gif"]
      reader = new FileReader()
      reader.onload = (e) =>
        @scope.$apply () =>
          @scope.attachment = e.target.result
      @scope.reader.readAsDataURL file

AngularJS two way data binding takes care of the messy details for us. The template is bound to @scope.attachment_url. We do some safety checks that the filetype is an image, and then we assign the attachment_url key to the base64 encoded image. A call to scope.apply() will repaint the screen, and the user will see the image they have attached displayed.

Thanks to Nick Karpenske, and Robert Lasch for help with the implementation!