Apple, Computers, Linux, Open-source, Ruby, Software, Thoughts, Web

PostgreSQL for Ruby on Rails on Ubuntu

My new desktop came in at work this week, and the installation was painless thanks to the great driver support of Ubuntu 11.10. For anyone setting up a Rails development box based on Linux, I have some tips to get around some pain points when using a PostgresSQL database.

Installation:

Postgres can be quickly and easily installed using apt-get on Debian or Ubuntu based distributions. Issue the command:

apt-get install postgresql

Ruby Driver

In order for Ruby to connect to PostgreSQL databases, you will need to install the pg gem. This gem will need the development package of PostgreSQL to successfully build its native extension. To install the PostgreSQL development package, issue the following command:

apt-get install libpq-dev # EDIT: postgresql-dev was replaced by this package on Ubuntu 11.10

Setup A PostgreSQL Role

You can configure PostgreSQL to allow your account to have superuser access, allowing your Rails tasks to create and drop databases. This is useful for development, but is strongly discouraged for a production. That being said, we can create a PostgreSQL role by logging into psql as postgres as follows:

su postgres -c psql

This will open a PostgreSQL prompt as the database owner postgres. Next, we need to create an account for our user. This should match the response from “whoami”:

create role  superuser login;

We can now exit from psql by issuing “q“. Try to connect to psql directly by issuing the following command from your shell account:

psql postgres

This should allow you to connect to the default database postgres without being prompted for credentials. You should now be able to issue the rake commands for creating, and dropping the database:

rake db:create

Rspec Prompts for Credentials

I was being prompted by Rspec for credentials when running my test suite. If you would like to remove this credential prompt, please read the following:

There are differences in how the PostgreSQL package is configured in Homebrew on OS X, and how it is packaged in the Ubuntu and across other distributions. One difference is in the level of security configured in the pg_hba.conf file. This file is responsible for identifying which sources using which authentication mechanisms should be allowed or denied. By default, Rspec will cause a prompt for a password even if your shell account has trusted permissions. This is because Rspec connects not as a local process, but to localhost. To allow connections to localhost to be trusted, you will need to modify the pg_hba.conf file.

Next, we can modify the pg_hba.conf file located at /etc/postgresql/<version>/main/pg_hba.conf

Comment out the lines any lines at the bottom of the file and append the following:

local   all             all                                      trust
host    all             all              127.0.0.1/32            trust
host    all             all              ::1/128                 trust

This will allow connections from the shell, as well as connections to 127.0.0.1 (localhost) using both IPv4 and IPv6.

You will need to restart PostgreSQL for the changes from this file to take affect:

/etc/init.d/postgresql restart

PostgreSQL Extensions

If you want to make use of any of the additional extensions to Postgres, including fuzzystrmatching, you will need to install the postgresql-contrib package:

apt-get install postgresql-contrib

The extensions will install to /usr/share/postgresql/<version>/extension/

Using the Postgres version 9, you can create these extensions in your database by using the new CREATE EXTENSION syntax. In the case of the fuzzystrmatch extensions, you can issue the following command from inside a PostgresSQL command prompt to load the extensions:

psql ;

Once inside your database:

create extension fuzzystrmatch;
Computers, Personal, Software, Thoughts, Web

One Step Forward and One Step Back

The ArchitectAnother section of the Fredrick Brook’s The Mythical Man Month seems to resonate with software development today:

“Lehman and Belady have studied the history of successive releases in a large operating system. They find that the total number of modules increases linearly with release number, but that the number of modules affected increases exponentially with the release number. All repairs tend to destroy the structure, to increase the entropy and disorder of the system. Less and less effort is spent on fixing original design flaws; more and more is spent of fixing flaws introduced by earlier fixes. As time passes, the system becomes less and less well-ordered. Sooner or later the fixing ceases to gain any ground. Each forward step is matched by a backward one. Although in principle usable forever, the system has worn out as a base for progress. Furthermore, machines change, configurations change, and user requirements change, so the system is not in fact usable forever. A brand-new from-the-groud-up redesign is necessary…”

“Things are always at their best in the beginning” – Pascal.

“That is the key to history. Terrific energy is expended – civilizations are built up – excellent institutions devised; but each time something goes wrong. Some fatal flaw always brings the selfish and cruel people to the top, and then it all slides back into misery and ruin. In fact, the machine conks. It seems to start up all right and runs a few yards, and then it breaks down” – C. S. Lewis

Maybe we are meant to throw our original ideas away and pave the road for its successor? Its not a failure, its just the next step. When we implement a project, we do the best we can with the tools and knowledge we have at our disposal. But at some point, the original project becomes so outdated because of changing functionality, or demands that its time to start over.

I would love to see us make progress on the next iteration of our project with all of the knowledge we have garnered from the sins of our current Portal. We could make something truly viable and revolutionary for its constituents. Its just a matter of time and energy. Lately, it seems that the mindset is to continue monkey-patching our current system and pray it helps. I am a firm believer that if something is broken enough, its just time to move on. We can only be as good as the limitations that are imposed upon us allow.

Computers, Events, Open-source, Personal, Ruby, Software, Thoughts

Change is in the Air (I’m So Excited… I’m so Scared)

Lately my work has been full of exciting developments. Inside of a week, our department head decided to tackle the single largest project in the history of our department. I am not supposed to talk about the specifics, so I apologize in advance for being vague.

The scope of this new project requires significant change in our department from our past approaches. Specifically we will need massive amounts of planning and collaboration with each other.

Until now, all of our assignments have been “Ben, make this, Luca, make that, etc”. Once the application has been written, the developer becomes the maintainer, and troubleshoots any issues with it and writes any updates for it in the future.

Advantages of this approach:

  • Faster development (only one person in the driver’s seat)
  • Troubleshooting is simplified;  Problems with program ‘x’ goes to Ben – he wrote it after all.
  • Each application is self reliant and self contained.

These could be considered “good thing[s]”, until we look at the disadvantages:

  • Mishmash of code bases, and interface design. (Not good for the perceived coherence of your team.)
  • Duplication of code
  • No integration of systems – which means walled gardens.
  • Orphaned applications when the developer leaves the department.

This approach works for the small stuff, but once you step into the enterprise arena, its a different ballgame. Clearly to pull off this beast of a project, we will need to adopt a new development approach.

So off I went to do some research on proven methods of development. I quickly found that opinions on programming methodologies are like assholes as they say, everyone has one, and everyone else’s stinks. After reading overviews of several different methods, Extreme Programming was the only one that made me sit up in my chair. The other methods seemed to reverborate the spewing bullshit ramblings of my Systems Analysis and Design instructor.

An interesting requirement of Extreme Programming requires programmers to work in pairs. An interesting approach, especially since the programming language we will be using (Ruby on Rails) is one that two people on the team are experienced at, and the other two are beginners at.  This article on Wikipedia cites many benefits to doing so:

  • Design quality: Shorter programs, better designs, fewer bugs. Pairs typically consider more design alternatives than programmers working solo, and arrive at simpler, more-maintainable designs, as well as catch design defects very early.
  • Learning and training: Knowledge passes easily between pair programmers: they share knowledge of the specifics of the system, and they pick up programming techniques from each other as they work.
  • Overcoming difficult problems: Pairs often find that seemingly “impossible” problems become easy or even quick, or at least possible, to solve when they work together.
  • Improved morale: Programmers report greater joy in their work and greater confidence that their work is correct.
  • Decreased management risk: Since knowledge of the system is shared among programmers, there is less risk to management if one programmer leaves the team.
  • Increased discipline and better time management: Programmers are less likely to skip writing unit tests, spend time web-surfing or on personal email, or other violations of discipline, when they are working with a pair partner. The pair partner “keeps them honest”.
  • Fewer interruptions: People are more reluctant to interrupt a pair than they are to interrupt someone working alone.

If our department can adopt this method, colloborate effectively, and communicate politely, I think it will help us on this venture. It will be interesting to see how certain cowboy coders adjust to being part of a team effort.

Computers, Events, Linux, Open-source, Personal, Ruby, Software, Thoughts, Web, Windows

SWAN Manager v2.0

It has been a trying last few weeks, but I have finally rolled out the new SWAN Manager. It ended up being almost a total rewrite, and I walked away with a lesson learned:

  • Even on a total rewrite, I should have consistently been checking in my code at “checkpoints”. Instead, I waited until everything was working, then did one single massive checkin. Inevitably I missed stuff, forgot to cleanup stuff, and had to resolve a couple of SVN conflicts where it just didn’t understand what the hell happened.

I have highlighted a few of the more significant changes in screenshots below:

I based the new login screen off of Google Docs login. It shows at a glance what services are offered inside the Manager, and is a little more friendly than just a login box. Also, you can see the new tabbed interface at the top.
The User model underwent the most significant of changes. First, I decided that it was running way to slow, so I reimplemented the way it looks up the data from an indirect (and unreliable) method, to querying the sources directly. Also, the data is cached using memcached for even more speed enhancements.

An area I am particularly proud of is the display of the icons the user should see. I take each role name, and do a Net::HTTP fetch on them, checking for a 200 result, and displaying it. This is all handled in a helper.

The channel model underwent significant changes as well. It has always directly queried for the data on each request, instead of caching the results. In addition to caching and other performance tweaking – I now know a lot more about the channels themselves. The entire model operates as an “acts_as_tree” with parent, and children nodes to show the sections, and sub-sections of a channel. If you can edit a section, it shows up as a link.
The announcements controller has been completely reworked as well. Before the user didn’t have the ability to do things like send to a role, or choose delivery / expiration times, or a destination. Now the user gets to pick all of this (Population Selection with a parameter is shown). A message can be sent with just a few clicks. The announcement model uses “acts_as_state_machine“, a seemingly dead but very useful plugin. The announcement goes through several states with validation checking and routing automatically handled. I have to thank Matt for turning me onto the idea.

Here I have an image of the announcement wizard further down on the same page as the image above. The date selected is handled by a Rails plugin called “unobtrusive_date_picker“. It allows some cool tricks like keyboard arrow navigation, and the ability to define starting / ending date ranges, and minute increments in the select box.

Additionally, once the announcement is sent, rather than going to get a cup of coffee while it runs the process (sometimes 10+ minutes), it now backgrounds it in a separate rake task.

All in all, I think that this is a word of difference from the previous SWAN Manager. This is stable, fast , and easy enough now that I feel it is something the campus as a whole can use without concern.  And it will need to live up to it expectations as well, as we have a few departments already lined up to start using its functionality as soon as we give the green light.

After I pat myself on the back, I suppose its time to get back to working on all those pesky channels…