Office Bet

Before leaving on vacation, my friend and coworker at Kissmetrics, Brett, offered $5.oo to anyone who could correctly guess the number of emails that he’d have in his inbox upon returning ten days later. He would use Price-is-Right rules, so whoever guessed closest without going over would win. Most people on the team guessed around 100 emails, with a few cutthroat engineers betting one email over their friends.

To be safe, I bided my time, and in the end submitted the highest guess of 2,ooo emails. That would mean roughly nine emails an hour for the next ten days…

I immediately set to work, and it was with great satisfaction that I saw a smiling Brett return only a few minutes later to declare that his inbox had been flooded with +2,ooo emails. Looks like I win???

I may have cheated, but it was all in good fun, and better still, I didn’t need to write a line of code or spend any money to do it.

Spam your Friends!

My first thought was to schedule a recurring email. For this, I turned to NudgeMail, which my friend Brantley had helped build. NudgeMail is a free service that parses emails sent to their @nudgemail.com domain. They look for a subject line specifying when they should send you a reminder email, called a “nudge”. It’s a great service, but I was definitely trying to abuse it here. I tried specifying every5m and combinations thereof, but it appears that NudgeMail is hip to this sort of thing. Eventually, I settled for setting numerous hourly reminders instead.

One crucial consideration was to avoid sending emails that Brett could unsubscribe from, so I had the nudges sent to my work email, where a gmail filter I set up would forward them along to him. Brett can’t really block my work email, as I could potentially send him something important. While this looked promising, I couldn’t help but want something bigger.

me --> nudgemail

nudgemail -hourly-> my filter -hourly-> brett

I immediately thought back to the Citizen Developer mentality my good friend Sherif tried to instill in me, which emphasized prototyping rapidly without writing any code at all, primarily using SaaS to accomplish what would inevitably take longer to code from scratch anyway. One service he had introduced me to was Zapier. Zapier is like IFFT, and it allows you to create actions for particular triggers. It acts as glue between various APIs.

I setup a webhook-to-email zap, so that any hit to a given URL would send me an email, and then I’d let the gmail filter forward the email along to Brett.

visit URL --> triggers email --> my filter --> brett

The trick now was to trigger the endpoint en masse. To do so, I created a facebook note. Facebook notes allow you to include HTML, and crucially, they don’t cache images with distinct query-string parameters. This means that when facebook encounters the HTML for a remote image:

<img src="https://www.images.com/some-image.jpg" />

It visits the source to load the image. Normally, requests to the same resource would be cached to prevent making several requests to that resource, but the facebook scraper considers resources with different query-string params to be distinct, and it does not cache them. That means the following will trigger one request to the given resource for each image:

<img src="https://zapier.com/hooks/catch/.../?parameter=1" />
<img src="https://zapier.com/hooks/catch/.../?parameter=2" />
<img src="https://zapier.com/hooks/catch/.../?parameter=3" />
...

This is a known bug. Thus, everytime someone views the note, hundreds of requests would be sent against the Zapier webhook. Moreover, the unique parameter can be captured by Zapier and passed into the email, which makes each email unique, so gmail’s spam blocker ignores them.

View note --> load images -x100-> URL -x100-> email -x100-> brett

To draft the hundreds of “images”, I just used the following in irb:

0.upto(100).each do |num|
  p "<img src='https://zapier.com/hooks/.../?parameter=#{num}' />"
end

and relied on my text editor to strip the " from each line.

In the end, we ignored my emails and played the game legitimately, but this was still good for a laugh or two(thousand).

Your Files are Filed in Files

Folders in Unix are actually just files: binary files containing a list of the other files it contains (actually, filename-inode number pairs). If you type ls -l in a directory containing a folder and a file, you’ll end up with something like this:

...
drwxr-xr-x   4 myusername  mygroup   136 Mar 28 01:15 folder_name
-rw-r--r--   1 myusername  mygroup   290 Mar 27 21:56 file_name.html
...

The first group of letters is common to all files. They’re the access modes for the file, i.e. whether or not the file is readable, writeable, or executeable for a given user, group, or ‘other’. If you have read access to a file, you can see what’s in it. If you have write access, you can change it. If you have execute access, you can run it. Typical files, like the one shown above, usually start with a -. Directories, on the other hand, start with a d.

So what does read, write, and execute mean for a directory file? You probably use it all the time! Read access allows you to list the file names in the directory file, think ls. Write access let’s you delete files (or remove a file’s entry from a directory file). With execute access, you can cd into a directory; otherwise, you can’t.

So, now, while your coworkers may insist that they’re filing their files in folders, you’ll know what’s really up: they’re filing their files in files.

Start Your Day with Spotlight

Everyday I come into work, I do the same thing:

• I open three tabs in Chrome: PivotalTracker, Gmail, and Github
• I open five tabs in iTerm: one for mongoDB, one for zeus, one for my server, one for a rails console, and one for git
• I open my editor to the root of my rails app

It’s routine, so I wanted to automate it. To do so, I wrote a pretty simple shell script:

#!/bin/bash

# open tabs in Chrome
open -a Google\ Chrome http://www.gmail.com
open -a Google\ Chrome https://www.pivotaltracker.com/n/workspace
open -a Google\ Chrome https://github.com/darbysmart/rails

# open iTerm2 from default terminal
open -a iTerm

# method to open tabs in iTerm and execute some command
launch () {
/usr/bin/osascript <<-EOF
tell application "iTerm"
    make new terminal
    tell the current terminal
        activate current session
        launch session "Default Session"
        tell the last session
            write text "{$1}"
        end tell
    end tell
end tell
EOF
}

# open tabs and execute code in each tab
launch "vg;cd rails;mongod"
launch "vg;cd rails;zeus start"
launch "vg;cd rails;zeus s"
launch "vg;cd rails;zeus c"
launch "vg;cd rails;subl ."

The only problem was that in order to run this script I’d have to open my terminal and run the script from there. That’s not cool.

To solve this, I saved the file with a .command suffix and made the file executable:

$ chmod a+x good_morning.command

Note the above makes the file executable for all users/groups.

And now I use Spotlight on my Mac to find it and execute it! So, when I sit down in the morning, I can just type cmd+Space to open Spotlight, type good morning, hit enter, and my workstation is automatically set up for me!

Hacky, but awesome.

Creating Pipelines with State Machine

State machine is a gem that let’s you assign state to your Rails models, easily create transitions between states, and create callbacks that are triggered during transitions. It can make Rails programming feel a little javascripty, and it can be great for making ‘pipelines’.

To illustrate what I mean, imagine having an Order model on an e-commerce site. Our order model could have states like cart, complete, and cancelled. We could implement transitions like #cancel, which would allow the order to move from complete to cancelled, but not from cart to cancelled (unless we wanted it to). We could also set callbacks that could restock inventory of purchased items when an order was transitioned from complete to cancelled, or anything else for that matter.

Let’s dive in to state machine and create an email marketing pipeline. For this example, we’ll assume that we have a User model and that our user model has an attribute which we’ll call email_list, that holds a string designating where they are in the email marketing pipeline. I may also assume a few relations with a Subscription model or Order model just to keep things moving.

An Email Marketing Pipeline

In designing our email marketing pipeline, we want the following to occur:

When a user signs up at our site, they enter the pipeline. After 48 hours, if they’ve made a purchase we’ll segment them based on what they’ve purchased, otherwise we’ll move them to a t0_no_action bucket, where t0 represents an arbitrary time frame they are in. As for segmenting them, if they purchased our subscription service, we’ll place them in the subscriber email list, otherwise we’ll bucket them in the vip email list. After another 48 hours, we’ll check on our user again. If they hadn’t purchased before, but have in the meantime, we’ll rebucket them, again segmented on what they’ve purchased. Next, we’ll check to see if they added something to their cart. If they did, we’ll bucket them in the added_to_cart email list. Otherwise, if they indicated some level of interest in our subscription service, but didn’t complete the sign up, we’ll place them in the interested email list. Finally, if they didn’t do any of the above, we’ll move them to the t1_no_action email list. In our last step, we’ll check in on our user again, this time another 48 hours later. If our user has purchased, we’ll segment them based on their purchase just as before. If they still have something in their cart, they’ll be added to the still_shoppin email list. If they still haven’t subscribed, we’ll assign them to the discounted_subscription email list. If they hadn’t taken any action before and still haven’t, we’ll put them on an email list called t2_no_action. After that, we’ll continually check back in every 48 hours to see if they’ve purchased, so we can rebucket them and segment them if need be. I’ve illustrated the process below where words preceeded by a colon indicate an email list and words not preceeded by a colon indicate an action:

#                 ### Fantasy Marketing Pipeline ###

#        :unbucketed                                          Day 0
#         /         \
#       *purchase    no-action                                Day 1
#       /       \             \
# :subscriber   :vip       :t0_no_action
#                         /   |    |     \
#                 *purchase cart interest no-action           Day 3
#                           /      |              \
#              :added_to_cart  :interested     :t1_no_action
#              / \              /      \         /    \
#     *purchase  no-action no-action    *purchase  no-action  Day 5
#                 |            |                        |
#      :still_shoppin    :discounted_subscription  :t2_no_action

This rather complex pipeline is easily modeled with state machine. The first step is to tell state machine what field to store state in on your model. In this case, our user model has an email_list attribute. We could add this directly to the model, but adding all the email pipeline logic to the user model could really fatten up our user model, which is probably fat enough already. Because everything we’re doing is going to relate to email marketing, let’s pull all the functionality out into a concern. Now, we’ll start with this:

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
     # set a default email list
    state_machine :email_list, initial: :unbucketed
  end
end

Let’s include the concern in our user model, so the user receives the functionality.

app/models/user.rb

class User < ActiveRecord::Base
  include EmailMarketable
end

Now, our user will behave as if the following were true:

class User < ActiveRecord::Base
  state_machine :email_list, inital: :unbucketed
end

The next step is to tell state machine what states we’re going to be using. In this case, the states are going to correspond to email lists. We can set that up easily enough:

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
    state_machine :email_list, initial: :unbucketed do
      state :unbucketed,             # adding possible states
            :subscriber,
            :vip,
            :t0_no_action,
            :t1_no_action,
            :t2_no_action,
            :added_to_cart,
            :interested,
            :still_shoppin,
            :discounted_subscription
    end
  end
end

At this point, a user can have one of several states: unbucketed, subscriber, vip, t0_no_action, t1_no_action, t2_no_action, added_to_cart, interested, still_shoppin, or discounted_subscription. But, we have not yet declared a way to transition between states. Let’s add one.

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
    state_machine :email_list, initial: :unbucketed do
      state :unbucketed,
            :subscriber,
            :vip,
            :t0_no_action,
            :t1_no_action,
            :t2_no_action,
            :added_to_cart,
            :interested,
            :still_shoppin,
            :discounted_subscription

      event :took_no_action do    # define a transition
        transition interested:    :discounted_subscription
        transition added_to_cart: :still_shoppin
        transition t1_no_action:  :t2_no_action
        transition t0_no_action:  :t1_no_action
        transition unbucketed:    :t0_no_action
      end
    end
  end
end

Here we’ve defined a transition from one set of states to another. If we had a user, and we called #took_no_action on that user, his state will change to the corresponding state. For example, a user with a current state of t0_no_action will change state to t1_no_action when we call user.took_no_action.

> user = User.create(email_list: :added_to_cart)
> user.took_no_action
> user.email_list
=> :still_shoppin

We have a few special cases not covered above, like when a user transitions to added_to_cart from t0_no_action, or when a user transitions to interested from t0_no_action. Let’s make special cases for each of those.

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
    state_machine :email_list, initial: :unbucketed do
      state :unbucketed,
            :subscriber,
            :vip,
            :t0_no_action,
            :t1_no_action,
            :t2_no_action,
            :added_to_cart,
            :interested,
            :still_shoppin,
            :discounted_subscription

      event :took_no_action do
        transition interested:      :discounted_subscription
        transition added_to_cart:   :still_shoppin
        transition t1_no_action:    :t2_no_action
        transition t0_no_action:    :t1_no_action
        transition unbucketed:      :t0_no_action
      end

      # handle special cases
      event :added_item_to_cart do
        transition t0_no_action: :added_to_cart
      end

      # handle special cases
      event :interested_in_subscribing do
        transition t0_no_action: :interested
      end
    end
  end
end

That handles all cases except for when we want to segment users that have purchased. Thankfully, state machine allows us to add conditionals in transitions. Let’s whip something up for detecting which segment the user belongs in.

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
    state_machine :email_list, initial: :unbucketed do
      state :unbucketed,
            :subscriber,
            :vip,
            :t0_no_action,
            :t1_no_action,
            :t2_no_action,
            :added_to_cart,
            :interested,
            :still_shoppin,
            :discounted_subscription

      event :took_no_action do
        transition interested:      :discounted_subscription
        transition added_to_cart:   :still_shoppin
        transition t1_no_action:    :t2_no_action
        transition t0_no_action:    :t1_no_action
        transition unbucketed:      :t0_no_action
      end

      event :added_item_to_cart do
        transition t0_no_action: :added_to_cart
      end

      event :interested_in_subscribing do
        transition t0_no_action: :interested
      end

      event :made_a_purchase do    # conditional transitions
        transition all => :subscriber, if: :purchased_subscription?
        transition all => :vip,    unless: :purchased_subscription?
      end
    end
  end

  # method to be used in conditional above
  def purchased_subscription?
    subscriptions.any? # assumes a relationship on user model
  end
end

That wraps up all of our state and transitions; now, let’s add a few callbacks. Ideally, we will limit the number of callbacks to limit the complexity of this state machine, but a few won’t hurt.

app/models/concerns/email_marketable.rb

require 'active_support/concern'

module EmailMarketable
  extend ActiveSupport::Concern

  included do
    state_machine :email_list, initial: :unbucketed do
      state :unbucketed,
            :subscriber,
            :vip,
            :t0_no_action,
            :t1_no_action,
            :t2_no_action,
            :added_to_cart,
            :interested,
            :still_shoppin,
            :discounted_subscription

      event :took_no_action do
        transition interested:      :discounted_subscription
        transition added_to_cart:   :still_shoppin
        transition t1_no_action:    :t2_no_action
        transition t0_no_action:    :t1_no_action
        transition unbucketed:      :t0_no_action
      end

      event :added_item_to_cart do
        transition t0_no_action: :added_to_cart
      end

      event :interested_in_subscribing do
        transition t0_no_action: :interested
      end

      event :made_a_purchase do
        transition all => :subscriber, if: :purchased_subscription?
        transition all => :vip,    unless: :purchased_subscription?
      end

      # callbacks
      before_transition any => any, do: :email_list_unsubscribe
      after_transition  any => any, do: :email_list_subscribe
    end
  end

  def purchased_subscription?
    subscriptions.any?
  end

   # callback triggers this method
  def email_list_subscribe
    Resque.enqueue(SubscribeUserToList, id, email_list)
  end

  # callback triggers this method
  def email_list_unsubscribe
    Resque.enqueue(UnsubscribeUserFromList, id, email_list)
  end
end

Here, we’re not too concerned with what our callbacks are doing, so I obscure the functionality by putting them in a background job, which would presumably make some API call to some third-party service. Whatever it does, it’s not important. What is important is understanding that the email_list_unsubscribe callback will fire before any transition from any one state to any another. Same with the email_list_subscribe callback. We could also specify a callback to be called when we transition between a specific state to any other specific state as well, but this suffices for this example.

Our state machine is ready to go! Let’s whip up a quick rake task to demonstrate how transitions could be called on our user object.

lib/tasks/assign_email_list.rake

desc "Updates a user's email list"

task assign_email_list: :environment do
  updateable_users = User.where.not(email_list: [:subscriber, :vip])
  updateable_users.each do |user|
    if user.orders.any?
      user.made_a_purchase
    elsif user.subscriptions.any?(&:incomplete?)
      user.interested_in_subscribing
    elsif user.cart.any?
      user.added_item_to_cart
    else
      user.took_no_action
    end
  end
end

This scheduled task simply runs every other day, pulls users that haven’t reached the highest priority email list (users that could be rebucketed into another list), checks which action the user should receive base on our criteria, and simply calls the state machine transition on the user, letting state machine handle all the rest.

The more I work with the state machine pattern, the more I enjoy it. I find that it can greatly simplify my backend rails code and that it is excellent for creating pipelines. The next time you need to create some sort of pipeline, give state machine a shot, and let me know what you think of it.

Cheers

Well-Tested Ruby - Stubs

Let’s build off our previous discussion of cancelling an order. Currently, #cancel will update the status attribute of our order object to 'cancelled'. Here is the code we ended with:

app/models/order.rb

class Order < ActiveRecord::Base

  def cancel
    update_attributes(state: 'cancelled')
  end
end

spec/models/order_spec.rb

require 'spec_helper'

describe Order, '#cancel' do
  it 'changes the order status to cancelled' do
    order = Order.new(status: 'complete')

    order.cancel

    expect(order.status).to eq 'cancelled'
  end
end

Before we get started, let’s replace our constructor with a factory; we’ll use the popular factory_girl library.

Now our code looks like this:

spec/models/order_spec.rb

require 'spec_helper'

describe Order, '#cancel' do
  it 'changes the order status to cancelled' do
    order = build(:order, status: 'complete') # factory

    order.cancel

    expect(order.status).to eq 'cancelled'
  end
end

And we have a factory file where we define our factories:

spec/factories.rb

FactoryGirl.define do
  factory :user
end

Factory girl is simple, it allows you to define a factory which is basically a model, assign default attributes to model objects you plan on using in your tests, and gives you a simple syntax for creating objects.

Now, onto business.

Let’s introduce another model: inventory units. Now, we’re tasked with #cancel having to restock the inventory units associated with our order. For simplicity’s sake, let’s assume an order can only be for one type of item, and that our inventory units represent the items being purchased, i.e. an item being purchased does not consist of components that are represented as inventory units. Bascially, an order consists of one or more of the same inventory unit. Let’s start with a unit test on the inventory unit model.

spec/factories.rb

FactoryGirl.define do
  factory :user
  factory :inventory_unit, class: Inventory::Unit
end

spec/models/inventory/unit_spec.rb

require 'spec_helper'

describe Inventory::Unit, '#restock' do
  it 'increments the inventory quantity on hand' do
    unit = build(:inventory_unit, on_hand: 10)

    unit.restock(20)

    expect(unit.on_hand).to eq 30
  end
end

The tests will drive the code:

app/models/inventory/unit.rb

class Inventory::Unit < ActiveRecord::Base
  def restock(quantity)
    update_attributes(on_hand: on_hand + quantity)
  end
end

There may be some setup involved if you want to namespace the inventory unit, and have it play nice with active record, but that’s for another time. Let’s also check that our quantity is always a positive number.

spec/models/inventory/unit_spec.rb

require 'spec_helper'

describe Inventory::Unit, '#restock' do
  it 'increments the inventory quantity on hand' do
    unit = build(:inventory_unit, on_hand: 10)

    unit.restock(20)

    expect(unit.on_hand).to eq 30
  end

  it 'will not deduct inventory' do # check for positive input
    unit = build(:inventory_unit, on_hand: 10)

    unit.restock(-5)

    expect(unit.on_hand).to eq 10
  end
end

Our resulting code:

app/models/inventory/unit.rb

class Inventory::Unit < ActiveRecord::Base
  def restock(quantity)
    update_attributes(on_hand: on_hand + quantity) if quantity > 0
  end
end

Great, now our unit tests are in order, but if our order model is going to communicate to our inventory unit model, they should probably be related. Because orders can only be for one inventory unit in this example, and inventory units will therefore have many orders/purchases, we could have a situation like this:

app/models/inventory/unit.rb

class Inventory::Unit < ActiveRecord::Base
  has_many :orders, class_name: '::Order'
end

app/models/order.rb

class Order < ActiveRecord::Base
  belongs_to :inventory_unit, class_name: 'Inventory::Unit'
end

But, do we really need to load the relation both ways? Not really. Right now, only our order model needs to communicate to our inventory unit. So, let’s not load both associations, and let’s only make the association one way. To test it, we can use the popular shoulda-matchers from thoughtbot.

spec/models/order_spec.rb

require 'spec_helper'

# here '{ }' replaces 'do end', the magic 'subject' replaces
# 'Order.new', and the it 'description' is ommitted
describe Order, 'associations' do
  it { expect(subject).to belong_to(:inventory_units) }
end

describe Order, '#cancel' do
  it 'changes the order status to cancelled' do
    order = build(:order, status: 'complete')

    order.cancel

    expect(order.status).to eq 'cancelled'
  end
end

And the code:

app/models/order.rb

class Order < ActiveRecord::Base
  belongs_to :inventory_unit, class_name: 'Inventory::Unit'

  def cancel
    update_attributes(state: 'cancelled')
  end
end

Now, we can restock the inventory when #cancel is called. And here is where we use a stub. A stub is used to control the effects of a method call. If we stub a method #bark on a Dog object, we can override the real effects of #bark and return anything we want. We can also tell if our Dog object ever had #bark called on it. For example:

dog = build(:dog)
dog.stub(:bark) # stub bark so nothing happens when it's called
dog.stub(:bark).and_return('quack') # override bark
dog.stub(:bark).and_yield('moo') # yield to a block

dog.bark

expect(dog).to have_received(:bark) # => true

With respect to our order model, we can stub the restocking communication sent to the inventory unit model like so:

spec/models/order_spec.rb

require 'spec_helper'

describe Order, 'associations' do
  it { expect(subject).to belong_to(:inventory_units) }
end

describe Order, '#cancel' do
  it 'changes the order status to cancelled' do
    order = build(:order, status: 'complete')

    order.cancel

    expect(order.status).to eq 'cancelled'
  end

  it 'restocks the inventory associated with the order' do
    order = build(:order, quantity: 5)
    order.inventory_unit.build
    order.inventory_unit.stub(:restock) # le stub

    order.cancel

    expect(order.inventory_unit).to have_received(:restock).with(5)
  end
end

Notice how the preparation section of our unit test is getting bigger? That’s no good. One thing we can do to refactor is pull out shared factories and put them in a let block.

The let block lets us define a variable and set its value:

let(:order) { build(:order, status: :complete) }
# order = Order.new(status: :complete)
let(:user) { create(:user, email: 'boom@pop.com'}
# user = User.create(email: 'boom@pop.com')

So now, our code looks like this:

spec/models/order_spec.rb

require 'spec_helper'

describe Order, 'associations' do
  it { expect(subject).to belong_to(:inventory_units) }
end

describe Order, '#cancel' do
  let(:order) { build(:order, status: 'complete', quantity: 5) }

  it 'changes the order status to cancelled' do
    order.cancel

    expect(order.status).to eq 'cancelled'
  end

  it 'restocks the inventory associated with the order' do
    order.inventory_unit.build
    order.inventory_unit.stub(:restock)

    order.cancel

    expect(order.inventory_unit).to have_received(:restock).with(5)
  end
end

Now, let’s drive the code:

class Order < ActiveRecord::Base
  belongs_to :inventory_unit, class_name: 'Inventory::Unit'

  def cancel
    inventory_unit.restock(quantity)
    update_attributes(state: 'cancelled')
  end
end

The reason we write it this way is because we’re only concerned about testing the model, we’re not interested in testing the inventory unit. We’ll let the inventory unit’s unit tests ensure that the methods are doing what they should. Notice we didn’t write code like this:

class Order < ActiveRecord::Base
  belongs_to :inventory_unit, class_name: 'Inventory::Unit'

  def cancel
    inventory_unit.update_attributes(
      on_hand: inventory_unit.on_hand + quantity
    )
    update_attributes(state: 'cancelled')
  end
end

With this code, the order model knows too much about the internals of the inventory unit model. By stubbing it, we ensure that the communication between the models has occurred, but neither model knows too much about the internals of the other.

Hopefully, this elucidates the purpose of stubs as well as providing a reasonable example. In practice, your model shouldn’t know about the internals of other models. If you’re testing a model with stubs, you can ensure communications occur between models and be certain that models aren’t aware of each others internals.

Mocks to come.